PR - Google Page Rank how it exactly works

Page Rank Theory :- It is the method of google to determine the relevancy and importance of the page. And it ranges from 0 to 10. 0 is the lowest page rank and 10 is the highest. It is based on incoming links.
[Perfect search engine optimization]

Page rank equation to calculate PR

PR(A) = (1-d) + d(PR(t1)/C(t1) + … + PR(tn)/C(tn))

How page rank works ?
“That’s the equation that calculates a page’s PageRank. In the equation ‘t1 – tn’ are pages linking to page A, ‘C’ is the number of outbound links that a page has and ‘d’ is a damping factor, usually set to 0.85.”

Some things to remember about page rank

1/ Not all links weight the same

Example :- If you had a web page with a PR8 and had 1 link on it, the site linked to would get a fair amount of PR value. But, if you had 100 links on that page, each individual link would only get a fraction of the value.

2/ Content is not taken into account when PageRank is calculated. 

3/ PageRank does not rank web sites as a whole, but is determined for each page individually.

4/ Each inbound link is important to the overall total. Except banned sites, which don’t count. 

5/ PageRank values don’t range from 0 to 10. PageRank is a floating-point number. “It’s more accurate to think of it as a floating-point number. 

6/ “Google tries to find pages that are both reputable and relevant. If two pages appear to have roughly the same amount of information matching a given query, we’ll usually try to pick the page that more trusted websites have chosen to link to.

7/ Anchor text is important. The more specific is the reference, the better Google can evaluate it and consider it in relates search queries. 

8/ Headers (h1, … ,h6), strong tags and semantic contentare important. (Update: But it doesn’t improve PageRank.) “Place it in the description and meta tags, place it in bold/strong tags, but keep your content readable and useful. Be aware of the text surrounding your keywords, search engines will become more semantic in the coming years so context is important.”

Some facts about google page rank :-

1. Frequent content updates don’t improve Page Rank automatically. Content is not part of the PR calculation.
2. High Page Rank doesn’t mean high search ranking.
3. DMOZ and Yahoo! Listings don’t improve Page Rank automatically.
4. .edu and .gov-sites don’t improve Page Rank automatically.
5. Google considers site age, backlink relevancy and backlink duration. PageRank doesn't. If backlink isn't relevant, it won't weight much.
6. Sub-directories don’t necessarily have a lower Page Rank than root-directories.
7. Wikipedia links don’t improve PageRank automatically (update: but pages which extract information from Wikipedia might improve PageRank).
8. Meta-Tags don’t improve PageRank. 
9. Links marked with nofollow-attribute don’t contribute to Google PageRank.
10. Efficient internal onsite linking has an impact on PageRank.
11. Related high ranked web-sites count stronger. But: “a page with high PageRank may actually pass you less if it has more links, because it’s spread too thin.”
12. Links from and to high quality related sites have an impact on Page Rank.
13. Multiple votes to one link from the same page cost as much as a single vote.
14. Bad incoming links don’t have impact on Page Rank.
15. Crawlers don’t analyze web-sites permanently. “It often takes two full monthly updates for all of your incoming links to be discovered, counted, calculated and displayed as backlinks.”
16. Links don’t give PR away, they are votes. “When a page votes its PageRank value to other pages, its own PageRank is not reduced by the value that it is voting.
17. Links from one page to itself don’t improve Page Rank.
18. Adding new pages can decrease Page Rank. 
19. Each inbound link is important to the overall total. Except banned sites.
20. Links from and to high quality related sites are important.:- “The more closely related the pages, the higher the PageRank amount transferred.” “Linking to high quality sites shows the search engines your site is very useful to your visitors. Unless your site has been around for years and is well established and trusted by Google, this factor will have an adverse effect on your site’s overall ranking. Linking only to high quality content sites will give your site an edge over your competition.”
21. Incoming Links from popular sites are important.
22. Site can be banned if it links to banned sites. “Be extremely careful of any out-going links from your site. Don’t link to bad neighborhoods (link farms, banned sites, etc.) Google will penalize you for bad links so always check the PageRank of the sites you’re linking to from your site.”
23. Illegal activities will penalize your PageRank and possibly ban your site from Google. “Hidden text, deceptive redirects, cloaking, automated link exchanges, or anything else against Google’s quality guidelines” can ban your site from Google.
24. Related high ranked web-sites count stronger (or don’t they?). “One-way inbound links from websites with topics that are related to your website’s topic will help you gain a higher Page Rank.” Other one-way inbound links from pages with high page rank but unrelated topics do help a little, but not nearly as much.  
25. Dangling links don’t have impact on Page Rank. “Dangling links are simply links that point to any page with no outgoing links.     

Google page rank tools

• Webmastereyes, Visual PageRank View
  The results will show the page given along with the PageRank of each link on that page. You also have the option to show “nofollow” and external links.
• Google PageRank Prediction
  The tool analyzes the popularity of a given web-site and tries to predict its future Google PageRank. More Page Rank Tools.
• Google Ranking Factors
  Alleged positive and negative SEO Google Ranking Factors

Google helpful services:-

• Google Quality Guidelines
  These quality guidelines cover the most common forms of deceptive or manipulative behavior, but Google may respond negatively to other misleading practices not listed here (e.g. tricking users by registering misspellings of well-known websites).
• Check if your site is in Google database
• Reinclusion request form
  Request reinclusion of a site that has violated the webmaster guidelines
• Google Tools
  A comprehensive overview on Dmoz.org.

Read More

Social bookmarking A to Z

Why do we need social bookmarking?

It is in fact one of the off-page search engine optimisation methods as it provides you with backlinks or inbound links to your site. Try to do dofollow social bookmarking submission because search engine spiders scour the sites and follow the links. It will also increase your page rank.
Social bookmarking not only helps in building in backlinks but also helps to increase traffic brought traffic in to your website. This will help increase your sales or website conversions and decrease your cost per acquisition.

Social bookmarking also helps you in branding as long as you choose your niche properly and start building keywords and tags from there.
People do social bookmarking for link juice and content syndication.
Link juice give us benefit of 0.49%PR of per website.

What is social bookmarking :- it is a method for internet user to share organized, searched and managed bookmarking resource / web resource

Steps of doing Social bookmarking:-

1/ Open bookmarking website and register / join / sign up there.

2/ Verify account by clicking confirmation link in your email id.

3/ Submit URl

4/ Submit story

5/ Fill category, title, description, tags etc.

6/ After submission copy and save backlink of the story.

Steps / tips that can help :-

1/ Pay attention to your Headlines :- catchy
2/ Write a meaningful & short description :- never use false fact but draws more attention
3/ Have a great first paragraph
4/ Content is king :- dont post junks
5/ Make it easy for others to vote / bookmark your site :- best when other user bookmark your site. it is helpful if you put bookmarking button at the end of content.
6/ Know when to submit :- On most social bookmarking sites you have only 24 hours to get to the front page and stay there. To get more traffic post around 3 p.m. US Central Time.
7/ Submit to the right category
8/ Build a top-profile
9/ Cooperate with other social bookmarkers :- 50 votes can get you to the top page of Digg.
10/ Submit in English
11/ Never submit old news
12/ Check your facts
13/ Check you spelling
14/ Not all topics do well
15/ Have Related Articles / Popular Articles
16/ RSS feeds, newsletter subscriptions, affiliate marketing :- these are all areas in which the traffic from social bookmarking sites can help you a lot.
17/ Do not use automated submitters
18/ Respond to comments on your stories
19/ Prepare your server for the expected traffic :- server crashed  due to exceeded bandwidth problem. 
20/ Submit approximately 25-30 bookmarking per domain per day.
21/ Add relative friend and same business
22/ Target community

Bookmarking submission report sample :-
Project name, URL, S.no., Bookmarking sitename, PR, Category, Keyword, Email id, Name, Backlink

Autofill form is also helpful here to submit forms. (how to use and setup autofill form) 

All working High PR dofollow social bookmarking list

http://baagdi.wordpress.com/2009/11/01/high-pr-bookmarking-sites/

Examples:-

diigo.com, mister-wong, delicious.com are not a do-follow social bookmarking websites.
DO follow sites :- technorati.com, digg.com, reddit.com, folkd.com, spurl.net, a1-webmarks.com, feedmarker.com

Some important social bookmarking tool websites:-
http://socialposter.com/
http://www.ping.fm
http://www.onlywire.com
http://www.socialmarker.com/
http://socialadr.com/

Read More

SEO vs SEM

SEO and SEM

Search engine marketing (SEM) refers to the short text advertisements which generally appear at the top or down the right hand side of search engine results. These are often labelled as "sponsored Links" or "sponsored results". These advertisements are generally purchased from Google^TM (the Google Adwords^TM product), or Yahoo! Search Marketing. SEM is broader than SEO. SEM as more expensive and quite possibly more targeted.

Ex:- viral marketing (email marketing), Paid inclusion (paying a search engine or a directory to add a site to its database immediately), traditional Ads(placing paid advertising on the search engine result pages), PPC (Pay per click,), Digital Marketing is the promoting of brands using all forms of digital advertising channels to reach consumers. This now includes Television, Radio, Internet, mobile, social media marketing and any other form of digital media, Search Engine Image Protection (SEIP), also known as search engine reputation management, combines the expertise of SEM and PR, and is designed to protect a company or brand against incorrect negative publicity via the Internet. Search Engine Reputation Management (or SERM),

A keyword cloud is a search engine marketing (SEM) term that refers to a group of keywords that are relevant to a specific website.

Search Engine Optimisation is a longer term approach, and encompases many 'best practice' web design standards. There are often no payments required to the search engines, as the major search engine spiders (robots) visit your site for free (i.e. Google^TM, and Yahoo!^TM), add it to their index, and rank it for relevancy against mathematical parameters. SEO is the act of optimising the HTML and other content of your website for relevant, targeted key phrases in order to attain higher natural listings than competing websites.

Read More

SEO content

Dynamic content is web content such as Search Engine Results Pages (SERPS) that are generated or changed based on database information or user activity. Web pages that remain the same for all visitors in every context contain "static content." Many e-commerce sites create dynamic content based on purchase history and other factors. Search engines have a difficult time indexing dynamic content if the page includes a session ID number, and will typically ignore URLs that contain the variable "?".Search engines will punish sites that use deceptive or invasive means to create dynamic content.
Length of On-Page Content : - you should have 250 words (or more) of actual inline content – i.e. not headers, not sidebar content.
When a page has 1,000+ plus, it has a much better chance of ranking for the keywords that it’s optimized for.

Scanable Line Paragraphs :- keeping paragraphs within 3-4 lines (5 lines max), you can create a text-experience that users can easily scan.
structuring your content:- Basically, you should section off your content using header tags (e.g. <h2> and <h3>). This will create a break in the content that (1) makes it seem easier (i.e. ‘not as long’) to read, and (2) provides visual cues to pull the user’s eye through the content.
Bullet Lists:- search engine regard them as “broken content”.
(1) you have 2 sentences/3 lines preceding the bullet list,
(2) you have another 2 sentences/3 lines following it, and
(3) there is a minimum of 250 words of “unbroken” content on the page
5 formula to rank on top of search engines :- 1/ Contribute content to other websites
2/ Write your own awesome content
3/ Contribute Freebies
4/ Help Others (yahoo answers)
5/ And then make sure you get backlinks from other sites with those keywords as anchor text.

Read More

Types of SEO

Different types of SEO :- ON page & Off page

And both of these includes :-
1/ White hat seo

2/ Black hat seo

3/ Gray hat seo

White Hat SEO :- Also known as natural search engine optimization or organic search engine optimization, ethical seo, white hat SEO is the legitimate use of keyword-focused copy and tags, Crawler-friendly site architecture, Search Engine Submissions and a quality Backlinks network to improve a site's Position, Page Rank and Click-Through Rate. White hat SEO does not involve the use of Cloaking, Spam or any other Black Hat SEO techniques.

Black Hat SEO :- Black hat SEO is the term used for unethical or deceptive optimization techniques. This includes Spam, Cloaking, or violating search engine rules in any way. If a search engine discovers a site engaging in black hat SEO it will remove that site from its Index.
Banned / Blacklisted / Delisted :- Also known as delisted or blacklisted, a banned site is a URL that has been removed from a search engine's Index, typically for engaging in Black Hat SEO. Banned sites are ignored by search engines.

Grey hat seo :- Gray hat SEO refers to Search Engine Optimization strategies that fall in between Black Hat SEO and White Hat SEO. Gray hat SEO techniques can be legitimate in some cases and illegitimate in others. Such techniques include Doorway Pages, Gateway Pages, Cloaking and duplicate content. It is a soft form of Black hat seo or we can say that it is tricky part of seo.

Spam
Spam refers to any and all Search Engine Marketing techniques that violate search engine guidelines or attempt to gain increased Rank for a site using content that is irrelevant, deceptive or of little value to users. Types of spam include Hidden Text, content that contains nonsensical Keyword repetition, deceptive Cloaking or numerous Doorway Pages that redirect users to the same Landing Page. If a search engine detects spamming the offending site will be Blacklisted or lose Position. Spam is a Black Hat SEO technique.

Ex:- Cloaking (Also known as “stealth”, cloaking is a technique that involves serving one set of information to known search engine spiders while displaying a different set of information on documents viewed by clients.), Keyword Stuffing, Hidden Text, Useless Meta Tags, Misuse of Directories, Hidden Tags, Email Spam, Redirect Spam, Misuse of Web 2.0 Formats (ie Wiki, social networking and social tagging)

Spamdexing, which is a word derived from “spam” and “indexing,” refers to the practice of search engine spamming.

Ex:- Mirror websites are simply sites that all have the same content but use a different URL, URL redirection, cloaking, Google bombing / Googlewashing (Making a page rank well for a specific search query by having multiple links point at it with the keywords in the anchor text.)

Read More

Complete On page seo points

COMPLETE ON PAGE SEO

• Don’t use <b> </b> Bold, &nbsp; and <br/> tag in web pages.
• For best performance make websites in smarty.
• Make URLs SEO friendly for dynamic websites (use .htaccess file for this). 
• Don't use session IDs for information that you'd like to be indexed by spiders. 
  
• URL Rewriting is done to make URL SEO friendly (you can remove .html / .php extension through .htaccess file but server "mod_rewrite" should be enabled).
• Background repeat images should be of 1px.
• Use images and logos in bg, repeat transparent images of 1px width and height on bg image to make clickable and to restrict download bg images.
• If requirement of flash in websites them convert it into gif images. This is only for small flash.
• Always try to use flash and submit form in last of the page because after submit tab content is not crawl by the search engine.
• Very long pages are also not good split these type of pages in more than one page.
• Always use text instead of flash or images to display keyword, content and links because google cant crawl text under images if you must use images than consider using "ALT" attribute with it but it should be descriptive and accurate don't do keyword stuffing here.
• Title should be of 66 characters.
• Description should be of 256 characters
• Keyword meta tag should be 260 characters but not very necessary tag, 10 keywords and all should be in body text. Maximum 20 keywords. It gets no attention from google but matters in bing and yahoo.
• Every page should have at least 250 words content.
• Try to make paragraphs with 3-5 lines. It is easily scanable.
• Avoid content from other websites and also avoid using rss updates form other web pages like twitter updates in sites. (Google panda changes)
• RSS feeds are very important when your website have unique content and updated on regular basis. Please make sure that content should unique and not used for article submissions. Frequently change of content is favourable but it is not great when you do small changes in existing content.
• Link your keywords internal and external web pages and blogs, please do not add more than 4 links in 300 words content, This will create a strong internal and external links for all blogs.
• Fewer the outbound links more beneficial for website.
• Use at least 3 text links to other inner pages at each page.
• The text that is immediately before and after the anchor text also matters because it further indicates the relevance of the link .
  
• Check all broken links and fix them.
• Never link take or give links from banned websites.
• Always exchange links within same category of websites.
• Use full path for inner pages link this will crawl many times and the page rank of home page will automatically jump to ‘1’.
• To jump 1 more PR regularly change text content or keep updating article.
• Use H1 and H2 tag.
• Use Title for each Hyperlink.
• Each paragraph content also should have title in <p> tag.
• Images and flash should not have size exceed than 50 kb.
• Page size should not be more than 25 kb (use compression in heavy pages).
• Make websites in <div> tag because if the websites is in <table> than content out of monitor screen is sometimes not crawled by the robots.
• Images should have ‘alt’ attributes.
• Place the same content from the Javascript in a no script tag. Placing different text in JavaScript than in a noscript tag violates webmaster guidelines.
• Title of content and links, alt tag, images name and paths or folder name (try to use keywords on these places also)
• Avoid hidden links and single pixel links.
• Don't use cloaking or javascript redirects and doorway pages
• If a 301 redirect or 302 redirect not applied properly it hurts a lot or it can be taken as blackhat trick if user redirect immediately to different page.
• Cloaking :- Serving a page of HTML text to search engines, while showing a page of images or Flash to users. And Serving different content to search engines than to users.
• When Googlebot indexes a page containing JavaScript, it will index that page but it cannot follow or index any links hidden in the JavaScript itself.
• Doorway pages are typically large sets of poor-quality pages where each page is optimized for a specific keyword or phrase. In many cases, doorway pages are written to rank for a particular phrase and then funnel users to a single destination.
• Use Original content for webpages.
• To check originality of content visit copyscape.com
• For error check of XHTML and CSS2 visit Validator.w3.org
• Use meta robots text .
• Use external CSS to make page easily loadable.
• Try to call javascript file in the last of the page.
• Use favicon make it with 16X16 px with .ico plugin of photoshop then it is visible to every web browsers.
• After making website always check for xhtml and css2  errors http://validator.w3.org/ and http://jigsaw.w3.org/css-validator/
• Try to make websites browser compatible
• Regularly monitor performance of the website using page speed, Web page test, Y SLow and many other tools.
• Always use XML sitemap. (http://www.xml-sitemaps.com)
• Use google webmaster verification code in meta tag after verifying websites.
• Use meta tag rating, copyright, robots, googlebots, revisit after and meta abstract.
• To verify website on yahoo sitesearch.
• To verify website on MSN webmaster.
• To verify on Alexa just get into personal account after login.
• Use RSS in websites
• Use Google analytics code.
• Use Google Adsense.
• Use “add this” share button to share on social networking websites.
• To invite Google after any changes in site use Google webmaster tool.
• If possible use keyword enabled domain because Google caffeine give preference to domain containing keyword.
• Put nofollow in URL of any other site in our webpage.
• Always use small caps in tags.
• Use different title and description for all pages.
• Use image expire date.
• URL should not be “canonicalization” for example http://www.example.com/, http://example.com/
• Use very less nested table because it make the page slow to load. Browser has to find the end of the table before it can display the entire page.
• Primary keyword should appear in <meta> and title tag.
• Don’t use a keyword more than 3times in a single page and 1time for every 5 links.
• Javascript file are also under 50kb.
• “:” remove this character from <meta> tag because this can act as delimiters or stab characters sometime.
• Spider food = hyperlink and Content.
• Deep Crawel = All hyperlink are craweled.
• Shallow Crawel = only index or home pages and few pages are craweled.
• Keyword stuffing is to put unnecessary many keywords illegally in <img> tag and with flash. (This is part of black hat seo)
• Black hat seo tricks can put site in spam.
• Search engine do not read content with script tag.
• Using <noframe> tag with frame tag means you are fixing it to become readable by search engines. But avoid using frames because it is very very bad for seo.
• Keyword density in body text 2-5%.
• Keywords are one to six words in length.
• Use hyphens "-" in link names or URLs.
• Google like sites whose content changes over time.
• Use URL Length somewhat less than the 2000 characters allowed by IE.
• Less than 1000 is good and less than 100 is better.
• Newer pages on an older site will get faster recognition.
• Redirect three refresh <meta> tags, don’t immediately send your visitor to another page.
• Image map links and JavaScript links are problematic.
• Domain expiration date register for 5 years, Google knows you are serious. Register for 1 year is it a throw away domain.
• File location on server and website theme also matters.
• Having a separate domain or folder is better instead of subdomain.
• First Come First Serve Search Engine robots will follow the first link they find to any particular page. They won’t follow additional links to the same page.
• Google has limited the number of URLs in its sitemaps to 50000 URLs and 1000 sitemaps.
• Use best search friendly keyword phrase.
• Keyword proximity is applicable for keyword phrases that consist of 2 or more words. Use keywords one after other not like one word in first paragraph and second word in third it counts less.
• Use 4 keywords in page title in between “|”, 7 words at most.
• Use GZIP for html and CSS compression it reduces page loading time.
  o   a/gzip for higher compression rate
  o   b/deflate (apache) it is a better choice if your site have 1000 unique visitors because it requires much less energy to compress a file.
• Increase load speed of webpage http://on.fb.me/foF4gX 

Read More

Directory submission tips & Dofollow link list

Why do we need Directory submission :-

Directories help you to increase traffic on your web site! and rank your keyword or high position in search engine & generate backlink.

Usually a site which is submitted to a directory (free directory) takes up to 15-20 days for link accept or rejection.

What is directory :-  collection of categories
Benefits of directories :- CTT (Click through traffic)

* 4 types of directories (regional, theme base, general, major)
Major directory = provided by search engine. Never submit keyword on these directories.
Eg. :- DMOZ, yahoo directories
Benefits of these type of directories are deep category submission, Multi lingual, easily indexing, quality base link or permanent link.

General directory
Benefits of these type of directories are Indexing, Ranking, Backlink,  Traffic, Popularity, PR.
Disadvantage :- in-relative category submission, bulk submission, lacking of quality based link.

Theme base directory = Category based directories related to website.
Benefit of these type directories are we can get more traffic and popularity, quality based index, these are fast and stable.

Regional directories = Country based and domain based.
Benefit of these type of directories are that we can easily promote in specific geographical area.

Directory submission Steps :- 

1/ Open directories websites for for submission

2/ Three radio options will be given in directory website

     a. Featured  b. Regular  c. Reciprocal

3/ Select the free regular option

4/ Choose the category -> Sub category (to choose category fast use google toolbar and write all related names in search box and highlight them)

5/ URL of the website (try to submit only domain not deep url, because most of the directories not accept inner pages url). Ex. http://www.baagdi.com

6/ Title :- Your keyword which you want to promote

7/ Description :- meta description

8/ Keywords :- meta keyword

9/ Name :- webmaster name

10/ Email :- your email id
11/ Submit url / suggest url / add url / submit link

Directory submission report sample :-
Project name, URL, S.no., Directories, Category, Keyword, Email id, Name

You can get all the above detail of any website by checking page source code Ctrl+U.

In most of the directory website you no need to log in.

This can also be called as one way link building.

Directory submission Tool

Autofill form :- It is a mozilla Addon here u can fill all these entries given above and can fill using shortcut Alt+J.

Steps to setup autofill form :-

1/ Click on the settings in dropdown of addon (Pencil icon)

2/ Then go to profile tab and import profile (Save all below text in a text file and then import it) :-

autofillForms@blueimp.net 0.9.5.2 Web designing (?:)

URL http://www.baagdi.com (?:url)|(?:URL) (?:) true true
TItle web designer (?:title)|(?:TITLE)Business listings (?:) true true
Description baagdi.com provides web design templates in web 2.0, flash, xhmtl, dhtml, xml its web designing  services also available in mumbai, chandigarh, delhi, rajasthan. (?:bodytext)|(?:des) (?:) true true
Keyword web designer, web designers, websites designing, website designs, web design mumbai, web design templates, web designing delhi, web design in india (?:keyword)|(?:tag) (?:) true true
NAME baagdi (?:name)|(?:NAME)|(?:name)|(?:reg_username) (?:) true true
E_mail baagdidir@gmail.com (?:EMAIL)|(?:email)|(?:E_MAIL)|(?:e_mail)|(?:reg_email) (?:) true true

3/ Rename your profile by editing in Name box in Profile tab and then click on Apply button

4/ To edit all fileds of this profile click on "Main tab".

5/ To edit all fields change their values in value box and click on OK in last.

Try to find dofollow list of web directories.

High PR directories are more better

To get better results :- Submit on 120 - 200 high pr dofollow directories for one domain per day.

Directory list :- directorycritic.com, vmoptions.com

Type of SE submissions:- Automatic and Manual. Google prefer manual submission.
http://mrdefinite.com/submit-website-url/
http://dofollowlist.blogspot.com/

You can find the dofollow directory list here :- 

http://www.freebacklinklist.com/Free_Dofollow_Directory/

http://bit.ly/eefQoB

http://bit.ly/hasZ7i

Read More

How Robots.txt and humans.txt works | syntax, types and benifits

Robots
Also known as Crawler or Spider, a robot is a search engine program that "crawls" the web, collecting data, following links, making copies of new and updated sites, and storing URLs in the search engine's Index. This allows search engines to provide faster and more up-to-date listings.

Robots.txt
This is a file written and stored in the root directory of a website that restricts search engine spiders from indexing certain areas of the website.

First Come First Serve Search Engine robots will follow the first link they find to any particular page. They won’t follow additional links to the same page.

robots.txt               

1.   Useragent:*

Disallow:/

[it doesnot crawel whole site]

2.   User_agent:*

Disallow:/abc/[abc=directoryname]

3.   User_agent:*

Disallow:/abc/123.html

[it doesnot crawel 123.html in abc directory]

4.    Meta tag Robots

<name =”robots”; content=index, follow>

<name =”robots”; content=noindex, follow>

<name =”robots”; content=noindex>

<name =”robots”; content=index, nofollow>

<name=”robots”; content=noindex, nofollow>

5.    Use nofollow with hyperlink in page under a href tag. This is used if we don’t want to follow a specific link.

6.    <meta name=”robots”; content = “noscript”>

[It doesn’t crawel description in search engine below title also doesn’t crawel cached pages. This is used when we change whole theme of our site with same domain.]

Meta Robots
The "NAME" attribute must be "ROBOTS".

Valid values for the "CONTENT" attribute are: "INDEX", "NOINDEX", "FOLLOW", "NOFOLLOW". Multiple comma-separated values are allowed, but obviously only some combinations make sense. If there is no robots <META> tag, the default is "INDEX,FOLLOW", so there's no need to spell that out. That leaves:

<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">

<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<META name="ROBOTS" content="NOARCHIVE">
<META name="ROBOTS" content="NOSNIPPET">
<META name="ROBOTS" content="NOODP">
<META name="ROBOTS" content="NOYDIR">

* FOLLOW – a command for the search engine crawler to follow the links in that webpage
* INDEX – a command for the search engine crawler to index that webpage
* NOFOLLOW – a command for the search engine crawler NOT to follow the links in that webpage
* NOINDEX – a command for the search engine crawler NOT to index that webpage
* NOARCHIVE -  Useful if the content changes frequently: headlines, auctions, etc. The search engine still archives the information, but won't show it in the results. 
* NOSNIPPET - Encourages the search engines to use the title only, and to suppress the "cache" link. Might be useful if the site has special plus box listings in search results, but otherwise, not so much. This is also used when we change theme(content) of any website.
* NOODP - Encourages search engines to use the page title tag, and match term in context, or META Description tag content instead of the ODP content, which may be misleading or outdated.
* NOYDIR - Yahoo Slurp robot only (same as above) 

SEARCH ENGINES & ROBOTS:-
Google =  googlebot
MSN Search = msnbot
Yahoo = yahoo-slurp
Ask/Teoma = teoma
DMOZ Checker = robozilla
Baidu = baiduspider

Humans.txt
It's an initiative for knowing the people behind a website.
If possible, you can also add an author tag to the <head> of the site:
<link type="text/plain" rel="author" href="http://domain/humans.txt" />

humans.txt
/* TEAM */

                Chef:Baagdi Solutions
                Contact: info [at] baagdi.com
                Twitter: @baagdi
                From:Chandigarh

                CSS3 and HTML5 boy: Rashtra
                Twitter: @rashtra
                From:Sri Ganganagar

                Grphics boy: Sunil
                Twitter: @sunil
                From:Rajasthan

/* SITE */

                Last update:2011/01/23
                Language:English
                Doctype:HTML5 with CSS3
                Tools:PHP
                IDE:Baagdi Solutions and Notepad++

Read More