WEB DESIGN AND MARKETING SOLUTIONS FOR BUSINESS WEBSITES
310
21. Google??™s can be found at www.google.com/addurl and Yahoo??™s at http://search.yahoo.com/
info/submit.html.
Google also provides developers with a meta tag specifically targeting the Google searchbot.
Called the Googlebot robots meta tag, it provides a few more options applicable to
just Google??™s services. Here is a possible example:
nofollow and noindex work the same. noarchive prevents Google from archiving content
in its cache, and nosnippet prevents it from retrieving a blurb with bolded terms. By
default, all terms are positive, so the tag should not be included unless there is a reason
for Google not to do something.
Robots.txt
Where the robots meta tag is good for directing search engines for individual pages, a
robots.txt file can provide global information for the site, as well as specific directories.
When a search engine first crawls a page, it actively seeks the file robots.txt (which must
be all lowercase), which is nothing more than a plain-text file.
Pages:
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600