« Yahoo Provides Access To Library Records | Main | Keeping SEO Secrets »

November 16, 2004

New Robots.txt Tool

The robots.txt file allows you to prevent search engines from indexing documents on your web server, assuming they respect the convention. Most major search engines do. Ian McCanerin, one of our forum moderators, has posted a nice new tool to help you generate a robots.txt file. You can also discuss it in this forum thread.

Posted by Danny Sullivan at November 16, 2004 7:10 AM

• Stumble It!
• Digg this!
• Add to del.icio.us
• 

Trackback Pings

TrackBack URL for this entry:
http://post.blog.searchenginestrategies.com/cgi-bin/mt/mt-tb.cgi/5129