« Google Maps Not In Google Search; Comparing Services & How Do They Do That? | Main | Become Shopping Search Beta Now Open »

February 11, 2005

Yahoo Tips On Crawling Issues

The Yahoo Search Blog has some webmaster tips out today: Yahoo! Search Tips for Webmasters: Saving Bandwidth. Gzip your files, and they'll unzip and save you bandwidth when crawling. Respond with the right HTTP code, and it will only download pages if there's been a change to them. Getting slammed by the Slurp crawler? Make it sip rather than slurp through the crawl-delay mechanism.

Posted by Danny Sullivan at February 11, 2005 8:22 AM

Trackback Pings

TrackBack URL for this entry:
http://post.blog.searchenginestrategies.com/cgi-bin/mt/mt-tb.cgi/5827