Ok, Fine, Googlebot and Slurp, Eat Up

january 16, 2005

After reconsidering, I think I'll let Googlebot and Yahoo! Slurp start spidering my sites again. I originally blocked those two spiders along with a bunch more from spidering the site because of security concerns -- worms using Google and Yahoo! to discover pages with vulnerabilities. But, I guess it would be nice to be indexed in Google to attract some visitors and stuff (like Mark said in the comments of this post). All other bots are still being blocked (via a firewall rule in my m0n0wall box), but Googlebot and Yahoo! Slurp can come back and feast on the content.