Screw It

february 1, 2005

Screw It. I am going to kill off Googlebot and Yahoo's Slurp once and for all. When I turned it back on the 16th of January and Google and Yahoo! came and ate away. As they processed the new info and put it into their databases I noticed an increase in spammers. I blocked Google and Yahoo! the day after Christmas and saw a noticeable decrease in traffic, both from the bots (which do about 100-150MB a month) and from external hosts (both legit and spammers). I am going to turn on the block at the firewall and lock out Googlebot and Slurp (and all other bots) for now. I am not going to let them back for a few months and see how things are going. With a worm attack that used Google to find vulnerable pages and all these spammers, I am pretty sure that they are using search engines to ferret out ways to spam my webserver. If you know about, you know how to get here. I post the URL in my signature on other forums and that will attract visitors. Otherwise, I guess there will be a lot less visitors on the site, which doesn't bother me much -- especially since Google seems to only bring in a lot of people who are looking for something that I don't write about or provide here on the site.