Baiduspider grab the bandwidth plug?
April 22, 2008 Posted by Al Castle
trackback
Categories: Castle, OSX, Puter Stuff, gnash-teeth
This spider from Baidu (translated by google - hence the blog title) crawls my site religiously and feverishly from a dozen different IP’s. I really doubt anyone in China cares about what I have to say. They should, especially with the Olympics coming up, but I know they don’t.
So with that I’m blocking you Baidu. I’m not editing my robots.txt, no not for you. All your various IPs will reach nothingness as my firewall goes ninja on you - mexi style.
字
(Chinese for “word”).
Comments»
i hear ya man, the baiduspider is a bandwidth theif. They are coming from many different ip addresses though, it seems a constant battle.
We just realised that our servers at work have been hammered by this Chinese cowards.
Its clear they switch IPs, that’s is why I would encourage everyone to block the host name and not rely on IP addresses only.