How to block bots from seeing your website

After dealing with a recent issue of BAD bots hogging up my server resources, I began to look into solutions for keeping these nasty critters out. At one point my web site was even shut down BY my ISP because it reported close to 414K hits from bots that were hogging the resources (on a shared server, that is)

I did give the solution “Black Hole” a whirl, however after following instructions, my site wouldn’t come up due to errors.
So instead of trying to figure out why a recommendation that was actually quite old (2010) wasn’t working I went a different direction.
(Note: if “Black Hole” is still working, then “perishablepress” might want to update their notes to current rather than adding on a addendum that seemed to confuse.

I did find a solution on the site hackrespair.com, that seemed to be just what I was looking for.

In short, by adding a bit of code to your .htaccess file, you can cut the BAD BOTS of at the pass

The link for the code is located here: http://pastebin.com/5Hw9KZnW

I suggest cutting and pasting FROM section marked “RAW Paste Data” on the bottom (also it makes sense to check back as the code is updated – current right now to June 2012)

Then follow these recommended steps from the site:

1.First, see the list using the above link.
2.Copy it to your computer’s memory.
3.Next open your .htaccess file via your editor.
4.Paste this list at the very bottom of your .htaccess file

Note: you can also block using the robots.txt file, however this method seemed to be a bit tighter in keeping the bots out than the robots.txt method

That’s it!