It's important to know that "bad" bots do not always honor the instructions that you have in the robots.txt - You may end up having to ban those IPs directly.

"Good" bots like Slurp and googlebot will honor your instructions.