650 visitor Bots

Is 650 visitor bots preventable? What can or should be done as it's slowing the site down.

 

Some search engines dont care how many bots they send to your site that is causing the slowdown of your site. If you know who is sending what you can block them just do a google search and you find many ways to block.

 

Majestic 12 was the cause.

 

That's a lot for Majestic bot, it's not one you'd usually link with coming in very large numbers

 

I looked at the last 1000 visitors and it was everywhere. I didn't see much activity from ips that I see the regular users using. I don't understand if Majestic is legit or not. I see conflicting information when I looked it up.

 

Majestic is a bot or crawler. Not sure which of the two but have had it listed on my other forums before in detected bots that show up.

http://www.majestic12.co.uk/projects/dsearch/mj12bot.php

Actually it shows as MJ12bot, I have it listed on mine. And they say you can can block it in robots.txt using.

User-agent: MJ12bot
Disallow: /

You should add a delay in your robots.txt file anyway really to try and limit all best you can, even if not all follow it.

User-agent: *
Crawl-Delay: 10

That will limit all bots that follow it to not hit your forum in huge numbers all at once, such as Yahoo Slurp!