If some specific IP's are causing the memory errors, then it would be better to block them and see how things will run.
Also, I would suggest you to create a robots.txt file in your public_html folder and blocking all bots except of those you'd like to visit your website.
You should be able to see in stats what exact file is usually accessed by bots. Based on the stats, we can further add security measures into the site by adding some redirect rules to block request or password protecting that file.