Replying to Google bots with errors will keep you out of the index, though they may keep retrying forever in case the page starts working again. A lot of times when I am retiring a site I will look at traffic logs and it will all be Google or Bing requesting content that is gone - sometimes years after the content was taken down. I think it is just greed on their part, forever hoping that if content was there before it might appear there again. No telling how much bandwidth gets consumed by that sort of traffic each year. If you are terminating traffic directly it can be really interesting to see all the connections that get opened but never receive an http request before being closed. A lot of that seems to be broken browser plugins, people scanning for live ports, or maybe seeing what certificate is offered in the TLS handshake.