Here is one that I have not seen in a long time. Some sneaky snakes were trying to get some free backlinks to their not so reputable websites on to my website. How? They simply pinged my site from their sites…a few hundred times and counting.
On my site, I HAD, live stats that anyone could see for this site. It was just a basic stat system designed for advertisers, but it was public. It was also not listed on my robot.txt file, meaning Google was caching it. Quickly taking advantage of this, these people simply started pinging my site with their sites url’s to the point that they were being listed as top referrers. Nice little game, but fortunately for me, caught quickly. Still though, the content is cached, and only time will remove them.
Not Allowing to Get Indexed
I don’t block much on my site, and rarely use the robots.txt file. It’s one of those set up when you set up the site, and forget about it kind of things. Unfortunately, in this case I should have remembered to update it.
To create your own robot.txt file to block search engines like Google from caching your site, all you need to do is create the file in your root directory. It may already be there. You then tell the types of robots to pay attention (that is my first line in the sample below), and then follow it with the details of what folders you don’t want the browsers to follow. You can find a great tutorial with more details about it at The Web Robots Website.
A good note for myself to remember? If there is something automated on the site, don’t get gamed by sneaky snakes. They are watching, and always looking for an easy score. Unwatched, automated functions, is most likely one of the easiest to take advantage of.