So I'm getting this on my Apache log on a small server for a local company:
1.2.3.4 - - [09/Nov/2015:17:00:16 +0000] "GET /wp-content/uploads/2012/08/gold-coins.jpeg HTTP/1.1" 301 342
about 500,000 times per day with 150,000 unique IP's. I've redirected that traffic outside my site (hence the 301) because it was crashing the server. But it is still using some bandwidth. What is the best way to remedy this situation?
This issue is called image hotlinking.
Nginx can be configured in a variety of ways to protect you from it. Read it about it here:
http://www.htpcbeginner.com/image-hotlink-protection-nginx/
http://nginxlibrary.com/hotlink-protection/
http://nodotcom.org/nginx-image-hotlink-rewrite.html
Trouble in Nginx hotlink protection
As you are saying some 150K IPs hit it 500K times, that means about 3 times each. With that in mind, you can also add browser caching of such static jpeg files in the mix, to avoid 66% of the requests. Serving static files from a so-called cookieless domain saves another bit.
Another route to consider is banning IPs at the firewall level, for example with the help of Fail2Ban. If you ban each after the first download of that file, it saves you 66% of the requests. Because it never reaches your webserver, this saves a lot of resources. But this doesn't stop it at the source.
Referrer based hotlink protection is ideal, and when you keep a list of such referrers, you can ask them to stop hotlinking so the issue hopefully goes away after a while.
