Blocking Referrer Spam

Posted on June 20, 2011
Filed Under Server Security, Web Hosting | Leave a Comment

We’ve just been dealing with what can only be described as the worst case of Referrer Spam and Comment Spam on a client’s wordpress blog site today…

At first, one our support technicians went about trying to create a list of IP addresses to block that sent referrer strings to the wordpress blog, but soon realised that it was going to be a long and arduous job. We considered using mod_security to automatically monitor for referrers and create a block list based on 3 or more referrers within a given time period, but with the absolute deluge of referrer spam coming in (up to 5 per second on many occasions) it was decided that it may take up too much by way of server resources.

So we opted to simply block any request that contained a referrer string that did not match the site’s domain name, i.e. any referrer which did not come from the site itself was blocked. In this instance we decided to return a 403 Forbidden error which uses up 0bytes.

So, using a mod_rewrite rule we added the following to the site’s htaccess file …..


<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !myhdpackage.com
#RewriteRule .* %{HTTP_REFERER} [L]
RewriteRule .* - [F,L,E=spam:refspam]
</IfModule>

Obviously you would replace the “the-domain-name.com” part with your actual domain name.

So, here we are checking for any referrer, then checking if any such referrer string contains the site’s domain name and if so we let it pass (i.e. continue to load the page as usual), but if it does not match the site’s domain name then it is sent to a 403 Forbidden page. This blanket approach was the simplest way to stop this absolute deluge of referrer spam, it’s unfortunate that while in operation ti will also block any genuine referrer, but the site owner is prepared to live with that as a temporary measure, having just used up 3gb of bandwidth per day by this referrer spam, it’s a small price to pay. Once the attack, because it can reasonably be described as a Referrer Spam Attack, has subsided then the rule can be commented out (leaving it there for instant reactivation if necessary). In the mean time, a search for an up to date wordpress plugin that detects and blocks referrer/comment spam is on. For now this is one simple and complete solution to Stopping Referrer Spam.

We did consider redirecting any request with a referrer string to the referrer itself (wouldn’t it be nice to burn up their bandwidth in the same way that they were burning up this site’s bandwidth?), but in reality it would have led to the site and server being penalised/blocked by other servers. That is not desirable at all.

As for the comment spam, disabling comments for now was the first course of action, then preventing links to external sites in comments and a comment spam plugin was the way to go.

Testing the Referrer Spam Block

We can see what happens when a referrer string is sent int he request to the site by running the following from SSH:

curl -e "http://www.somesite.com/spamlink/" http://www.your-site.com

This will output the following (which is exactly what would be sent back to the bot/browser):


<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /
on this server.</p>
<p>Additionally, a 500 Internal Server Error
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>

You can test further to see that in the absence of a referrer string, or if the referrer contains the site’s domain name; the output will be the actual page requested.





Comments

Leave a Comment

If you would like to make a comment, please fill out the form below.

You must be logged in to post a comment.

Recently


Categories


Archives


website promotion