Blocking Referrer Spam

Posted on June 20, 2011
Filed Under Server Security, Web Hosting | Leave a Comment

We’ve just been dealing with what can only be described as the worst case of Referrer Spam and Comment Spam on a client’s wordpress blog site today…

At first, one our support technicians went about trying to create a list of IP addresses to block that sent referrer strings to the wordpress blog, but soon realised that it was going to be a long and arduous job. We considered using mod_security to automatically monitor for referrers and create a block list based on 3 or more referrers within a given time period, but with the absolute deluge of referrer spam coming in (up to 5 per second on many occasions) it was decided that it may take up too much by way of server resources.

So we opted to simply block any request that contained a referrer string that did not match the site’s domain name, i.e. any referrer which did not come from the site itself was blocked. In this instance we decided to return a 403 Forbidden error which uses up 0bytes.

So, using a mod_rewrite rule we added the following to the site’s htaccess file …..


<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !myhdpackage.com
#RewriteRule .* %{HTTP_REFERER} [L]
RewriteRule .* - [F,L,E=spam:refspam]
</IfModule>

Obviously you would replace the “the-domain-name.com” part with your actual domain name.

So, here we are checking for any referrer, then checking if any such referrer string contains the site’s domain name and if so we let it pass (i.e. continue to load the page as usual), but if it does not match the site’s domain name then it is sent to a 403 Forbidden page. This blanket approach was the simplest way to stop this absolute deluge of referrer spam, it’s unfortunate that while in operation ti will also block any genuine referrer, but the site owner is prepared to live with that as a temporary measure, having just used up 3gb of bandwidth per day by this referrer spam, it’s a small price to pay. Once the attack, because it can reasonably be described as a Referrer Spam Attack, has subsided then the rule can be commented out (leaving it there for instant reactivation if necessary). In the mean time, a search for an up to date wordpress plugin that detects and blocks referrer/comment spam is on. For now this is one simple and complete solution to Stopping Referrer Spam.

We did consider redirecting any request with a referrer string to the referrer itself (wouldn’t it be nice to burn up their bandwidth in the same way that they were burning up this site’s bandwidth?), but in reality it would have led to the site and server being penalised/blocked by other servers. That is not desirable at all.

As for the comment spam, disabling comments for now was the first course of action, then preventing links to external sites in comments and a comment spam plugin was the way to go.

Testing the Referrer Spam Block

We can see what happens when a referrer string is sent int he request to the site by running the following from SSH:

curl -e "http://www.somesite.com/spamlink/" http://www.your-site.com

This will output the following (which is exactly what would be sent back to the bot/browser):


<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /
on this server.</p>
<p>Additionally, a 500 Internal Server Error
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>

You can test further to see that in the absence of a referrer string, or if the referrer contains the site’s domain name; the output will be the actual page requested.

DoS-Deflate blocks numbers not IP addresses

Posted on August 5, 2008
Filed Under Denial of Service Attacks, Server Security, Web Hosting | Leave a Comment

We’ve seen a large number of problems with the impementation of the highly acclaimed (D)DoS-Deflate script which on occasions reads the netstat command incorrectly and subsequently blocks a single number instead of the IP address which exceeds the number of connections into the server on which it is installed. Please see http://deflate.medialayer.com/ to obtain the script and installation instructions.

This is purely down to the netstat command that DOS-Deflate uses, it does not account for some elements of the strings returned, particularly when the string ‘::ffff:’ that is added to http (port 80) connections.

To overcome this error a rewrite of the netstat command in the ddos.sh file (located in /usr/local/ddos directory if you installed in the default fashion). Line 117 reads…

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr > $BAD_IP_LIST

this should be rewritten to read as follows…

netstat -ntu | grep ':' | awk '{print $5}' | sed 's/::ffff://' | cut -f1 -d ':' | sort | uniq -c | sort -nr > $BAD_IP_LIST

NOTE: this command should be written on a single line, you should also check each character as selecting and copying can sometimes lead to different characters being pasted!

We use the sed command to replace the ‘::ffff:’ with nothing (an empty string), thus removing it.

A slight improvement to a widely used and highly acclaimed script 🙂

Recently


Categories


Archives

website promotion