Spam comes in may forms.

I had been noticing some odd traffic appearing in my referrer logs from “buttons-for-website.com”, and a few other places. Odd, I thought, but I wasn’t too concerned.

A client recently asked me about it, since similar traffic was starting to appear in their analytics for a brand new site. I did a little bit of research, and it turns out that this is actually a spam attack.

Basically, the spammer hits your site and sets a referrer header containing a url and their spam message (keywords + another url, usually). Since a small percentage of sites make their referrer logs public (either deliberately or through misconfiguration), when these are indexed, they can be used to game the search engine of the site they’re trying to boost.

Stopping the spam with mod_security

I don’t like spammers, and it was starting to make my logs (and those of my client’s) a little noisy. So, I decided to do something about it. So, using mod_security, I added a couple of simple rules, which would drop the traffic where the referrer contained certain keywords.

Simple, but effective:

This seemed to put an end to the worst of it.

I also noticed that a few spammers were posting with obvious spam keywords in the referrer header, so I added a similar rule to block those for good measure:

Testing

To test your rules, you can use curl to hit your site and send a triggering referrer, e.g.

Or

Hope that helps!

Mod_security is a plugin for the popular Apache web server that lets you block malicious traffic on your web applications based on rules you define. Essentially, it acts as a firewall for web apps, blocking suspicious traffic and malformed requests. It is highly configurable, and comes with a good set of default rules to get you started.

I had need to configure it for a particularly security sensitive client site recently, and while I was at it I thought it’d be cool to fire it up on my personal server. I installed the module, reloaded Apache and began watching the output of the audit log.

Holy Pingback spam Batman!

Pingback, is a way of notifying a site that you’ve written about them in your blog. The receiving blog then usually renders this out as a link in the comments section of the post, allowing visitors to read the expanded discussion.

Watching the mod_security audit log was an eye opener, because I found that my site was being hit by a metric shitload of bogus pingback requests (the order of a couple every minute), all from different sources. None of these messages had made it as far as appearing on my site of course, seemingly they have been blocked by something in WordPress itself (probably Akismet), so I honestly didn’t realise that this was a thing.

ModSecurity was flagging them up because the body of the message was malformed XML, so was kicking them out (legitimate wordpress to wordpress pings were being accepted, at least when I tested it), and closed the connection with a 400 response. In every case the request had similar signatures; it was chunk encoded, it was always linking to a real site (in most cases some poor schmo’s pwned wiki, which didn’t mention the post that was pinging), and the User-Agent was always “PHP/5.2.10”, so we’re clearly dealing with a script kiddy.

One thing I noticed was that, although I was getting pingback spam from multiple sources, each IP would retry every couple of minutes. This meant that my web server was having to spool up to handle each request, even if the spam did not make it through. Negligible in the grand scheme of things, but irritating nonetheless.

Since I am a firm believer in both a strength in depth approach to security, and I like quiet logs, I wrote a fail2ban script to catch these messages. As before, because I’m operating behind a reverse proxy I’m keying off the squid logs (until I can work out how to change mod_security’s log to spit out X-Forwarded-For anyway).

After a couple of hours, this is what my munin graph looked like:

Holy crap.