Fail2Ban is a simple, but powerful, open source intrusion detection and prevention system which can run on most POSIX compliant operating systems. It works by monitoring various system logs for signs of intrusion attempts (failed logins etc), and on finding them, executes a preconfigured action.

Typically, this action is to block further access attempts from the remote host, using local firewall rules.

Out of the box, Fail2Ban comes configured to monitor SSH for signs of intrusion. However, since it works by monitoring log files, Fail2Ban can be configured to monitor many other services. I figured it would be pretty cool if you could also use it to protect Elgg sites as well.

Elgg already has a per user account lockout on login, however it is not without its limitations. It is pretty basic, and while it protects against access to specific accounts, it does not protect against dictionary attacks against multiple or non-existent accounts. Using Fail2Ban, you can protect against multiple access attempts from the same IP address easily, and the cut them off at the network level, frustrating the attack.

Installing Fail2Ban

The first step to getting this all working is to install Fail2Ban.

This is covered in detail elsewhere, but on Debian/Ubuntu it was a simple matter of pulling it from the apt repo:

sudo apt-get install fail2ban

Out of the box Fail2Ban will block using IPTables, but if you use shorewall, as I do, you’ll need to modify the actions to use that.

Getting Elgg to log access

It is an omission (quite possibly on my part), but the default Elgg login action does not explicitly log login attempts and login errors. While it is quite probable that you could hack together some regexp to parse the apache error logs, these are often quite noisy, highly changeable, often stored in odd locations, and, more often than not, are turned off in production environments.

I thought I’d make things a little easier on myself, and so I wrote a tiny Elgg plugin which overrides the default login action and outputs explicit error messages to the system auth.log, on both success and failure.

Once installed, you should begin to see logging messages start to appear in your server’s auth log (usually /var/log/auth.log) along the lines of this:

Mar 22 18:24:43 web elgg(web.example.com)[16483]: Authentication failure for fakeuser from 111.222.333.444
Mar 22 18:25:05 web elgg(web.example.com)[16483]: Accepted password for admin from 111.222.333.444

Again, to keep things simple, and to avoid getting a regular expression headache, I kept the authentication messages similar to those used by the SSH filter.

Monitoring the log with Fail2Ban

Finally, you need to configure fail2ban to look out for the Elgg messages in the auth.log.

  • Copy the elgg.conf into your fail2ban filters directory, on Debian this is in /etc/fail2ban/filters.d/
  • Create a jail.local in /etc/fail2ban/ if you have not already done so, and then create a rule, along the lines of the following:

    [elgg]
    enabled = true
    filter = elgg
    logpath = /var/log/auth.log
    port = all

Restart Fail2Ban, and you should be up and running! To test, attempt to log in (using a machine on a different machine if at all possible) and try a few failed logins.

A future enhancement of this that you could consider, especially if running in a production environment, is to modify the block action to redirect queries from the offender’s IP to a place-holder page explaining why they have been banned. This could probably be done quite easily using a REDIRECT rule, although I’ve not tried it yet.

Anyway, code, as always, is on github. Have a play!

» Visit the project on Github…

mysql-logo3 So, the other week, I migrated all my sites over to a new server. This was accomplished with minimum fuss, using lots of rsync magic and juggling of DNS ttls.

The part of the migration I imagined would be the most complicated, moving several tens of MySQL databases running on the old server to the new, turned out to be pretty straightforward, and essentially completed with one command. I thought others might find it handy to know how…

First, I brought down Apache on both servers, so I could be sure that nobody was going to try and write to the database. This may not really be required, since mysqldump can handle dumping live databases consistently, for MyISAM at least, but in never hurts to be paranoid.

Next, I needed to move all the databases, together with their access permissions and users (so the full MySQL schema) to the other server. One way to do that is to copy the whole /var/lib/mysql directory over. However, I had a lot of cruft in there (old bin-logs etc), plus there were a number of articles suggesting that the straight binary copy had a number of issues, especially for mixed storage engine environments. So, I opted for the mysqldump method.

Traditionally, this takes a lot of SCPing. Here’s how to do it with one command, using the magic of Unix pipes:

mysqldump -u root -pPASSWORD --all-databases | ssh USER@NEW.HOST.COM 'cat - | mysql -u root -pPASSWORD'

Boom. This ran surprisingly quickly for me, and you can of course just as easily use this method to transfer a single database.

Three gotchas:

  • If the link dies, you need to start again, so don’t do this over a flakey connection, and I suggest you run the command in a screen if the first server isn’t localhost.
  • You need to restart the database server on the target machine for the new users and privileges to come into effect.
  • On Debian, you may see an error along the lines of:

    Got error: 1045: Access denied for user ‘debian-sys-maint’@’localhost’ (using password: YES) when trying to connect

    Fix this by executing the command:

    GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'THEPASSWORD' WITH GRANT OPTION;

    Where THEPASSWORD is the password found in /etc/mysql/debian.cnf

One final note: this will only work if both servers are the same major version number. I was moving between two Debian 6.0 installs, so YMMV.

Happy Easter!

This is just a quick note to readers of this website to give you advanced warning of some planned server maintenance.

Over the next few days I will be migrating all sites and services over to a new hosting facility. Unfortunately, the migration will necessitate short periods of downtime while I move things over and repoint DNS etc. We will do the utmost to minimise any disruption caused, but over the next week or so, you may notice that certain services that you use will become unavailable for short periods
of time.

If you have any questions please get in touch!