6a0120a85dcdae970b016301e98de2970d-800wi

We all know how important it is to secure web servers with encryption. As I’ve mentioned before, port 80 HTTP should be considered deprecated at this point!

Just as important (potentially more so), but often overlooked, is to ensure that your email server is also secure.

STARTTLS is a technology that lets you start an encrypted session during a standard SMTP connection. In the same way as HTTPS secures web, STARTTLS will secure email in transit from your mail client to the server, and from server to server. This makes it much harder to passively read the traffic, and having more encrypted traffic on the internet is only ever a good thing.

This only protects email in transit from server to server of course, so this is not a replacement for end to end encryption methods like PGP, but it does complement it… and since most email is still sent insecurely, this adds extra security without requiring your users do any extra work.

It’s easy to set up (for Exim at least), and it transparently runs on port 25, so there’s no reason not to!

Generate your keys

As with web, you’ll need a server key and certificate file.

For my public mail and MX relay servers, I decided to use valid certificate authority certificates. Clients, and some relaying servers, will throw a certificate error for self signed certificates, but others will not. Better safe than sorry, and since I already had a valid certificate on my site for the server in question, I simply recycled the certificate.

If this is your internal server, you can use a certificate signed by your own certificate authority, supported by the machines in your organisation.

The default exim configuration expects to find certificates in /etc/exim4/exim.key and /etc/exim4/exim.crt.

Enable TLS

The basic STARTTLS configuration by simply editing exim4.conf.template and setting MAIN_TLS_ENABLE = yes in the tlsoptions section. Restart exim and you should have STARTTLS support enabled.

As with a web server, you can configure ciphers etc at this stage. On my server at least, the defaults seemed reasonably strong, but as we learn which ciphers have been compromised by GCHQ and the NSA, we might need to tweak these.

Test your configuration

Next, you should test your configuration.

To do this, the simplest way is to use a program called swaks, which you should find in your distro’s package library.

Should produce a result something like…

If you get an error when starting TLS examine your exim log for the cause.

Like many people out there who have servers, I use cross site backups for the purposes of disaster recovery.

While I also use “proper” backup software as well (which maintains a historic delta of changes, among other things), I’m a big fan of redundancy, so like many other server administrators out there, I also perform regular site to site copies using Rsync over SSH.

Rsync, for those who aren’t familiar, is a file copy tool, which, after the first copy, will only send changes during subsequent updates. This makes it a very efficient tool, especially when used over an internet connection. SSH stands for Secure Shell, and is an encrypted connection between two computers.

Anyway, to enable rsync from server A to server B, it is common to perform the login via key. This means that on Server A you’d generate a SSH keypair for your backup user, then copy the public key that was generated into the ~/.ssh/authorized_keys file for your backup user on Server B. There’s a good guide to doing that here.

Because rsync is going to be executed automatically via cron script, it is necessary to create the key file without a password.

Backups are running, so what’s the big?

Ok, so the key for your backup user isn’t password protected, but that doesn’t mean that anybody can log in as that user, they’d still need to have a copy of the cryptographic key.

If someone is able to break into Server A and steal your backup user’s key, they will be able to log in to Server B as that user. If that user is a non privileged user, that isn’t absolutely terrible (although they could, for example, read your /etc/passwd file, or anything else you’ve not locked down) and you can revoke this key at any time. However, since strength in depth and compartmentalisation are two concepts you should always remember when building a secure system, it’d be nice if it was possible to do something about it.

Thankfully there is.

Jail time

The solution is to configure Server B’s ssh client to place the backup user’s sessions into what’s called a “Jail” using a Linux tool called chroot. When the backup user logs in, they will not have access to any other parts of the system, other than what’s in their jail cell.

It’s a little fiddly to set up, but most of the hard work is done by SSH.

  • First, install chroot: apt-get install dchroot debootstrap
  • Configure your SSH server
    • Open up /etc/ssh/sshd_config
    • Make sure there’s a line that says
    • At the end of the file, tell SSH to create a chroot jail for your backup user:

      Note, because of the way chroot works, you’ll need to make sure the chroot directory is owned by ROOT, even if it’s actually the home directory of your backup user.
    • Save, and restart your SSH server.

This gets you part of the way, you should now be able to SFTP into Server B using your backup user, and a normal SSH session will be refused. When connected, you will be restricted to the location set in ChrootDirectory.

Unfortunately, rsync needs more than this, and in order to copy files it’ll need access to the shell (I’m assuming bash), as well as the rsync application itself, together with whatever libraries are required.

Therefore, it becomes necessary to create a partial chroot image in the backup user’s chroot directory. You could do this the traditional way (e.g. by using something like debootstrap), which will create a mirror of your base operating system files in the chroot jail. However, this generally takes a few hundred megabytes at least, and if all you want is to copy some files, you don’t want to give access to more than you need.

Instead, I opted to create a skeleton chroot jail by hand.

The goal here is to mirror the filesystem of your server inside the chroot jail, so that if a file exists in /foo/bar, then you need to copy it to /home/backup-user/foo/bar, and make sure it’s owned by root.

  • Copy bash from /bin/bash to the directory /home/backup-user/bin/
  • Copy rsync (on my system this was in /usr/bin)
  • Next, you need to copy the symbolic link libraries to which these files are linked against. You can use the tool ldd to interrogate the executable and get a list of files to copy, e.g:
    Copy the files which have directories into the appropriate locations, e.g. /lib/x86_64-linux-gnu/libtinfo.so.5 should go into /home/backup-user/lib/x86_64-linux-gnu/
  • Do the same for /usr/bin/rsync

All being well, your rsync backups from Server A should now function as before, however the backup user will not be able to touch anything outside of the chroot jail.

Hope that’s useful to you!

6a0120a85dcdae970b016301e98de2970d-800wi

So, let us talk plainly. You absolutely, definitely, positively should be using TLS / HTTPS encryption on the sites that you run.

HTTPS provides encryption, meaning that anyone watching the connection (and yes, people do care, and are absolutely watching), will have a harder time trying to extract content information about the connection. This is important, because it stops your credit card being read as it makes its way to Amazon’s servers, or your password being read when you log into your email.

When I advise my clients on infrastructure, these days I recommend that all pages on a website, regardless of that page’s contents, should be served over HTTPS. The primary reason being a feature of an encrypted connection which I don’t think gets underlined enough.

Tamper resistant web

When you serve content over HTTPS, it is significantly harder to modify. Or, to put it another way, when you serve pages unencrypted, you have absolutely no guarantee that the page your server sends is going to be the page that your visitor receives!

If an attacker controls a link in the chain of computers between you and the site you’re visiting it is trivial to modify requests to and from a visitor and the server. A slightly more sophisticated attacker can perform these attacks without the need to control a link in the chain, a so called “Man on the side” attack – more technically complex, but still relatively trivial with sufficient budget, and has been widely automated by state actors and criminal gangs.

The capabilities these sorts of attacks give someone are limited only by budget and imagination. On a sliding scale of evil, possibly the least evil use we’ve seen in the wild is probably the advertising injection attack used by certain ISPs and Airplane/hotel wifi providers, but could easily extend to attacks designed to actively compromise your security.

Consider this example of an attack exploiting a common configuration:

  • A web application is installed on a server, and the site is available by visiting both HTTP and HTTPS endpoints. That is to say, if you visited both http://foo.com and https://foo.com, you’d get the same page being served.
  • Login details are sent using a POST form, but because the developers aren’t complete idiots they send these over HTTPS.

Seems reasonable, and I used to do this myself without seeing anything wrong with it.

However, consider what an attacker could do in this situation if the page serving the form is unencrypted. It would, for example, be a relatively trivial matter, once the infrastructure is in place, to simply rewrite “https://” to “http://”, meaning your login details would be sent unencrypted. Even if the server was configured to only accept login details on a secure connection (another fairly common practice), this attack would still work since the POST will still go ahead. A really sophisticated attacker could intercept the returning error page, and replace it with something innocuous, meaning your visitor would be non the wiser.

It gets worse of course, since as we have learnt from the Snowden disclosures, security agencies around the world will often illegally conscript unencrypted web pages to perform automated attacks on anyone they view as a target (which, as we’ve also learnt from the Snowden disclosures, includes just about everybody, including system administrators, software developers and even people who have visited CNN.com).

Lets Encrypt!

Encrypting your website is fairly straightforward, certainly when compared to the amount of work it took to deploy your web app in the first place. Plus, with the new Lets Encrypt project launching soon, it’s about to get a whole lot easier.

You’ll need to make sure you test those configurations regularly, since configuration recommendations change from time to time, and most shared hosts & default server configurations often leave a lot to be desired.

However, I assert that it is worth the extra effort.

By enabling HTTPS on the entire site, you’ll make it much much harder for an attacker to compromise your visitor’s privacy and security (I say harder, not impossible. There are still attacks that can be performed, especially if the attacker has root certificate access for certificates trusted by the computer you’re using… so be careful doing internet banking on a corporate network, or in China).

You also add to the herd immunity to your fellow internet users, normalising encrypted traffic and limiting the attack surface for mass surveillance and mass packet injection.

Finally, you’ll get a SEO advantage, since Google is now giving a ranking boost to secure pages.

So, why wait?