Category Archives: Software

Posted on
by

StatsD, created by Etsy, is a simple Node.JS server that provides powerful way to collect numerous statistics about your web application, and to do so simply and quickly.

At Etsy, they graph everything, which they use to great effect.

Collecting statistics is cool because it gives you hard data about how your software is performing. This in turn gives you powerful analytical tools to dig deep into your code, and find out what’s really going on (especially true when combined with a visualisation tool such as Graphite). It let you see the impact of code or infrastructure changes, and to quickly identify problems. Perhaps most importantly, it is only when armed kind of hard data that you can even begin to grasp at the nettles of code optimisation, growth and scalability.

Introducing StatsD for Elgg

With this in mind, and because I needed to collect some stats for a couple of client projects, I’ve put up on Github the first version of an Elgg statsD module. When installed and configured, this module will interface with a statsd server and collect a whole bunch of statistics from your running Elgg site.

Out of the box the current version of the plugin can log:

  • Events & Hooks (which in turn give you things like user signup events and object creation)
  • Exceptions
  • PHP Errors, Notices and Warnings
  • Elgg popups (system messages and error messages)
  • Database calls
  • Script execution time

In addition you can record your own statistics by making a call to wrapper functions contained in the plugin itself.

All data will be logged into a custom “bucket”, which is by default derived from your Elgg site name. This lets you log statistics from multiple different sites to a common statsd server.

Installation is pretty straightforward once you’ve installed the base infrastructure. Follow the instructions for installing graphite, nodeJS and statsd from the various sites around the internet, and then upload the elgg-statsd plugin to your Elgg site’s mod directory.

Once activated, you can specify the statsd server you wish to log to and configure what statistics you want to record.

Have fun!

» Visit the project on Github…

P.S. If you try and set this up and are seeing errors in your graphite log along the lines of “create() takes at most 5 arguments (6 given)“, then you are likely falling foul of this bug.

My solution was to build Whisper from the latest code in master rather than the stable 0.9.x branch. This worked for me, but of course YMMV.

Posted on
by

iCal is a simple format for sharing calendar, event and todo list information between applications. It is widely supported by many many applications.

I have recently been hacking on a couple of projects which required me to implement the format, and during this I hit on a couple of less well documented gotchas.

So, I’ve jotted them down here in order to hopefully save you some time…

  • Line endings: The specification says that line endings should be CRLF only, and some less tolerant clients puke if they’re not.
  • No blank lines: Ensure that your output contains no blank lines, again some clients with puke or interpret these as the end of the file.
  • End in .ics (the biggy): This is important if you dynamically generate an iCal feed for use with Google Calendar. You should set header type appropriately, e.g.

    header("Content-Type: text/calendar");
    header('Content-Disposition: attachment; filename="calendar.ics"');

    But critically, if all your entries are showing up only as “Busy”, you must also end the URL you pass to Google calendar as http://foo.com/bar/whatever.ics

You may find one of these validator tools useful too.

Hope that helps!

Image “Calendar Logo” by Quadmod

Posted on
by

WordPress, the popular blogging software written by Automattic, has a problem with SSL self signed certificates. Basically, they don’t work well in any of their newer software products or services.

In order to post an update, I must first log into my blog. This requires me entering a username and password into a login box in the usual way. By default, WordPress does not use the secure HTTPS protocol for this, instead it sends this password in the clear over HTTP.

This is not good, so I, like many others, force WordPress to carry out login and administration functions over HTTPS. This is relatively straightforward, and well documented in WordPress’ own documentation, but requires a SSL certificate.

You can obtain a SSL certificate in one of two ways. Either you pay for a third party issuer to give you one (which has the benefit of not triggering a warning in the browser), or you generate one yourself – a so called “Self Signed” certificate.

Self signed certificates are perfectly valid, but browsers will display a warning on sites which use them. A problem if you’re running a public facing service, but not if it’s just for your own private blog, and crucially the traffic is still encrypted.

The Problem

Unfortunately WordPress don’t seem to like self signed certificates.

The iOS WordPress client once worked fine with self signed certificates, but this functionality was removed in an update a few months ago. Attempts to connect now display an error about the certificate’s self signed status, but unlike all browsers, will not give you the option to proceed.

Jetpack, which is now replacing much of the functionality previously provided by separate WordPress plugins (most importantly WordPress stats), is completely broken.

When you attempt to activate the plugin, Jetpack complains about being unable to communicate with the site with the following error:

Error Details: The Jetpack server was unable to communicate with your site [IXR -32300: transport error: http_request_failed SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed]

There is no way to bypass this, since the cURL error originates on the Jetpack servers and would require a code change at their end to allow the self signed certificate.

WordPress remain tight-lipped

I am not alone in encountering these problems, but so far attempts to contact WordPress/Automattic for support by various mechanisms have all gone unanswered.

It is a legitimate point of view that certificate failures caused by self signed certificates should be a fatal. Personally, I think providing a mechanism to bypass these errors for those who know what they’re doing, is a better solution, but making it fatal is a legitimate point of view from a security standpoint.

I could resolve this issue by buying a certificate, although I have a number of good reasons, some financial and some technical, for why I have not yet done so. If Automattic were to point blank refuse to support self signed certificates in their products then I would have to find a way of making it work.

I also accept the possibility that I could have made a mistake in configuration, although I’m not sure what this could possibly be, and it is only Automattic products that are having issue.

I accept all this, however all requests for support in forum threads and direct, from myself and others, go unanswered. Bug reports for the iOS client are months old and are ignored. Similarly, direct support requests to Jetpack go unanswered.

Automattic: If self certified certificates are a feature that just won’t be supported, then please communicate with me and your other users, or at least update your FAQ. If you think I’ve made a configuration error then please say so. Please communicate, because this silence is infuriating!

Update 20/11/12: After much chasing around I’ve got a response, about JetPack at least. Seems that not allowing self signed certificates was originally a design decision (a clearer error message would have been nice!), however this decision has been re-thought and it is now seen as a bug. There is currently no time-scale as to when the issue will be addressed.