Posted on
by

The server-status page gives you a wealth of information about your apache server, and among other things it is necessary for the Apache munin plugin to work. However, by default it exposes sensitive data when run behind a squid reverse proxy.

In order to lock this down, you need to modify your /etc/squid/squid.conf file slightly…

acl no_stats urlpath_regex /server-status
http_access deny no_stats

This defines an ACL that will prevent squid from giving access to any request ending in /server-status. Because, in a reverse proxy configuration, it is squid that is hit when a client request a page from port 80, this prevents public access.

You’re not quite done though, because this setting assumes that your real web server (sitting behind the squid proxy on a different port) is NOT publicly visible. This can be achieved by placing it behind the firewall.

Munin

If you’re using munin to monitor your apache processes, you’ll need to make a small modification to the /etc/munin/plugin-conf.d/munin-node file, add/modify the following:

[apache_*]
env.url http://127.0.0.1:%d/server-status?auto
env.ports [apache server port]

This will tell the apache plugins where the server status page is, and what port the real apache instance is running on.

Posted on
by

MuninMunin is a powerful and highly customisable network monitoring tool. It lets you collect data about pretty much anything, from the number of apache processes running, to the number of IP addresses blocked by fail2ban, to the current temperature of your CPU.

All this data is collected and graphed over time, and provides you with a powerful insight into the performance of your infrastructure, and helps track down the root cause of any failure or outage.

I monitor just about anything I can, and one of the many things that I graph are the number of PHP errors appearing in my apache logs. This gives me an idea of the overall health of some of the platforms that I develop, as well as spotting instances where I introduce a regression which would otherwise not be spotted.

Powering this, I use a munin plugin called loggrep, which, as the name implies, works by grepping a log file for a certain bit of regular expression and returning a count of every instance found. This is simple but very effective, however, it took a little bit of fiddling to get it working.

The symptoms

You install munin, and all the other plugins are present and generating graphs, but those generated by loggrep are missing. You follow the munin troubleshooting steps and you find that while you can connect to to the node and the plugin appears to be running and collecting data:

host #: telnet localhost 4949
# munin node at host
fetch loggrep_foo
count.value 35
errors.value 12
.

The plugin does not appear in the list when you telnet in and type “list“.

The fix

After a lot of hacking about, I managed to fix this, but there are a number of things that can catch you out.

  1. Patch the loggrep plugin: Loggrep, by default, seems to mis-report the host_name variable, displaying a file path to the plugin, you’ll see what I mean if you run munin-run loggrep_foo config. Fix this by editing the munin loggrep plugin, which can usually be found in /usr/share/munin/plugins/, and change the line:

    (my $host_name = $0) =~ s|.*_([^_]+)_.*|$1|;

    to

    (my $host_name = $name) =~ s|.*_([^_]+)_.*|$1|;

  2. Check your config: Check your loggrep configuration… the plugin will silently fail if there are duplicate labels in a configuration section, or if it is missing any of the required environment variables. So, for example, if you’ve got two env.label_foo definitions defined either in the same section or in a section that overrides it, you will have problems.
  3. Check your plugin names: For some reason, probably related to the hostname regexp, when you create your symlink in /etc/munin/plugins/ names like loggrep_foo will work, but names containing two underscores like loggrep_foo_bar will not.

This got it working for me, but of course YMMV. Hopefully, if you’re having the same problem, this’ll save you a few hours hair pulling!

Posted on
by

As I am sure most of you are aware by now, I’m a keen pilot, and as I’m sure most of you can work out, knowing what the weather is doing is kinda critical.

My club operates a weather station, and during the tower’s operating hours, the data is uploaded to the internet. I thought it’d be pretty cool if I could pull this data in a machine readable form in order to be able to some coding using the sensor data.

After some research, I discovered that the weather station software outputs the raw data in a text file, which, with a little bit of coding, you can parse and extract useful data. I dug up the specification for the file from some old documentation I found on the web (to save you searching, I’ve stuck the file format spec up on gist for convenience), and using this I was able to write a little tool to parse the file and output values requested in a highly customisable way.

Usage

Pass the code the URL of your clientsraw.txt and optionally specify a format string (it will default to outputting some basic useful weather information).

python wd-parse.py -u http://example.com/path/to/clientraw.txt [-o "formatted list of output"]

I find this useful, not least because it provides me with a quick way of telling if it’s a flyable day before getting into the car. But of course the real power of this tool is being able to hook it into other things, for example, you could graph weather over time using a tool like munin, output a METAR, or even cross reference the wind direction and runway orientation and calculate crosswind components.

All of these I plan to do if I get the time!

» Visit the project on Github…