<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Marcus Povey &#187; howto</title>
	<atom:link href="http://www.marcus-povey.co.uk/tag/howto/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.marcus-povey.co.uk</link>
	<description>Making the world a better place, one byte at a time...</description>
	<lastBuildDate>Mon, 06 Feb 2012 19:13:58 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.1.1</generator>
	<atom:link rel='hub' href='http://www.marcus-povey.co.uk/?pushpress=hub'/>
		<item>
		<title>Life hack: Connecting @ifttt to @rememberthemilk</title>
		<link>http://www.marcus-povey.co.uk/2012/01/19/life-hack-connecting-ifttt-to-remember-the-milk/</link>
		<comments>http://www.marcus-povey.co.uk/2012/01/19/life-hack-connecting-ifttt-to-remember-the-milk/#comments</comments>
		<pubDate>Thu, 19 Jan 2012 16:34:56 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[Web]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[ifttt]]></category>
		<category><![CDATA[lifehack]]></category>
		<category><![CDATA[remember the milk]]></category>
		<category><![CDATA[rtm]]></category>
		<category><![CDATA[twitter]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=956</guid>
		<description><![CDATA[For a little while now I have been using the online todo list Remember The Milk to keep track of the numerous disparate tasks and projects I&#8217;ve got going on at any one time. Using lists, tags and smart searches I&#8217;ve managed to not only never lose track of a task. I estimate this has [...]]]></description>
			<content:encoded><![CDATA[<p><img src="http://www.marcus-povey.co.uk/wp-content/ifttt-logo.png" align="right" />For a little while now I have been using the online todo list <a href="http://www.rememberthemilk.com">Remember The Milk</a> to keep track of the numerous disparate tasks and projects I&#8217;ve got going on at any one time. </p>
<p>Using lists, tags and smart searches I&#8217;ve managed to not only never lose track of a task. I estimate this has made me about a billion times more efficient, and has reduced stress levels a thousand fold. </p>
<p>Additionally, I use the new and <a href="http://ifttt.com">increasingly indispensable ifttt beta</a> to automate a bunch of tasks around the internet; e.g. to grab a copy of Facebook pictures I get tagged in, <a href="http://www.marcus-povey.co.uk/2011/12/08/idea-lets-give-roads-junctions-and-intersections-urls/">send me an SMS when the there&#8217;s a problem on my girlfriend&#8217;s route to work</a>, etc.</p>
<p>Every so often something happens on the internet and, rather than undertaking a specific action, you want to be prompted to undertake some appropriate action. Wouldn&#8217;t it be sweet if when these things happened you could have a virtual PA drop a note on your daily todo list?</p>
<p><strong>Twitter to the rescue!</strong></p>
<p>Sadly, Ifttt doesn&#8217;t have a Remember the milk channel (yet) but, like many web apps, Remember the milk has a <a href="http://twitter.com/rtm">twitter bot</a>. If you add this bot as a friend and <a href="https://www.rememberthemilk.com/services/twitter/">associate your RTM account with your twitter account</a> you are able to add things to your task list by sending the bot a direct message.</p>
<p>Ifttt has a twitter task, so all you need to do to add something to your task list when an action is triggered is begin the twitter message with &#8220;d rtm&#8221;, e.g.</p>
<blockquote><p>d rtm Write about Latakoo&#8217;s latest blog ^today #work</p></blockquote>
<p>You can use RTM markup in your message to control what list it goes to, set due dates etc.</p>
<p>My standard use case is to prompt me to write a blog post in response to a client updating their blog, or to tell me cover my car&#8217;s windscreen when it&#8217;s forecast to snow the next day. I&#8217;m sure there&#8217;s much more you can do with it!</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2012/01/19/life-hack-connecting-ifttt-to-remember-the-milk/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2012/01/19/life-hack-connecting-ifttt-to-remember-the-milk/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2012%2F01%2F19%2Flife-hack-connecting-ifttt-to-remember-the-milk%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2012/01/19/life-hack-connecting-ifttt-to-remember-the-milk/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Howto: Printing to any printer with an iPad/iPhone and Apple Airprint</title>
		<link>http://www.marcus-povey.co.uk/2011/11/11/howto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint/</link>
		<comments>http://www.marcus-povey.co.uk/2011/11/11/howto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint/#comments</comments>
		<pubDate>Fri, 11 Nov 2011 08:59:38 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[Software]]></category>
		<category><![CDATA[airprint]]></category>
		<category><![CDATA[apple]]></category>
		<category><![CDATA[avahi]]></category>
		<category><![CDATA[cups]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[ipad]]></category>
		<category><![CDATA[iphone]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[print]]></category>
		<category><![CDATA[printer]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=762</guid>
		<description><![CDATA[Apple Airprint is a technology (a zeroconf implementation under the bonnet) which allows apple devices to detect, configure and print without any overt configuration on the part of the user. The bad news is that in only works for a handful of printers natively. But never fear, Linux to the rescue! At this point I&#8217;m [...]]]></description>
			<content:encoded><![CDATA[<p><img src="http://www.marcus-povey.co.uk/wp-content/airprint.jpg" alt="" width="150" align="right" />Apple Airprint is a technology (a <a href="http://en.wikipedia.org/wiki/Zero_configuration_networking">zeroconf</a> implementation under the bonnet) which allows apple devices to detect, configure and print without any overt configuration on the part of the user.</p>
<p>The bad news is that in only works for a handful of printers natively. But never fear, Linux to the rescue!</p>
<p>At this point I&#8217;m going to assume that you have a Linux box somewhere on your network acting as a file and print server (pretty regular kit in most geek homes).</p>
<p><strong>Set up your printer</strong></p>
<p>The first step is to set up CUPS on your linux server and then installing the appropriate printer driver for your printer.</p>
<p>I won&#8217;t go into detail here as there are numerous guides out there on the wider web, but mostly this is a matter of installing cups and foomatic and then visiting the cups configuration website on the server (localhost:631 usually) and adding your printer.</p>
<p>Make sure that the printer is shared. Print a test page.</p>
<p>One gotcha I found is that my default configuration only allowed connections from the local machine, even though the printer was marked as shared (although frankly I was cheating since most of my computers printed to my server over a Samba relay, but that&#8217;s by the by).</p>
<p>Take a look at <code>/etc/cups/cupsd.conf</code> and make sure that <code></code> has an <code>Allow From</code> from your local network. E.g.</p>
<blockquote><p><code><br />
&lt;Location /&gt;<br />
Order Deny,Allow<br />
Deny From All<br />
Allow From 127.0.0.1<br />
Allow From 192.168.0.0/24<br />
&lt;/Location&gt;</code></p></blockquote>
<p><strong>Export the printer</strong></p>
<p>The next step is to install the zeroconf demon, which is called <a href="http://en.wikipedia.org/wiki/Avahi_(software)">Avahi</a>. This varies from system to system, but on debian this is pretty much a matter of <strong>apt-getting avahi-daemon</strong>. You may also want <strong>avahi-discover</strong> so you can browse the exported devices on your network.</p>
<p>Assuming you&#8217;ve correctly set up and shared your printers in CUPS the next step is to generate an avahi configuration for it. Thankfully, there&#8217;s a handy Python script called <a href="https://github.com/tjfontaine/airprint-generate">airprint-generate, available on github</a> which does much of the donkey work.</p>
<p>Copy the resultant file to <code>/etc/avahi/services</code> and restart the avahi demon. </p>
<p>If your printer is password protected, you will want to add a <code>&lt;txt-record&gt;air=username,password&lt;/txt-record&gt;</code> field to the file before doing so. Where username and password is the literal cleartext strings sent.</p>
<p><strong>Profit!!!</strong></p>
<p>Theoretically that should be it. After the configuration, and restarting the various demons involved your printer should be available to the various iOS devices kicking around your network. </p>
<p>Let me know if you have any questions!</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2011/11/11/howto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2011/11/11/howto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2011%2F11%2F11%2Fhowto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2011/11/11/howto-printing-to-any-printer-with-an-ipadiphone-and-apple-airprint/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Tips for building web services on Amazon AWS</title>
		<link>http://www.marcus-povey.co.uk/2010/11/17/tips-for-building-web-services-on-amazon-aws/</link>
		<comments>http://www.marcus-povey.co.uk/2010/11/17/tips-for-building-web-services-on-amazon-aws/#comments</comments>
		<pubDate>Wed, 17 Nov 2010 10:52:26 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[Software]]></category>
		<category><![CDATA[amazon]]></category>
		<category><![CDATA[aws]]></category>
		<category><![CDATA[ec2]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[s3]]></category>
		<category><![CDATA[services]]></category>
		<category><![CDATA[tips]]></category>
		<category><![CDATA[web]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=579</guid>
		<description><![CDATA[Amazon, who are most widely known for their online shop, has another project called Amazon Web Services (AWS) offering website authors a collection of very powerful hosting tools. This is not a sideline for Amazon &#8211; a recent conversation I had with an Amazon guy I met at a Barcamp revealed that in terms of revenue AWS [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://aws.amazon.com"><img title="Amazon Web Services" src="http://www.marcus-povey.co.uk/wp-content/logo_aws.gif" alt="" width="164" height="60" align="right" /></a><a href="http://www.amazon.com">Amazon</a>, who are most widely known for their <a href="http://www.amazon.co.uk">online shop</a>, has another project called <a href="http://aws.amazon.com">Amazon Web Services</a> (AWS) offering website authors a collection of very powerful <a href="http://aws.amazon.com/products/">hosting tools</a>.</p>
<p>This is not a sideline for <a href="http://www.amazon.com">Amazon</a> &#8211; a recent conversation I had with an Amazon guy I met at a <a href="http://barcamp.org/w/page/401306/BarCampOxford">Barcamp</a> revealed that in terms of revenue AWS exceeds the store &#8211; and it dramatically lowers the cost of entry for web authors. Using <a href="http://aws.amazon.com">AWS</a> it is possible to compete with the big boys without the need to mortgage your house for the data center and it solves many storage and scalability problems, what&#8217;s more &#8211; its pay as you go.</p>
<p>To be clear then, Amazon is now a hosting company, and its rather curious as to why people (<a href="http://www.amazon.com">Amazon</a> included) are not making a big deal of this.</p>
<p>Anywho, I&#8217;ve recently had the opportunity to work on a few projects which use <a href="http://aws.amazon.com">AWS</a> extensively, and since I did run into a few gotcha&#8217;s I&#8217;d thought I&#8217;d jot some notes down here, more as an <em>aide-mémoire</em> as anything else.</p>
<p><strong>Getting an AWS account</strong></p>
<p>Before you begin, you&#8217;re going to need an account. So, <a href="https://aws-portal.amazon.com/gp/aws/developer/registration/index.html">head over here and create one</a>.</p>
<p><strong>Creating a server</strong></p>
<p>You are going to want to create a server to run your code on. The service you need here is <a href="http://aws.amazon.com/ec2/">Amazon Elastic Compute Cloud (EC2)</a>, so sign up for EC2 from the <a href="https://console.aws.amazon.com/s3/home">AWS Management Console</a>.</p>
<p>You are going to have to select an EC2 Image to start from, and there are a number of them available. An <em>Image</em> (called AMIs) is a snapshot of your server, its disks and any software you want to run. Once configured you can start one or more <em>Instances</em> of it, which are your running servers.</p>
<p>Once you&#8217;ve configured your server, you can create your own image which you can share or use to boot other instances. This whole process is made <a href="http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?creating-an-ami-ebs.html">infinitely easier</a> by using an <a href="http://aws.amazon.com/ebs/">EBS</a> based image. EBS is a virtual disk, these can be added like normal disks to your server &#8211; sizes 1GB to 1TB. EBS backed images store data to these disks and so configuration changes are preserved, additionally new server images can be cloned with the click of a button.</p>
<p>You can add extra disks to your server, but they must be in the same availability zone (datacenter) as your EC2 image and currently can&#8217;t be shared between EC2 instances.</p>
<p>So:</p>
<ul>
<li>Save your sanity and use an EBS backed image &#8211; if you&#8217;re looking for a <a href="http://www.debian.org">Debian</a> or <a href="http://www.ubuntu.com">Ubuntu</a> based one, I recommend the ones created buy <a href="http://alestic.com">Alestic</a>. You can search for community managed images from within the management console, be sure to select EBS backed! For reference, the AMI I often use is identified as ﻿﻿ami-209e7349.</li>
<li>Grab an <a href="http://aws.amazon.com/articles/1346?_encoding=UTF8&amp;jiveRedirect=1">Elastic IP</a> and assign it to your image, point your domain at this.</li>
<li>Your goal is to build an AMI consisting of a ready to go install of your web project, so install the necessary software.</li>
<li>Note however, that you want to build your AMI so that you can run multiple instances of it in order to handle scalability. Since EBS disks can&#8217;t be shared you may have to change your architecture slightly:
<ul>
<li>Don&#8217;t store user data on the server &#8211; user icons etc &#8211; these can&#8217;t be shared across instances. Store these instead on <a href="http://aws.amazon.com/s3/">Amazon S3</a> and reference them directly using URL where possible. Note, temporary files like caches are probably ok.</li>
<li>If you use a database, use <a href="http://www.mysql.com">MySQL</a>, but only install the client libraries on your image &#8211; you won&#8217;t be using a local server (see below)!</li>
</ul>
</li>
</ul>
<p>Once you&#8217;ve configured your server create an AMI out of it, this can then be booted multiple times.</p>
<p><strong>Using a database</strong></p>
<p>The database service you will most likely want to use is called <a href="http://aws.amazon.com/rds/">Amazon Relational Database Service (RDS)</a> which can function as a drop in replacement for MySQL.</p>
<p>Sign up for it using the management tool, create a database (making note of the passwords and user names you assign) and allow access from your running server instance(s) via the security settings.</p>
<p>The database is running on its own machine, so make a note of the host name in the properties &#8211; you will need to pass this in your connect string.</p>
<p><strong>Conclusion</strong></p>
<p>AWS is a powerful tool, if you use it right. There is a lot more complexity to cover of course, but this should serve as a start.</p>
<p>I&#8217;ll cover the more advanced scalability stuff later when I actually get to use them in anguish (and get the time to write about it!)</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2010/11/17/tips-for-building-web-services-on-amazon-aws/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2010/11/17/tips-for-building-web-services-on-amazon-aws/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2010%2F11%2F17%2Ftips-for-building-web-services-on-amazon-aws%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2010/11/17/tips-for-building-web-services-on-amazon-aws/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>How to set up ProFTP, MySQL and Virtual Users</title>
		<link>http://www.marcus-povey.co.uk/2010/06/15/proftp-mysql-virtual-users-howto/</link>
		<comments>http://www.marcus-povey.co.uk/2010/06/15/proftp-mysql-virtual-users-howto/#comments</comments>
		<pubDate>Tue, 15 Jun 2010 10:59:45 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[Software]]></category>
		<category><![CDATA[ftp]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[php]]></category>
		<category><![CDATA[proftp]]></category>
		<category><![CDATA[ubuntu]]></category>
		<category><![CDATA[virtual users]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=551</guid>
		<description><![CDATA[ProFTP is a configurable FTP server available on most *nix platforms. I recently had the need to get this working and authenticating off a PHP maintained MySQL backend, and this post is primarily to aid my own memory should I ever have to do it again. Installing ProFTP In order to use MySQL as a [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.proftp.org"><img src="http://www.marcus-povey.co.uk/wp-content/proftpd.png" alt="" width="150" align="right" />ProFTP</a> is a configurable FTP server available on most *nix platforms.</p>
<p>I recently had the need to get this working and authenticating off a PHP maintained MySQL backend, and this post is primarily to aid my own memory should I ever have to do it again.</p>
<p><strong>Installing ProFTP</strong></p>
<p>In order to use MySQL as a back end you need to install some packages. If you&#8217;re using a <a href="http://www.debian.org">Debian</a> based distro like <a href="http://www.ubuntu.com">Ubuntu</a>, this is easy:</p>
<blockquote><p><code>apt-get install mysql-server proftpd proftpd-mod-mysql</code></p></blockquote>
<p><strong>The database schema</strong></p>
<p>Next, you need to install the database schema to store your users and passwords.</p>
<blockquote><p><code>CREATE TABLE IF NOT EXISTS users (<br />
userid varchar(30) NOT NULL default '',<br />
passwd <strong>varchar(128)</strong> NOT NULL default '',<br />
uid int(11) default NULL,<br />
gid int(11) default NULL,<br />
homedir varchar(255) default NULL,<br />
shell varchar(255) default NULL,<br />
UNIQUE KEY uid (uid),<br />
UNIQUE KEY userid (userid)<br />
) TYPE=MyISAM;</code></p>
<p><code>CREATE TABLE IF NOT EXISTS groups (<br />
groupname varchar(30) NOT NULL default '',<br />
gid int(11) NOT NULL default '0',<br />
members varchar(255) default NULL<br />
) TYPE=MyISAM;</code></p></blockquote>
<p>One important thing to note here &#8211; that caused me a fair amount of hair pulling when I tried to use encrypted passwords &#8211; is that the password field shown in many howtos on the internet is much too short. This causes the hashed password to be quietly truncated by MySQL when saved.</p>
<p>This results in a somewhat misleading &#8220;<strong>No such user found</strong>&#8221; error to appear in the logs when using encrypted passwords.</p>
<p>To end all argument I&#8217;ve allowed passwords up to 128 chars, but this field could probably be a good deal shorter.</p>
<p>The user table looks much like /etc/passwd and is largely self explanatory. The <strong>uid</strong> &amp; <strong>gid</strong> fields correspond to a system user in most cases, but since we&#8217;re using virtual users they can largely be ignored. <strong>Homedir </strong>points to a location which will serve as the user&#8217;s default directory. <strong>Shell</strong> is largely unused and can be set to <strong>/bin/false</strong> or similar.</p>
<p><strong>Configuring ProFTP</strong></p>
<p>Next, you need to make some changes to the ProFTP configuration files stored in /etc/proftpd. While doing this it is handy to run proftp in debug mode from the console:</p>
<blockquote><p><code>proftpd -nd6</code></p></blockquote>
<p><strong><em>proftpd.conf</em></strong></p>
<ol>
<li>Make sure the AuthOrder line looks like:<br />
<blockquote><p><code>AuthOrder mod_sql.c</code></p></blockquote>
</li>
<li>Ensure that the following line is uncommented:<br />
<blockquote><p><code>Include /etc/proftpd/sql.conf</code></p></blockquote>
</li>
<li>For belts and braces I&#8217;ve included the following at the end, although I&#8217;m not entirely sure it&#8217;s strictly required:<br />
<blockquote><p><code>&lt;IfModule mod_auth_pam.c&gt;<br />
AuthPAM off<br />
&lt;/IfModule&gt;<br />
</code></p></blockquote>
</li>
<li>Our users don&#8217;t need a valid shell, so:<br />
<blockquote><p><code>RequireValidShell off</code></p></blockquote>
</li>
</ol>
<p><em><strong>modules.conf</strong></em></p>
<ol>
<li>Make sure the following lines are uncommented:<br />
<blockquote><p><code>LoadModule mod_sql.c<br />
LoadModule mod_sql_mysql.c</code></p></blockquote>
</li>
</ol>
<p><strong><em>sql.conf</em></strong></p>
<ol>
<li>Set your SQL backend and ensure that authentication is turned on:<br />
<blockquote><p><code>SQLBackend mysql<br />
SQLEngine on<br />
SQLAuthenticate on</code></p></blockquote>
</li>
<li>Tell proftp how passwords are stored. You have a number of options here, but since I was using mysql&#8217;s PASSWORD function, I&#8217;ll defer to the backend.<br />
<blockquote><p><code>SQLAuthTypes backend</code></p></blockquote>
</li>
<li>Tell proftp how to connect to your database by providing the required connection details, ensure that the user has full access to these tables.<br />
<blockquote><p><code>SQLConnectInfo <em>database</em>@<em>host</em> <em>user</em> <em>password</em></code></p></blockquote>
</li>
<li>Define your table structure in the format <em>tablename</em> <em>fields&#8230;.</em><br />
<blockquote><p><code>SQLUserInfo users userid passwd uid gid homedir shell<br />
SQLGroupInfo groups groupname gid members</code></p></blockquote>
</li>
</ol>
<p><strong>Adding users</strong></p>
<p>I manage users from within a <a href="http://php.net">PHP</a> web application that I&#8217;m developing, but in a nutshell adding FTP users from this point is a simple insert statement looking something like:</p>
<blockquote><p><code>mysql_query("REPLACE INTO users<br />
(userid, passwd, uid, gid, homedir, shell)<br />
VALUES<br />
('$userid', PASSWORD('$password'), $uid, $gid, '$homedir', '$shell')");</code></p></blockquote>
<p>Have fun!</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2010/06/15/proftp-mysql-virtual-users-howto/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2010/06/15/proftp-mysql-virtual-users-howto/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2010%2F06%2F15%2Fproftp-mysql-virtual-users-howto%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2010/06/15/proftp-mysql-virtual-users-howto/feed/</wfw:commentRss>
		<slash:comments>12</slash:comments>
		</item>
		<item>
		<title>Using an XBOX 360 wireless controller with XBMC on Ubuntu</title>
		<link>http://www.marcus-povey.co.uk/2009/12/29/using-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu/</link>
		<comments>http://www.marcus-povey.co.uk/2009/12/29/using-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu/#comments</comments>
		<pubDate>Tue, 29 Dec 2009 17:48:30 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[Software]]></category>
		<category><![CDATA[360]]></category>
		<category><![CDATA[controller]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[ubuntu]]></category>
		<category><![CDATA[wireless]]></category>
		<category><![CDATA[xbmc]]></category>
		<category><![CDATA[xbox]]></category>
		<category><![CDATA[xbox 360]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=357</guid>
		<description><![CDATA[This Christmas I finally bit the bullet and treated myself to a shiny XBox 360. Ostensibly this was so that I could experiment with console development, but mostly I have used it to play Gears of War. In a slight departure from what I usually talk about, I thought I&#8217;d quickly jot down how I [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.flickr.com/photos/miskan/25241898/"><img src="http://farm1.static.flickr.com/22/25241898_aa45084bd8_m.jpg" border="0" alt="" align="right" /></a>This Christmas I finally bit the bullet and treated myself to a shiny XBox 360. Ostensibly this was so that I could experiment with console development, but mostly I have used it to play <a href="http://en.wikipedia.org/wiki/Gears_of_War">Gears of War</a>.</p>
<p>In a slight departure from what I usually talk about, I thought I&#8217;d quickly jot down how I got the wireless controller to work with my Ubuntu <a href="http://xbmc.org/">XBMC</a> media PC.</p>
<p>The wireless controller provides a slightly more usable remote than my <a href="http://remote.collect3.com.au/">iPhone</a> (which must first be unlocked making quick pauses impossible) or rather flaky wireless keyboard, so hopefully this will be useful to someone.</p>
<p><strong>Getting started<br />
</strong></p>
<p>My media PC currently runs Ubuntu Karmic with XBMC. To begin with you will need to install the XBox kernel driver (already installed on Karmic).</p>
<p>Most importantly however, you will need to get yourself a <a href="http://support.microsoft.com/kb/933711?sd=xbox">XBox wireless gaming receiver for Windows</a> &#8211; which I got included with my second controller. Xbox controllers do not use standard bluetooth, so you can&#8217;t just pair in the normal way using your existing hardware.</p>
<p>This <a href="https://help.ubuntu.com/community/Xbox360Controller">howto has some more info</a>&#8230;</p>
<p><strong>Configuring XBMC</strong></p>
<p>Assuming you have your module installed and controller paired you will need to tell XBMC about it by configuring a keyfile:</p>
<ol>
<li>I used <a href="https://xbmc.svn.sourceforge.net/svnroot/xbmc/trunk/system/keymaps/joystick.Microsoft.Xbox.Controller.S.xml">this keyfile as a starting point</a>. Download and save it as <code>~/.xbmc/userdata/keymaps/Keymap.xml</code> (note case).</li>
<li>Find out what your computer thinks the controller is by looking at the output from: <code>cat /proc/bus/input/devices</code> &#8211; you want a <code>"Name"</code> that says something like <code>"Xbox 360 Wireless Receiver"</code></li>
<li>Replace all occurrences of <code>"Microsoft Xbox Controller S"</code> with this value.</li>
</ol>
<p>At this point if you start XBMC it should respond to the controller. If you are lucky this is all you will have to do, however for me I had to mess around with the key bindings a bit since the example keymap file didn&#8217;t match my controller exactly.</p>
<p>If this happens to you there&#8217;s not much I can suggest other than to bind one key at a time, restart XBMC and see what button that maps to then repeat until all your keys are mapped. I&#8217;m sure there must be an easier way that I&#8217;ve overlooked, feel free to comment below!</p>
<p>For what it&#8217;s worth, here is my modified (but somewhat incomplete) key file which has largely sensible bindings. Hack away to get it working how you like.</p>
<p>» <a href="http://www.marcus-povey.co.uk/wp-content/Keymap.zip">Modified Keymap</a></p>
<p><small><em>Image &#8220;<a href="http://www.flickr.com/photos/miskan/25241898/">XBMC</a>&#8221; by <a href="http://www.flickr.com/photos/miskan/">Miskan</a></em></small></p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2009/12/29/using-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2009/12/29/using-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2009%2F12%2F29%2Fusing-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2009/12/29/using-an-xbox-360-wireless-controller-with-xbmc-on-ubuntu/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Running Elgg on a MySQL cluster</title>
		<link>http://www.marcus-povey.co.uk/2009/09/21/running-elgg-on-a-mysql-cluster/</link>
		<comments>http://www.marcus-povey.co.uk/2009/09/21/running-elgg-on-a-mysql-cluster/#comments</comments>
		<pubDate>Mon, 21 Sep 2009 19:01:49 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[elgg]]></category>
		<category><![CDATA[#ue]]></category>
		<category><![CDATA[cluster]]></category>
		<category><![CDATA[clustering]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[mysql-proxy]]></category>
		<category><![CDATA[patch]]></category>
		<category><![CDATA[scalability]]></category>
		<category><![CDATA[tutorial]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=308</guid>
		<description><![CDATA[I have recently been exploring some aspects of the Elgg scalability question by exploring how easy it would be to get the latest version of Elgg (1.6) running on a MySQL cluster. In this article I will document the process, but first I should point out: This is highly experimental and not endorsed in any [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://en.wikipedia.org/wiki/File:Us-nasa-columbia.jpg"><img src="http://upload.wikimedia.org/wikipedia/commons/3/3d/Us-nasa-columbia.jpg" alt="" width="200" align="right" /></a>I have recently been exploring some aspects of the <a href="http://www.marcus-povey.co.uk/2009/01/04/elgg-scalability/">Elgg scalability</a> question by exploring how easy it would be to get the latest version of <a href="http://elgg.org/getelgg.php?forward=elgg1.6.1.zip">Elgg (1.6)</a> running on a <a href="http://en.wikipedia.org/wiki/MySQL_Cluster">MySQL cluster</a>.</p>
<p>In this article I will document the process, but first I should point out:</p>
<ul>
<li>This is highly experimental and not endorsed in any way.</li>
<li>It is built against <a href="http://elgg.org/getelgg.php?forward=elgg1.6.1.zip">Elgg 1.6.1</a></li>
<li>This is not canonical and doesn&#8217;t reflect anything to do with the roadmap</li>
<li>This has not been extensively tested so <em>caveat emptor.</em></li>
</ul>
<h2>Setting up the cluster</h2>
<p>The first step is to set up the cluster on your equipment.</p>
<p>A MySQL cluster consists of a management node and several data nodes connected together by a network. Because I was running rather low on hardware, I cheated here and created each node as a <a href="http://www.virtualbox.org/">Virtual Box</a> image on my laptop &#8211; but the principle is the same.</p>
<p>Each node is an <a href="http://www.ubuntu.com/">Ubuntu</a> install (although you can use pretty much any OS) with two (virtual) network cards, one connected to the wider network (to install packages) and another on an internal network. If you do this for real you should consider removing the internet facing card once you&#8217;ve set everything up since a cluster isn&#8217;t secure enough to be run on the wider internet.</p>
<p>In my test configuration I had three nodes with name/internal IP as follows:</p>
<ul>
<li>HHCluster1/192.168.2.1 &#8211; Management node &amp; web server</li>
<li>HHCluster2/192.168.2.2 &#8211; First data node</li>
<li>HHCluster3/192.168.2.3 &#8211; Second data node</li>
</ul>
<p><strong>HHCluster1 &#8211; The management node</strong></p>
<p>Install mysql, apache etc. This should be a simple matter of apt-getting the relevant packages. Clustering (ndb) support is built into the version of mysql bundled with Ubuntu, but this may not be the case universally so check!</p>
<p>You need to create a file in /etc/mysql/ called <code>ndb_mgmd.cnf</code>, this should contain the following:</p>
<blockquote><p><code><br />
[NDBD DEFAULT]<br />
NoOfReplicas=2 # How many nodes you have<br />
DataMemory=80M    # How much memory to allocate for data storage (change for larger clusters)<br />
IndexMemory=18M   # How much memory to allocate for index storage (change for larger clusters)<br />
[MYSQLD DEFAULT]<br />
[NDB_MGMD DEFAULT]<br />
[TCP DEFAULT]</code></p>
<p><code>[NDB_MGMD]<br />
HostName=192.168.2.1 # IP address of this system</code></p>
<p><code># Now we describe each node on the system</code></p>
<p><code># First data node<br />
HostName=192.168.2.2<br />
DataDir=/var/lib/mysql-cluster<br />
BackupDataDir=/var/lib/mysql-cluster/backup<br />
DataMemory=512M<br />
[NDBD]<br />
# Second data node node<br />
HostName=192.168.2.3<br />
DataDir=/var/lib/mysql-cluster<br />
BackupDataDir=/var/lib/mysql-cluster/backup<br />
DataMemory=512M<br />
</code></p>
<p><code>#one [MYSQLD] per data storage node<br />
[MYSQLD]<br />
[MYSQLD]</code></p></blockquote>
<p><strong>Data nodes (HHCluster2 &amp; 3)</strong><br />
You must now configure your data nodes:</p>
<ol>
<li>Create the data directories, as root type:<br />
<blockquote><p><code>mkdir -p /var/lib/mysql-cluster/backup<br />
chown -R mysql:mysql /var/lib/mysql-cluster</code></p></blockquote>
</li>
<li>Edit your /etc/mysql/my.cnf and add the following to the [mysqld] section:<br />
<blockquote><p><code>ndbcluster<br />
# Replace the following with the IP address of your management server<br />
ndb-connectstring=192.168.2.1</code></p></blockquote>
</li>
<li>Again in /etc/mysql/my.cnf uncomment and edit the [MYSQL_CLUSTER] section so it contains the location of your management server:<br />
<blockquote><p><code>[MYSQL_CLUSTER]<br />
ndb-connectstring=192.168.2.1</code></p></blockquote>
</li>
<li>You need to create your database on each node (this is because clustering operates on a table level rather than a database level):<br />
<blockquote><p><code>CREATE DATABASE elggcluster;</code></p></blockquote>
</li>
</ol>
<p><strong>Starting the cluster</strong></p>
<ol>
<li>Start the management node:<br />
<blockquote><p><code>/etc/init.d/mysql-ndb-mgm start</code></p></blockquote>
</li>
<li>Start your data nodes:<br />
<blockquote><p><code>/etc/init.d/mysql restart<br />
/etc/init.d/mysql-ndb restart</code></p></blockquote>
</li>
</ol>
<p><strong>Verifying the cluster</strong><br />
You should now have the cluster up and running, you can verify this by logging into your management node and typing <code>show</code> in <code>ndb_mgm</code>.</p>
<p style="text-align: center;"><a href="http://www.marcus-povey.co.uk/wp-content/screenshot.png"><img class="aligncenter" src="http://www.marcus-povey.co.uk/wp-content/screenshot.png" border="0" alt="" width="400" /></a></p>
<p style="text-align: left;"><strong>A word on access&#8230;<br />
</strong></p>
<p style="text-align: left;">The cluster is now set up and will replicate tables (created with the ndbcluster engine &#8211; more on that later), but that is only useful to a point. Right now we don&#8217;t have a single endpoint to direct queries to, so this direction needs to be done at the application level.</p>
<p style="text-align: left;">We could take advantage of Elgg&#8217;s built in split read and writes, but this would only allow us to use a maximum of two nodes. A better solution would be to use a load balancer here such as <a href="http://www.ultramonkey.org/">Ultramonkey</a> to direct the query to the appropriate server allowing us to scale much further.</p>
<p style="text-align: left;">I didn&#8217;t really have time to get into this, so I am using the somewhat simpler <a href="http://forge.mysql.com/wiki/MySQL_Proxy">mysql-proxy</a>.</p>
<ol>
<li>On HHCluster1 install and run mysql-proxy:<br />
<blockquote><p><code>apt-get install mysql-proxy<br />
mysql-proxy --proxy-backend-addresses=192.168.2.2:3306 --proxy-backend-addresses=192.168.2.2:3306</code></p></blockquote>
</li>
<li>On your data nodes edit your /etc/mysql/my.cnf file. Find <code>bind-address</code> and change it&#8217;s IP to the node&#8217;s IP address. Also ensure that you have commented out any occurrence of <code>skip-networking.</code></li>
<li>Again on your client nodes, log in to mysql and grant access to your cluster table to a user on HHCluster1 &#8211; for example:<br />
<blockquote><p><code>GRANT ALL ON elggcluster.* TO `root`@`HHCluster1.local` IDENTIFIED BY '[some password]'</code></p></blockquote>
</li>
</ol>
<h2>Installing elgg</h2>
<p>Unfortunately as it stands, you need to make some code changes to the vanilla version of Elgg in order for it to work in a clustered environment. These changes are necessary because of the restrictions placed on us by the ndbcluster engine.</p>
<p>Two things in particular cause us problems &#8211; ndbcluster doesn&#8217;t support FULLTEXT indexes, and it also doesn&#8217;t support indexes over TEXT or BLOB fields.</p>
<p>FULLTEXT is for searching and is largely not used in the vanilla install of elgg, so I removed them. Equally, most indexes blobs one can live without, the exception being on the metastrings table.</p>
<p>Metastrings is accessed a lot, so the index is critical. Therefore I added an extra varchar field which we&#8217;ll modify the code to include the first 50 characters of the indexed text &#8211; this is equivalent to the existing index:</p>
<blockquote><p><code>CREATE TABLE `prefix_metastrings` (<br />
`id` int(11) NOT NULL auto_increment,<br />
`string` TEXT NOT NULL,<br />
<strong>`string_index` varchar(50) NOT NULL,</strong><br />
PRIMARY KEY (`id`),<br />
<strong>KEY `string_index` (`string_index`)</strong><br />
) ENGINE=ndbcluster DEFAULT CHARSET=utf8;</code></p></blockquote>
<p>And the modified query:</p>
<blockquote><p><code>$row = get_data_row("SELECT * from {$CONFIG-&gt;dbprefix}metastrings where string=$cs'$string' and string_index='$string_index' limit 1");</code></p></blockquote>
<p>Mysql&#8217;s optimiser checks the index first so this doesn&#8217;t lose a significant amount of efficiency (at least according to the explain command).</p>
<p>» <a href="http://www.marcus-povey.co.uk/wp-content/elgg-mysql-cluster.sql">Modified schema</a></p>
<p>The next problem is that the system log currently uses INSERT DELAYED to insert the log data. This is also not supported under the clustered engine.</p>
<p>There are a number of approaches we could take including using <a href="http://reference.elgg.org/database_8php.html#d03f827016fde8428fb40f3ee87aa887">Elgg&#8217;s delayed write functionality</a> or writing a plugin which replaces and logs to a different location.</p>
<p>For the purposes of this test I decided to just comment out the code in <a href="http://reference.elgg.org/system__log_8php.html#3ef1dd1a4a83c58012ad92ef2841b356">system_log()</a>.</p>
<p><strong>What won&#8217;t work</strong><br />
Currently there are a couple of core things that won&#8217;t work under these changes, here is a by no means complete summary:</p>
<ul>
<li>The system log (as previously described). This isn&#8217;t too much of a show stopper as the river code introduced in Elgg 1.5 no longer uses this.</li>
<li>The log rotate plugin as this attempts to copy the table into the archive engine type and we can&#8217;t guarantee which node it will be executed on in this scenario.</li>
<li>Any third party plugins which attempt to access the metastrings table directly (of which there should be none as direct table access is a big no no!)</li>
</ul>
<p>Anyway, here is a patch I made against the released version of 1.6.1 with all the code changes I made. Once you have applied this patch to your Elgg install you should be able to proceed with the normal Elgg install.</p>
<p>Let me know any feedback you may have!</p>
<p>» <a href="http://www.marcus-povey.co.uk/wp-content/ElggCluster1.6.1.patch">Elgg Clustering patch for Elgg 1.6.1</a></p>
<p><small><em>Top image &#8220;Birds-eye view of the 10,240-processor SGI Altix supercomputer housed at the NASA Advanced Supercomputing facility.&#8221;</em></small></p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2009/09/21/running-elgg-on-a-mysql-cluster/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2009/09/21/running-elgg-on-a-mysql-cluster/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2009%2F09%2F21%2Frunning-elgg-on-a-mysql-cluster%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2009/09/21/running-elgg-on-a-mysql-cluster/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Using Elgg&#8217;s REST-like API</title>
		<link>http://www.marcus-povey.co.uk/2009/08/25/using-elggs-rest-like-api/</link>
		<comments>http://www.marcus-povey.co.uk/2009/08/25/using-elggs-rest-like-api/#comments</comments>
		<pubDate>Tue, 25 Aug 2009 14:20:04 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[elgg]]></category>
		<category><![CDATA[#ue]]></category>
		<category><![CDATA[api]]></category>
		<category><![CDATA[endpoint]]></category>
		<category><![CDATA[get]]></category>
		<category><![CDATA[handler]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[post]]></category>
		<category><![CDATA[rest]]></category>
		<category><![CDATA[token]]></category>
		<category><![CDATA[tutorial]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=272</guid>
		<description><![CDATA[Another one of Elgg&#8216;s less documented but very powerful features is the ability to expose functionality from the core and user modules in a standard way via a REST like API. This gives you the opportunity to develop interoperable web services and provide them to the users of your site, all in a standardised way. [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://xkcd.com/262/"><img title="in_ur_reality" src="http://www.marcus-povey.co.uk/wp-content/in_ur_reality.png" alt="in_ur_reality" width="200" align="right" /></a>Another one of <a href="http://www.elgg.org">Elgg</a>&#8216;s less documented but very powerful features is the ability to expose functionality from the core and user modules in a standard way via a REST like API.</p>
<p>This gives you the opportunity to develop interoperable web services and provide them to the users of your site, all in a standardised way.</p>
<h2>The endpoint</h2>
<p>To make an API call you must direct your query at a special URL. This query will be either a GET or a POST query (depending on the command you are executing), the specific endpoint you use depends on the format you want the return value returned in.</p>
<p>The endpoint:</p>
<blockquote><p><code>http://yoursite.com/pg/api/<strong>[protocol]</strong>/<strong>[return format]</strong>/</code></p></blockquote>
<p>Where:</p>
<ul>
<li><strong>[protocol]</strong> is the protocol being used, in this case and for the moment only &#8220;rest&#8221; is supported.</li>
<li><strong>[return format]</strong> is the format you want your information returned in, either &#8220;php&#8221;, &#8220;json&#8221; or &#8220;xml&#8221;.</li>
</ul>
<p>This endpoint should then be passed the method and any parameters as GET variables, so for example:</p>
<blockquote><p><code>http://yoursite.com/pg/api/rest/xml/?method=test.test&amp;myparam=foo&amp;anotherparam=bar</code></p></blockquote>
<p>Would pass &#8220;foo&#8221; and &#8220;bar&#8221; as the given named parameters to the function &#8220;test.test&#8221; and return the result in XML format.</p>
<p>Notice here also that the API uses the &#8220;PG&#8221; page handler extension, this means that it would be a relatively simple matter to add a new API protocol or replace the entire API subsystem in a module &#8211; should you be so inclined.</p>
<p><strong>Return result</strong></p>
<p>The result of the api call will be an entity encoded in your chosen format.</p>
<p>This entity will have a &#8220;status&#8221; parameter &#8211; zero for success, non-zero denotes an error. Result data will be in the &#8220;result&#8221; parameter. You may also receive some messages and debug information.</p>
<h2>Exporting a function</h2>
<p>Any Elgg function &#8211; core or module &#8211; can be exposed via the API, all you have to do is declare it using <code><a href="http://reference.elgg.org/api_8php.html#3ee7de907c28ff90ec8b7ffc52254484">expose_function()</a></code> from within your code, passing the method name, handler and any parameters (note that these parameters must be declared in the same order as they appear in your function).</p>
<p><strong>Listing functions</strong></p>
<p>You can see a list of all registered functions using the built in api command &#8220;system.api.list&#8221;, this is also a useful test to see if your client is configured correctly.</p>
<p>E.g.</p>
<blockquote><p><code>http://yoursite.com/pg/api/rest/xml/?method=system.api.list</code></p></blockquote>
<h2>Authorising and authenticating</h2>
<p>Most commands will require some form of authorisation in order to function. There are two main types of authorisation &#8211; protocol level which determines whether a given client is permitted to connect, and user level where a command whereby a user requires a special token in lieu of a username and password.</p>
<p><strong>Protocol level authentication</strong><br />
Protocol level authentication is a way to ensure that commands only come from approved clients for which you have previously given keys. This is in keeping with many web based API systems and permits you to disconnect clients who abuse your system, or track usage for accountancy purposes.</p>
<p>The client must send a <a href="http://en.wikipedia.org/wiki/HMAC">HMAC</a> signature together with a set of special HTTP headers when making a call. This ensures that the API call is being made from the stated client and that the data has not been tampered with.</p>
<p>Eagle-eyed readers with long memories will see a lot of similarity with the <a href="http://www.marcus-povey.co.uk/2008/02/21/getting-started-with-the-elggvoices-api-part-1-api-basics/">ElggVoices API</a> I wrote about previously.</p>
<p>The HMAC must be constructed over the following data:</p>
<ul>
<li>The Secret Key provided by the target Elgg install (as provided easily by the APIAdmin plugin).</li>
<li>The current unix time in microseconds as a floating point decimal, produced my microtime(true).</li>
<li>Your API key identifying you to the Elgg api server (companion to your secret key).</li>
<li>URLEncoded string representation of any GET variable parameters, eg &#8220;method=test.test&amp;foo=bar&#8221;</li>
<li>If you are sending post data, the hash of this data.</li>
</ul>
<p>Some extra information must be added to the HTTP header in order for this data to be correctly processed:</p>
<ul>
<li><strong>X-Elgg-apikey</strong> &#8211; The API key (not the secret key!)</li>
<li><strong>X-Elgg-time</strong> &#8211; Microtime used in the HMAC calculation</li>
<li><strong>X-Elgg-hmac</strong> &#8211; The HMAC as hex characters.</li>
<li><strong>X-Elgg-hmac-algo</strong> &#8211; The algorithm used in the HMAC calculation &#8211; eg, sha1, md5 etc</li>
</ul>
<p>If you are sending POST data you must also send:</p>
<ul>
<li><strong>X-Elgg-posthash</strong> &#8211; The hash of the POST data.</li>
<li><strong>X-Elgg-posthash-algo</strong> &#8211; The algorithm used to produce the POST data hash &#8211; eg, md5.</li>
<li><strong>Content-type</strong> &#8211; The content type of the data you are sending (if in doubt use <code>application/octet-stream</code>).</li>
<li><strong>Content-Length</strong> &#8211; The length in bytes of your POST data.</li>
</ul>
<p>Much of this will be handled for you if you use the <a href="http://reference.elgg.org/api_8php.html#8b98a982e4462bfa0c752e124bb98c0c">built in Elgg API Client</a>.</p>
<p><strong>User level tokens</strong></p>
<p>User level tokens are used to identify a specific user on the target system, in much the same way as if they were to log in with their user name and password, but without the need to send this for every API call.</p>
<p>Tokens are time limited, and so it will be necessary for your client to periodically refresh the token they use to identify the user.</p>
<p>Tokens are generated by using the API command &#8220;auth.gettoken&#8221; and passing the username and password as parameters, eg:</p>
<blockquote><p><code>http://yoursite.com/pg/api/rest/xml/?method=auth.gettoken&amp;username=foo&amp;password=bar</code></p></blockquote>
<p><strong>Anonymous methods</strong><br />
Anonymous methods (such as &#8220;system.api.list&#8221;) can be executed without any form of authentication, thus accepting connections from any client and regardless of whether they provide a user token. This is useful in certain situations and it goes without saying that you don&#8217;t expose sensitive functionality this way.</p>
<p>To do so set $anonymous=true in your call to expose_function().</p>
<p><small>Image “<a href="http://xkcd.com/262/">In UR Reality</a>” by <a href="http://www.xkcd.com">XKCD</a></small></p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2009/08/25/using-elggs-rest-like-api/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2009/08/25/using-elggs-rest-like-api/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2009%2F08%2F25%2Fusing-elggs-rest-like-api%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2009/08/25/using-elggs-rest-like-api/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
		<item>
		<title>Its a question of time</title>
		<link>http://www.marcus-povey.co.uk/2008/10/27/its-a-question-of-time/</link>
		<comments>http://www.marcus-povey.co.uk/2008/10/27/its-a-question-of-time/#comments</comments>
		<pubDate>Mon, 27 Oct 2008 18:52:01 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[elgg]]></category>
		<category><![CDATA[#ue]]></category>
		<category><![CDATA[cron]]></category>
		<category><![CDATA[howto]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=107</guid>
		<description><![CDATA[Sometimes things need to be done without user interaction &#8211; for example, database optimisation or log rotation. For this, Elgg has a cron endpoint. Cron is a unix tool which executes commands at a specific time of day (other operating systems have similar tools). This keys off a file called a crontab &#8211; an example [...]]]></description>
			<content:encoded><![CDATA[<p>Sometimes things need to be done without user interaction &#8211; for example, database optimisation or log rotation.</p>
<p>For this, Elgg has a cron endpoint.</p>
<p>Cron is a unix tool which executes commands at a specific time of day (other operating systems have similar tools). This keys off a file called a crontab &#8211; an example is given file is included and called <code>crontab.example</code>.</p>
<p>The crontab calls simplified yet powerful cron endpoint &#8211; <code>http://yoursite/pg/cron/<strong>PERIOD</strong></code>, where <strong>PERIOD</strong> is one of the following:</p>
<ul>
<li><strong>reboot</strong> &#8211; Execute on system reboot</li>
<li><strong>minute</strong> &#8211; Execute every minute</li>
<li><strong>fiveminute</strong> &#8211; Execute every five minutes</li>
<li><strong>fifteenmin</strong> &#8211; Execute every fifteen minutes</li>
<li><strong>halfhour</strong> &#8211; Execute every half hour</li>
<li><strong>hourly</strong> &#8211; Execute once every hour</li>
<li><strong>daily</strong> &#8211; Execute every day</li>
<li><strong>weekly</strong> &#8211; Execute weekly</li>
<li><strong>monthly</strong> &#8211; Execute once a month</li>
<li><strong>yearly</strong> &#8211; Execute every year</li>
</ul>
<p>When these endpoints are triggered by your crontab a plugin hook is triggered. To make use of this, register a plugin hook as follows:</p>
<blockquote><p><code>register_plugin_hook('cron', <strong>PERIOD</strong>, 'my_cron_handler');</code></p></blockquote>
<p>Where PERIOD is one of the key words listed above. Here is some sample code using Cron &#8211; in this case it is taken from the system log rotation module I added to SVN today.</p>
<blockquote><p><code>&lt;?php<br />
/**<br />
* Elgg log rotator.<br />
*<br />
* @package ElggLogRotate<br />
* @license http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU Public License version 2<br />
* @author Curverider Ltd<br />
* @copyright Curverider Ltd 2008<br />
* @link http://elgg.com/<br />
*/</code></p>
<p><code>/**<br />
* Initialise the plugin.<br />
*<br />
*/<br />
function logrotate_init()<br />
{<br />
$period = get_plugin_setting('period','logrotate');<br />
switch ($period)<br />
{<br />
case 'weekly':<br />
case 'monthly' :<br />
case 'yearly' :<br />
break;<br />
default: $period = 'monthly';<br />
}</code></p>
<p><code>// Register cron hook<br />
register_plugin_hook('cron', $period, 'logrotate_cron');<br />
}</code></p>
<p><code>/**<br />
* Trigger the log rotation.<br />
*<br />
*/<br />
function logrotate_cron($hook, $entity_type, $returnvalue, $params)<br />
{<br />
$resulttext = elgg_echo("logrotate:logrotated");<br />
if (!archive_log())<br />
$resulttext = elgg_echo("logrotate:lognotrotated");</code></p>
<p><code>return $returnvalue . $resulttext;<br />
}<br />
</code><br />
<code>// Initialise plugin<br />
register_elgg_event_handler('init','system','logrotate_init');<br />
?&gt;</code></p></blockquote>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2008/10/27/its-a-question-of-time/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2008/10/27/its-a-question-of-time/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2008%2F10%2F27%2Fits-a-question-of-time%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2008/10/27/its-a-question-of-time/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Adding to the River in an Elgg 1 plugin</title>
		<link>http://www.marcus-povey.co.uk/2008/06/09/adding-to-the-river-in-an-elgg-1-plugin/</link>
		<comments>http://www.marcus-povey.co.uk/2008/06/09/adding-to-the-river-in-an-elgg-1-plugin/#comments</comments>
		<pubDate>Mon, 09 Jun 2008 23:04:19 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[elgg]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[river]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=42</guid>
		<description><![CDATA[In Elgg 1, we will finally have native support for a River &#8211; that is, a stream of short updates of what you and your friends are up to on your profile. Here is a short post explaining how you as a plugin writer could add river reporting to your code! Events The key to [...]]]></description>
			<content:encoded><![CDATA[<p>In Elgg 1, we will finally have native support for a River &#8211; that is, a stream of short updates of what you and your friends are up to on your profile.</p>
<p>Here is a short post explaining how you as a plugin writer could add river reporting to your code!</p>
<p><strong>Events</strong></p>
<p>The key to how the river works is the Elgg 1 events system and the system log.</p>
<p>The system log will listen to events and some events pass an object. If the object implements the <code>Loggable</code> interface it will automatically be included in the system log.</p>
<p><strong>The view</strong></p>
<p>In order for things to appear in the river you need to provide a view. This view should be <code>/river/CLASSNAME/EVENT</code>, where <code>CLASSNAME</code> is the class you&#8217;re interested in and <code>EVENT</code> is the event.</p>
<p>For example, if you want to output <code>create</code> events for all <code>ElggObject</code>s then you would need to create a file called <code>create.php</code> in a directory <code>/river/ElggObject/create.php</code>.</p>
<p>This file will be passed a number of variables via <code>$vars</code>.</p>
<ul>
<li><code>$vars['performed_by']</code> : An <code>ElggUser</code> object of the user that performed the action.</li>
<li><code>$vars['log_entry']</code> : The system log row (which includes the event).</li>
<li><code>$vars['object']</code> : The subject of the event.</li>
</ul>
<p>You can use this information to put together a very customisable view, don&#8217;t forget to internationalise your strings!</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2008/06/09/adding-to-the-river-in-an-elgg-1-plugin/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2008/06/09/adding-to-the-river-in-an-elgg-1-plugin/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2008%2F06%2F09%2Fadding-to-the-river-in-an-elgg-1-plugin%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2008/06/09/adding-to-the-river-in-an-elgg-1-plugin/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Introducing the Elgg 1 Query object</title>
		<link>http://www.marcus-povey.co.uk/2008/06/04/introducing-the-elgg-1-query-object/</link>
		<comments>http://www.marcus-povey.co.uk/2008/06/04/introducing-the-elgg-1-query-object/#comments</comments>
		<pubDate>Wed, 04 Jun 2008 15:47:07 +0000</pubDate>
		<dc:creator>Marcus Povey</dc:creator>
				<category><![CDATA[elgg]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[query]]></category>

		<guid isPermaLink="false">http://www.marcus-povey.co.uk/?p=40</guid>
		<description><![CDATA[Elgg 1 is introducing a new (to elgg at least) way of executing arbitrary database queries. Basically, SQL is abstracted away into a class that can be used to construct multiple different types of query through a defined interface. This provides two main advantages: The application programmer does not have to write the SQL themselves. [...]]]></description>
			<content:encoded><![CDATA[<p>Elgg 1 is introducing a new (to elgg at least) way of executing arbitrary database queries. Basically, SQL is abstracted away into a class that can be used to construct multiple different types of query through a defined interface.</p>
<p>This provides two main advantages:</p>
<ol>
<li>The application programmer does not have to write the SQL themselves.</li>
<li>It becomes considerably easier to change database back ends later down the line.</li>
</ol>
<p>In addition, we can add free stuff like access controls etc.</p>
<p>From the documentation:</p>
<blockquote><p>Query provides a framework to construct complex queries in a safer environment.</p>
<p>The usage of this class depends on the type of query you are executing, but the basic idea is to construct a query out of pluggable classes.</p>
<p>Once constructed SQL can be generated using the toString method, this should happen automatically if you pass the Query object to get_data or similar.</p>
<p>To construct a query, create a new Query() object and begin populating it with the various classes that define the various aspects of the query.</p>
<p>Notes:</p>
<ul>
<li>You do not have to specify things in any particular order, provided you specify all required components.</li>
<li>With database tables you do not have to specify your db prefix, this will be added automatically.</li>
<li>When constructing your query keep an eye on the error log &#8211; any problems will get spit out here. Note also that __toString won&#8217;t let you throw Exceptions (!!!) so these are caught and echoed to the log instead.</li>
</ul>
</blockquote>
<p>Here is an example of a select statement:</p>
<blockquote><p><code>// Construct the query<br />
$query = new Query();</code></p>
<p><code>// Say which table we're interested in<br />
$query-&gt;addTable(new TableQueryComponent("entities"));</code></p>
<p><code>// What fields are we interested in<br />
$query-&gt;addSelectField(new SelectFieldQueryComponent("entities","*"));</code></p>
<p><code>// Add access control (Default access control uses default fields on entities table.<br />
// Note that it will error without something specified here!<br />
$query-&gt;setAccessControl(new AccessControlQueryComponent());</code></p>
<p><code>// Set a limit and offset, may be omitted.<br />
$query-&gt;setLimitAndOffset(new LimitOffsetQueryComponent(10,0));</code></p>
<p><code>// Specify the order, may be omitted<br />
$query-&gt;setOrder(new OrderQueryComponent("entities", "subtype", "desc"));</code></p>
<p><code>// Construct a where query<br />
//<br />
// This demonstrates a WhereSet which lets you have sub wheres, a<br />
// WhereStatic which lets you compare a table field against a value and a<br />
// Where which lets you compare a table/field with another table/field.<br />
$query-&gt;addWhere(<br />
new WhereSetQueryComponent(<br />
array(<br />
new WhereStaticQueryComponent("entities", "subtype","=", 1),<br />
new WhereQueryComponent("entities","subtype","=", "entities", "subtype")<br />
)<br />
)<br />
);</code></p>
<p><code>get_data($query);</code></p></blockquote>
<p>Constructing a query out of classes in this way provides a lot of scope for expanding the functionality within a common framework, as well as providing a structured way of constructing complicated database queries. </p>
<p>However, if it looks too complicated, you can use about 99% of <code>Query</code>&#8216;s functionality by using the subclass <code>SimpleQuery</code>, which provides methods for performing much of the standard functionality.</p>
<div class="wsbuttons">
	<div class="shareblob facebook">
		<div class="fb-like" data-href="http://www.marcus-povey.co.uk/2008/06/04/introducing-the-elgg-1-query-object/" data-send="false" data-layout="box_count" data-width="60" data-show-faces="false" data-colorscheme="light"></div>
	</div>

	<div class="shareblob google">
		<div class="g-plusone" data-size="tall" data-href="http://www.marcus-povey.co.uk/2008/06/04/introducing-the-elgg-1-query-object/"></div>
	</div>

	<div class="shareblob twitter">
		<div class="twitter">
			<a href="https://twitter.com/share?url=http%3A%2F%2Fwww.marcus-povey.co.uk%2F2008%2F06%2F04%2Fintroducing-the-elgg-1-query-object%2F&count=vertical" class="twitter-share-button" data-lang="en">Tweet</a>
			<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
		</div>
	</div>

</div>
	]]></content:encoded>
			<wfw:commentRss>http://www.marcus-povey.co.uk/2008/06/04/introducing-the-elgg-1-query-object/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>

