“I am creating an Elgg network and I am expecting 10 million users, can it cope?”

This question gets asked in one form or another almost every week, but I believe this is the wrong question to be asking. Perhaps the more pertinent scalability question is:

“I am creating an Elgg network, how do I attract 10 million users?”

This is by far the hardest question to answer and it must be solved before you can seriously address the comparatively simple task of hardware and software scalability.

Attracting users can be accomplished in many ways but it is mainly a matter of marketing, and of course to have a killer idea. As Ben discussed in his presentation at the recent Elgg Conference, this idea must be as useful to user 1 as user 10 million.

The idea has to be solid from day one,  so forget all the trendy “long tail” and “wisdom of crowds” buzzwords!

Once you manage to solve this most tricky of problems, you can begin to look at the infrastructure. So, can Elgg handle 10 million users out of the box?

Simply, no script in the world can handle this level of usage straight away without some modification and a serious investment in both time and money. You will not be able to unpack Elgg on a cheap shared host and have it handle 10 million users.

This is not an issue with Elgg’s design (which actually lends itself to many scalability techniques), but simple realism. Elgg has had substantial work done on scalability and optimisation – reducing queries, caching etc – and currently performs very well page for page against competitors like Ning and Buddypress.

Asking how many users an Elgg install can support is also a pointless question, because the answer is always going to be “it depends”. How many users Elgg can support depends on your hardware, your host (shared or dedicated), your database server, how your users behave and how many of them are active at any given time.

So what should you pay attention to?

Elgg itself is fairly optimal, and will improve over time. If you are dealing with millions of user you will be wanting to look at your server infrastructure – database server, bandwidth, memory, caching at every level. After this you can look at customised code to squeeze out the last percentage points of performance.

If you are serious about handling high load there is no avoiding the need to spend some time and money investing in your infrastructure. But, these are good problems to have, because it means that you have a successful network!

So in conclusion, my answer to the scalability question is “Don’t worry about it until you have to worry about it!”, get your users in first. Make a killer service that is useful from day one, and then worry about how you will handle millions of concurrent users.

Scalability is a largely solved problem… building a successful service isn’t, and is the thing you should be concerned with.

Sometimes things need to be done without user interaction – for example, database optimisation or log rotation.

For this, Elgg has a cron endpoint.

Cron is a unix tool which executes commands at a specific time of day (other operating systems have similar tools). This keys off a file called a crontab – an example is given file is included and called crontab.example.

The crontab calls simplified yet powerful cron endpoint – http://yoursite/pg/cron/PERIOD, where PERIOD is one of the following:

  • reboot – Execute on system reboot
  • minute – Execute every minute
  • fiveminute – Execute every five minutes
  • fifteenmin – Execute every fifteen minutes
  • halfhour – Execute every half hour
  • hourly – Execute once every hour
  • daily – Execute every day
  • weekly – Execute weekly
  • monthly – Execute once a month
  • yearly – Execute every year

When these endpoints are triggered by your crontab a plugin hook is triggered. To make use of this, register a plugin hook as follows:

register_plugin_hook('cron', PERIOD, 'my_cron_handler');

Where PERIOD is one of the key words listed above. Here is some sample code using Cron – in this case it is taken from the system log rotation module I added to SVN today.

<?php
/**
* Elgg log rotator.
*
* @package ElggLogRotate
* @license http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU Public License version 2
* @author Curverider Ltd
* @copyright Curverider Ltd 2008
* @link http://elgg.com/
*/

/**
* Initialise the plugin.
*
*/
function logrotate_init()
{
$period = get_plugin_setting('period','logrotate');
switch ($period)
{
case 'weekly':
case 'monthly' :
case 'yearly' :
break;
default: $period = 'monthly';
}

// Register cron hook
register_plugin_hook('cron', $period, 'logrotate_cron');
}

/**
* Trigger the log rotation.
*
*/
function logrotate_cron($hook, $entity_type, $returnvalue, $params)
{
$resulttext = elgg_echo("logrotate:logrotated");
if (!archive_log())
$resulttext = elgg_echo("logrotate:lognotrotated");

return $returnvalue . $resulttext;
}

// Initialise plugin
register_elgg_event_handler('init','system','logrotate_init');
?>

Themes for Elgg are both extremely easy to develop and incredibly powerful. Using themes you can completely change how an Elgg install looks and feels (and even behaves).

Since there has been a fair amount of discussion of themes on the groups, I thought it would be a good idea to write a brief post about it.

Themes use two key Elgg concepts – namely, the plugin architecture and the views system.

By far the easiest and flexible way to make a theme for Elgg is to build it as a plugin. This makes it easy to distribute (since they are self contained) and lets you turn the theme on and off from the admin panel (making the theming process far less invasive!)

What you must first do is create a new plugin directory under /mod (documented here). In a nutshell; create a directory in the name of your theme, a new start.php and a new manifest.xml.

Once you’ve done this you then can start modifying views. This can be done either by extension or by view overriding.

View extension
The first way is to add extra stuff to an existing view via the extend view function from within your start.php’s initialisation function.

For example, the following start.php will add mytheme/spotlight to the already existing site spotlight:

<?php

function mytheme_init()
{
extend_view('page_elements/spotlight','mytheme/spotlight');
}

register_elgg_event_handler('init','system','mytheme_init');

?>

View overriding
The next method is to override an existing theme, completely replacing it with the one provided by your plugin.

View files provided by plugins automatically take precedence over views from the core. So all we have to do to entirely replace the existing spotlight is to create a new spotlight.php in the appropriate hierarchy.

So, if the original view is stored in:

/elgg/views/default/page_elements/spotlight.php

We need to create the file:

/elgg/mod/mytheme/views/default/page_elements/spotlight.php

Now, when we go to the admin panel and activate our theme the spotlight will be replaced by whatever you put in that file. Simple eh?

You can of course do this with any view.

Using a combination of these methods means you can replace the entire look and feel of a site very quickly indeed, although I would suggest that you start slowly since many views do some quite complicated things.