Smoothing a mashup of moodle, mahara and drupal

I have been involved with the development of a system for an international school level qualification organisation over the last few months which has recently gone live in the USA. The work was undertaken in association with a UK learning resources company that was hosting the system.

This work was funded by a charitable foundation in the US, and provides community support and training to teachers at schools in deprived areas, aiming to bolster the low numbers of their students moving on to Higher Education.

The software was a mashup of PHP applications, Moodle the leading open source virtual learning environment, Mahara the e-portfolio app and Drupal the popular CMS.

This multiplied by three, the usual challenges of customising software written by third party developers to fit the clients requirements. It also involved a high level of integration work. Then to add to the challenge the software was to be developed on our servers but deployed to Triple A's. Finally PHP LAMP applications are not known for their clean separation of code, configuration and content.

All these factors meant that the development of a means for well controlled release management and content and configuration transfer between deployments was an essential factor for the success for the project.

Having used Fabric for simpler deployment tasks of python frameworks already, it was the obvious choice here too. Since although PHP also has shell frameworks, eg. PHP CLI, they are not as mature as Fabric, and PHP is not a natural object orientated shell scripting language.

Eventually we developed a set of fabric scripts that automated a number of the functions that allowed us to easily develop, deploy and debug the system. A single install server command would download the three applications on to the target machine, install them, extend drupal (with the help of drush) then add all the customization and integration code.

Similarly deploy to tag name could be used to release a version of the code set to the demo server for test and approval, prior to using the same command with just a server name change to deploy to the external hosts servers.

Further commands ran the involved single sign on / RPC user integration procedure for Moodle and Mahara (the instructions for doing this manually run to a 37 page PDF!)

Over all the complexity of all these operations and the different server environments lead to the process of maintaining the scripts was rather time consuming. However this time was easily recovered in allowing for automated deployment and set up. Perhaps more importantly the ability to quickly roll across database content, SSO integration etc against different code versions and servers, gave us an invaluable tool for both development and debug of critical issues that arose.

Ideally with more time / budget for tighter release management and pre-release testing and we could of had an even smoother and more robust ride.
But without a good shell framework we would have struggled to make a system at all, given such a mashup has hundreds of Mbs of third party code, and hundreds of tables across the three applications schemas.

Finally we now have a set of deployment tools that we can reuse to give us a big productivity boost in creating bespoke, VLE, e-portfolio, CMS systems in future.