Sunday 7 August 2011

Dashing off an egg

Last month I sacrificed a weekend to python coding, having got a 2 day pass from the kids. Taking part in a django dash. The local DBUG community organised three teams to take part at the House of Omni, in Bristol. As it turned out the dash had to be delayed (results due next week), but unable to rearrange, the Bristol teams just went ahead
with our own dash anyway.

The idea behind the dash is to put together a working and hopefully useful django site within a strict 48 hours time period - competing to be judged the best against other teams of 2-3 developers / designers. Same as PyWeek but shorter and it doesn't have to be a game! Hence the competitive, and non-framework focussed elements distinguish a dash from a sprint.

I formed a team with Tom Dunham and we worked on a project Tom had found posted by Fraser Stephens of the Helios Foundation, to create a demonstrator for the sharing of supply chain data between aid agencies. The idea was to use a web based database that would allow a range of organisations to upload their inventory details along with location data to cater for the easy pooling and exchange of resources on the ground - hence aiding rapid targeted distribution.

Although there is an existing HELIOS supply chain system, this one was aimed at being a light weight alternative for groups that may not have HELIOS or other full inventory databases in place. So instead they could use the lowest common denominator of exporting databases or spreadsheets to CSV files, for manual or automatic scheduled synchronisation.

For ease of working we decided to split the system into two eggs (one each), the project specific one that sets up the django site and models, etc. and a generic CSV import egg.
We just about managed to deliver a prototype, which is on the HELIOS site.

Luckily since we weren't bound by the rules of the real competition, so I could make sure it was up and running after the 48 hour deadline. Perhaps more importantly since I worked on the CSV import egg, I could add some documentation and release it as django-csvimport.
If you do use it, let me know of issues, so I can refine it.

Its just a starting point, but in time it hopefully should result in less django developers wasting time writing custom CSV import scripts. It also meant that I became more familiar with character encoding in python and some of the useful relevant libraries in this area, ie the core csv library, chardet and xlrd.

I am ashamed to admit that I didn't know until I needed to write some unicode parsing tests that in order to be able to use readable non-ASCII characters in python code (rather than escape sequences), you just add the appropriate encoding line to the top of your file

# -*- coding: utf-8 -*-

... I guess its an ignorance born of being a linguistically challenged native English speaker :-}