Getting up to speed with django

I finally tracked down why my experience of django has been really slow, when supposedly one of its qualities is speed. So it is meant to be faster than a lot of LAMP frameworks, eg. Symfony, and also Ruby on the Rails, so certainly better than a non-rdbms centric platform like zope/plone - yet I was getting really terrible performance.

Initially with a standard Apache set up I was getting 10-20 secs/req ! ... gak unusable what was wrong?
Simply adding KeepAlive Off to the apache conf had a radical effect. Now I was getting 3-4 secs/req.
So pretty terrible but not totally unusable. Things were a bit better under mod_wsgi so 2-3 secs/req.

However the real nub of the issue was Oracle, the backend was not using pooled connection so each page was renegotiating an Oracle connection. With our distributed Oracle server network that could take 2-3 secs (ie around 90% of the request time).

I investigated pyorapool and it doubled performance, but had issues and it seemed overkill to run something more suited to pooling connections across servers.

So I went back to a post about a pooled oracle backend by Taras Halturin, which I initially couldnt get working. Having fixed it for current django 1.1.1 I have packaged it up in our repo as ilrtdjango.oracle_pool I guess if I add some extra features to it such as logging etc, I should contact Taras about punting it up on pypi.


This really deals with the problem by wrapping up cx_Oracle's own connection pooling. Now I am seeing 0.4 secs/req on average, with cached pages down to 0.1 secs.
There is a lot more that could be done with better fragment caching and use of memcache rather than a simple file system backend. But for the moment I am happy that django really is fast, ie. the framework can do 10 req/sec out of the box. With good queries and materialised views for remote databases, using a pooled backend a few Oracle queries should at most halve that speed, so a 5 req/sec uncached baseline.

This is a much better performance point to work from for database applications, and if I need the sort of 50 req/sec performance of a high traffic site then I have all the caching headroom to use (e.g. memcache or a web cache frontend).

Plone conference keynote

This is an attempt to paraphrase Alexander Limi's keynote speech, rather than reflect any of my own views (added as comments instead).

Intro

Good to see two new plone books are out, e.g. Erik Rose
plone 3 for education -
and contrary to Google trends - usage is still increasing.

plone 4/5 roadmap:

native blobs in 4

performance should be twice as fast as 3 -
largely due to Chameleon and its compiled ZPT template acceleration (also used by Pylons for Genshi)

code base should be 30% smaller

deliverance - more into the core

dexterity - to replace archetypes but will still have AT in 5

deco - UI layout design framework using tiles to replace viewlets and portlets. All of the page elements can be drag and dropped rearranged.
- so edit mode becomes a design mode too! (Live demo of deco drag and dropping layout followed)

Issues about promoting plone

Cant compete against cost and simplicitiy of PHP CMS for web page publishing - so sees goal as the high end of the market - enterprise collaborative CMS, easy to extend with simple apps.

Hence the competition is Alfresco, RedDot, Sharepoint etc - Plone community need to create standard content importers from these.

Fix plone.org to drive plone again. More user groups, etc.