A potted history of web development paradigms

Recently I had reason to review how we had approached web development for CMS and small web applications at the University where I work. It made me ponder the wider context of the evolution of web development approaches. Having been involved in writing web applications since 1998 and remember using mosaic when the web started, I realise that I am a bit of a historical artifact myself - and always keen to rewrite history ;-)

Server side programming languages
So back to basics, the top seven languages (some way ahead of the many minor languages) in use for server side web development in rough order of current popularity for this sector with their birth year are:

1. Java 1994
2. PHP 1994
3. Python 1989
4. C# 1999
5. Perl 1986
6. Ruby 1993
7. Visual Basic 1991

That popularity order has shifted around a bit since the inception of the web, and a lot more new languages entered the fray, perhaps most notably Java styled syntax Python/Ruby clones such as Scala and Groovy. C# and Java have replaced use of C/C++, and Ruby has been popular enough to enter the top seven, due to the advent of Ruby on the Rails in 2004.

Client side languages
Client side things are much simpler, Javascript (1995) has always ruled and seen off challenges by Java applets and Microsoft ActiveX then Silverlight over the last decade. As a part of HTML5 via canvas etc. it is designed to replace Flash and other proprietary client UI tools.
It is also increasingly becoming the default language for the restful APIs of modern web / cloud services, where it's JSON data format is supplanting the more heavyweight XML/SOAP (1999) APIs and web services architectures.

The dawn of web development, CGI and Perl
The common gateway interface was defined as a standard for calling programs from a web server in 1993, the same year HTML form elements first appeared, although not standardized until HTML2 in 1995. The position of Perl at the time as the ubiquitous unix scripting language - and in particular its suitability to text parsing, soon lead it to dominate early web applications.
In the early days applications may be little more than a form to email handler or such like. If rendering of HTML by the code was required, it would just be done by constructing a web page as a big string and sending it back.
Web development was more likely to be done by the unix administrator who would also cover content editing in their webmaster role.
The few more complex web applications were likely to be written in C.

The rise of embedded scripting, PHP, CFM, ASP, JSP
PHP began in 1994 rapidly followed by cold fusion 1995, active server pages 1998, then java server pages in 1999. This era saw the beginning of dedicated web developers.
As web applications began to grow a little more sophisticated, the big string approach was proving unwieldy.
Server side includes were already in use for limited embedding of code in web pages. But for applications where significant code was required, it was still often less than the page content they were returning. So rather than create all this HTML within the code, more web specific scripting languages that embed code in HTML arrived, lead by PHP.
As web traffic increased the process per request of CGI gave way to web server modules that could deliver better performance by pooling resources.
Eventually this development paradigm replaced the use of CGI scripting.

Web frameworks are born, cold fusion and zope
The most significant early web frameworks, still surviving (just!) today are Cold fusion, started in 1995, and Zope, which followed in 1996. The early frameworks bundled up standard APIs for handling requests, forms, data storage, email etc. with embedded scripting languages. Zope was the more complex bundling an object NOSQL database, and publishing paradigm based on traversing objects and calling methods on them via the url space.

Use of VM based server runtimes, Enterprise software
Java fully entered the web world in 1997 with the first Servlet implementation, and .NET C# in 2000. They employ similar architectures using languages that precompile to byte code for running on a virtual machine that communicates via a standard API to the web server.
Part of the purpose of these architectures were to allow for the same paradigms, IDEs and development practises to be used for web applications as were already established for desktop development. Hence common elements such as database APIs could be leveraged. This also helped usher in more formal development practises brought from existing OS based application development.

Through the web development and content management systems, TTW & CMS
PHP has produced the most CMS, both generic and more specialist. An early example being TYPO3 in 1997. Today there are thousands of CMS in existence catering for all sorts of niches. Many CMS are either built on top of frameworks, or have ended up verging towards becoming them, as their features become increasingly generic and pluggable. Their approaches and technologies differ widely, however the key common development practice that tends to distinguish them is that since they cater for through the web content editing, they tend to add code and configuration editing TTW too. This is fine for quick prototyping but has proved unviable for long term code maintenance and scaling. Hence the more mature / large scale CMS have evolved features that allow for all code to be pushed out to the file system, along with configuration (usually as XML) for versioning and release management. There is now also a cross platform standard for content and authorisation data transfer CMIS. These various elements allow separation of concerns and delivery which may usher in a new era. Various CMS for different purposes (intranet, blog, portal, wiki, VLE, CRM, specialist etc.) as content repositories locally or in the cloud, with loose coupling of front end delivery and integration web sites separate from the content creation and administration software.

Templating languages
As soon as web applications began to become more complex the problems of embedding raw code in HTML, ie the early PHP, ASP, JSP approach began to reveal itself. Mixing content and code so tightly together doesn't scale to larger applications, so templating languages were invented to help separate the presentation from the logic layer.
Clear silver was one of the first templating languages dating from 1999, originally targeting larger web apps written in C/C++. This was followed by Apache Velocity in 2001 and zope page templates, TAL in 2002. Since then templating engines have proliferated, and it is now unusual to develop a web application that delivers markup - whether HTML, XML, SVG etc. without one. Hence raw embedded scripting has largely been superceded.
In general there are two schools of thought with templating engines.
The first's primary aim being for templates to allow full markup validation prior to code rendering. This approach is closer to embedded scripting since it caters for snippets of the server side language.
The second school aims for templating languages to be as simple as possible and doesn't allow embedded code, limiting the capabilities of template tags to ensure clean separation of logic from the template presentation layer. This approach doesnt usually provide pre-rendering validation.
Examples of the former are TAL, Genshi, Facelets. Whilst the latter include Django templates, Velocity, Closure, Smarty.

MVC web frameworks replace embedded scripting
Although the model view controller pattern dates from the early days of GUI at Xerox with Smalltalk in the 70s, it took a surprisingly long time to reach its position of dominance in web development. Early 90s web frameworks such as the ones mentioned above didn't use it, but from their introduction in 1999 Java's JSP pages posited a model2 (MVC) approach rather than the embedded (model1) default. The first successful MVC framework was Apache Struts (2000) implemented using JSPs as the view, and servlets as the controller. These and other early Java MVC frameworks utilized the pattern to an extent but without a full stack implementation. Hence no ORM for full object abstraction of data storage as models. Although tag libraries could reduce embedded Java, JSP usage as the view rather than a template library, lead to poor logic separation.
It was Ruby on the Rails in 2004 that really proved the effectiveness of a full stack MVC framework. It was followed by the other 4 leading web MVC frameworks in 2005, django, cake, symfony and codeigniter. Perhaps surprisingly there is no full stack purely MVC Java framework. JSF is missing ORM features and Spring is a heavier weight framework (not soley for the web) but there are optional MVC layers that can be run on it, Spring MVC and now Grails - although here development is with Groovy not Java. Finally Microsoft launched ASP.NET-MVC in 2007.
The features and productivity that a full stack framework can provide are significant, usually at least double that of developing a fully bespoke solution. So increasingly purely bespoke development is only for specialist areas and large commercial applications. As a result it has dwindled as a proportion of web development, especially wrt. embedded scripting, and largely been replaced either by adapting generic applications, such as CMS, or using MVC frameworks.

Some near futures for web development?

• As web development becomes the predominant software medium then more formal development practises will hopefully become the norm. The use of continuous integration, testing and release management - possibly via continuous deployment methodologies, is likely to become much more widespread. Driven by its match with the feed back loop of usability and user centered design.
• HTML5 and AJAX based rich client user interface design its currently in its infancy, but it looks likely to replace most other client approaches, reversing the current short trend to return to platform specific development with mobile - Android, Objective C, etc.
• Cloud services are likely to continue their rise, so development that uses restful APIs to create integrated mash up solutions are likely to become more standard. These may gradually replace the current use of local generic applications, such as CMS, with cheap or free cloud hosted alternatives.
• Full stack frameworks and other solutions that help increase productivity are likely to continue their rise - with more complete end to end features, ie data modeller (ORM) to client (HTML5) libraries included. Possibly more of these will be used in a software as a service manner, along the lines of Google Apps or Microsoft Azure.
• The rise of automation and the cloud will result in the concept of small, medium enterprise scale (SMB/SME) fading - along with local installation / licensing. Replaced by web scale platform and software as a service - charged by the scale and uptime requirements.