A number of ways of deploying websites have been used and one way that is becoming increasingly popular is to produce native software packages of the site for deployment. I first encountered this approach a few years ago in forums and it does work rather well. You can create your own repositories and allow the underlying package management to automatically install the dependencies and keep everything neat and tidy.

This is particularly useful with sites which are designed to scale out as they grow - just take a fresh node from a VPS provider of your choice (or even real hardware if you are dealing with any kind of business critical data that you need to keep secure and away from other people's access), drop the packages on and it's all ready to go. A fully packaged deployment can mean new nodes can be online within minutes allowing very rapid scaling.

In my case I typically use Debian or Ubuntu so I package into .deb packages. The basic packaging is taken care of very well in the Debian New Maintainers Guide, but there are a few things that I have seen coming up repeatedly in forums specific to packaging for websites that never seem to be very well answered and I want to deal with those here.

Packaging strategy & planning

Odds are some code exists and has been tested locally by the time anyone starts thinking about packaging and deployment. There are three main things that need some thought before we leap into the packaging as a whole:

Partitioning of the code/data

Sites are seldom simple enough to make a single-package install practical. You may have considerable static data (eg. images, software you sell, documents), maintenance scripts and tools, and other support scripts, data or other tools.

It's often worth breaking sites up into multiple packages so that when a new version of the code gets deployed, it's not necessary to update all the images, or perhaps regularly updated parts of the site (eg. FAQs or other documentation) can be deployed much more regularly without having to update the whole site. Likewise platform configuration (eg. firewalls, database configuration etc.) can also be packaged separately so as not to have to be changed every time new minor updates to the code are made.

Another thing to consider is following an example like used with kernel packaging where multiple versions can be installed and then you choose which to run. This means that if there is a major breakage discovered in a newly deployed version, the previous version is still installed and can immediately be brought back into operation.

It's well worth thinking through how to partition up your site before packaging it.

Dependencies

Let's take a basic LAMP stack (Linux, Apache, MySQL, PHP) for the purposes of this example. While developing I have been testing (either locally or via a Virtual Machine) and have installed the LAMP stack plus a load of libraries and other packages I need to run the site, and that's where we can start hitting problems.

When we package stuff it would be smart to have valid dependencies for the package so when it's dropped onto a freshly installed host, the dependencies get installed and it is all fully working. I have come across plenty examples of this not being done and then someone deploys a new node and everything ends up horribly (and sometimes cryptically) broken because some vital dependency.

Managing releases & versioning

With all but the most trivial site, sooner or later there will be problems if you can't have an audit trail as to what was (or should have been) deployed through to what is actually running on the live servers.

Some sort of version control is important and these days starting off with a  modern distributed version control often makes alot of sense. In my case I am using Mercurial which has the advantage of being platform agnostic and there are good guides to getting up and running quickly with Mercurial.

On top of that some kind of change / issue tracker is needed. I am using my favorite FogBugz which has comprehensive documentation, time tracking and management tools built in. It also has the big advantage of being free for up to two users.

To tie it all together case numbers are referenced in the checkins and I add the Mercurial ID into the changelog (see later) when I build a package which allows me to trace running code all the way back through the system.

All this is simply good pratice and good developers / devops / management would not proceed without it.

Initial packaging

Now that we have a plan for what packages we are going to build we can start creating the packaging.

As we are natively packaging our code the instructions for converting unpackaged code are not relevant. For this example I am going to assume we have a "bin" package which includes things like database backup scripts, scheduled maintenance etc., and that the site packages are individually versioned so we can have multiple versions available together and switch between them. In reality any site of significance will be partitioned much further. In my case my initial partitioning of the site is 4 packages.

For both sets of packages we need to (at least temporarily) rename the tree to match the package naming we are going to use into the format: <packagename>-<version>

Then in that directory run the initial package creation:

$ dh_make --native

This will prompt for the type of package and as it's LAMP (or shell scripts in the case of the management tools) with no binary extensions or anything, I choose "i" for binary idependent and confirm the config. At that point a debian/ directory is created and it's safe to change directory names back if you have temporarily renamed them.

This has created a load of example (template) files and a skeleton configuration. Go into the debian/ directory and we will start to turn this into a real package.

$ cd debian/

First up we probably don't need the README files:

$ rm README.Debian README.source

The copyright file is probably inappropriate for the site unless you are releasing your site as GPL, so best put a copyright notice that is suitable for your site:

$ vi copyright

A load of example scripts configuration has been created and I am assuming that there are no daemons, emacs extensions, X stuff or anything like that involved in the site so we can loose those too:

$ rm emacsen-* <package>.d* manpage.* init.d.* menu.ex watch.ex

We will leave post/pre script and cron examples which may be useful later so we will leave those examples around.

At this point the debian/ directory is likely sufficient and we need to produce a means of installing our code into the build directories. The simple way of doing this is to create a Makefile in the main tree (one level above the debian/ directory) that will do all the work. It really only needs to be very simple, especially for the "bin" package:

#!/usr/bin/make

all: # nothing to build

install:
    install -d $(DESTDIR)/usr/sbin/
    install site-management-scirpt $(DESTDIR)/usr/sbin/

.... plus anything else needed for your "bin" package. For the individually versioned packages for the site we may want to be a little more careful and have a version file where the version number we are building is kept that can then be used in scripts:

#!/usr/bin/make

all: # nothing to build

install:
    install -d $(DESTDIR)/usr/sbin/
    install sitename-switch-script $(DESTDIR)/usr/sbin/sitename-switch-`cat version`
    install -d $(DESTDIR)/usr/share/sitename/`cat version`
    cp -r www/ phplib/ config/ db/ $(DESTDIR)/usr/share/sitename/`cat version`

.... plus anything else needed for your site package. This assumes a script that can be run that will switch between versions of the site (eg. by symlinking) and we bundle all the relevant files into one versioned directory, and also name the switching script for the version it will switch us to.

The switching script can also be very simple:

#!/bin/sh
SITE=/usr/share/sitename
VER=`echo $0|sed 's/^.*-([0-9.]*)$/1/'`
if [ $VER = $0 ]; then
    echo "FATAL - can't get version!"
    exit 1
fi

# sort config
[ -h /etc/apache2/sites-available/sitename ] && rm /etc/apache2/sites-available/sitename
ln -s $SITE/$VER/config/apache.conf /etc/apache2/sites-available/sitename
[ -h /etc/php5/conf.d/sitename.ini ] && rm /etc/php5/conf.d/sitename.ini
ln -s $SITE/$VER/config/php.ini /etc/php5/conf.d/sitename.ini
# sort PHP libraries for site
[ -h /usr/lib/php5/sitename ] && rm /usr/lib/php5/sitename
ln -s $SITE/$VER/phplib/ /usr/lib/php5/sitename
# sort site
[ -h /var/www/sitename ] && rm /var/www/sitename
ln -s $SITE/$VER/www/ /var/www/sitename

# sanity checks
[ ! -h /etc/apache2/sites-enabled/sitename ] && echo "WARNING - sitename is not enabled (a2ensite sitename) in Apache config"
[ ! -h /etc/apache2/mods-enabled/rewrite.load ] && echo "WARNING - Apache mod_rewrite (rewrite.load) is not enabled (a2enmod rewrite) in Apache config"
[ ! -f /etc/sitename/siteconfig.php ] && echo "WARNING - /etc/sitename/siteconfig.php should exist on non-test platforms"
[ `stat --format=%a /etc/sitename/siteconfig.php` != '640' ] && echo "WARNING - /etc/sitename/siteconfig.php should be permission 640"
[ `stat --format=%G /etc/sitename/siteconfig.php` != 'www-data' ] && echo "WARNING - /etc/sitename/siteconfig.php should have group www-data"

# reload apache
/etc/init.d/apache2 reload || echo "WARNING - Apache reload failed with exit $?"

This works out the version from the name it was executed as and changes the site symlinks to the new version, does a load of sanity checks which could be doen first and stop the script to prevent enabling the site under circumstances it wouldn't work. Finally it reloads Apache which is good practice if anything has changed in the Apache or PHP configs.

Versioned packages for the site

To include the version number in the package name, the version number needs to be included/updated in the debian/control file. My approach to this was to rename my initial control file to control.master, and then have a version-increment script that creates a new control file from that with the version number added. To do that a tool like sed may be used - assuming the new version number was in $newversion this code could be part of your version-increment script:

sed -e "s/^Package: mozgq-site-master$/\\0-$newversion/" <debian/control.master >debian/control

The only other thing to watch out for is that the temporary install directory debian/<packagename> and associated files will not be cleaned up each time the package name changes as the packaging system will be looking for the new version to clean up. To fix that you can add in code to remove it in your build scripts or debian/rules.

Special Permissions

Sometimes we need a file or directory with special permissions and dh_fixperms overrides what our Makefile does leaving us with broken permissions. The way to handle this is to add an override in debian/rules:

override_dh_fixperms:
    dh_fixperms
    chgrp somegroup debian/packagename/var/lib/somedirectory/ -R
    chmod 770 debian/packagename/var/lib/somedirectory/ -R

That simply runs dh_fixperms as would normally happen at that point in the build, but then allows you to add further commands to set the permissions the way you need them before continuing.

These sorts of overrides can also be added for other things where build or packaging customisation is needed.

Dependencies

These may be added in the debian/control file. You can also add to Build-Depends which can ensure you have any tools needed for the build. For example a tool like yui-compressor may be needed for minification during build.

Config

The packaging system handles these intelligently regarding things in /etc as config, but if you need to tell it to handle more files as config files then list them in debian/conffiles.

On install/upgrade config files are checked for changes against existing versions and the user prompted to replace the file if it has been changed since the previous install.

Versioning

Whenever the version is changed the debian/changelog file needs to be edited. To make it easier use the dch tool with -i to add a new (incremented) version to the changelog, or -a to add changes to the current version.

This calls $EDITOR so you could incorporate it in a version-increment script to automatically handle the changelog and even add useful information like the version control checking id. An example $EDITOR script may be:

#!/bin/sh
hgrelease=`hg id -nibt`
sed -i -e "s/^  \*\\\* $/\\0hg $hgrelease/" $2
vi $@

A bit more scripting and you could even pull log entries from version control and/or ticketing systems.