30

What's the best practise to deploy new code on a live (e-commerce) site?

For now I have stopped apache for +/- 10 seconds when renaming directory public_html_new to public_html and old to public_html_old. This creates a short down-time, before I start Apache again.

The same question goes if using Git to pull the new repo to the live directory. Can I pull the repo while the site is active? And how about if I need to copy a DB as well?

During the tar (backup purpose) compression of the live site I noticed that changes occurred in the media directory. That indicated to me that files keep on changing periodically. And if these changes can interfere if Apache is not stopped during deployment.

Teun Zengerink
  • 197
nicoX
  • 611

8 Answers8

32

The fastest and easiest is to use a version directory such as 

/var/www/version/01
/var/www/version/02

and use a current symbolic link as your html_root :

/var/www/html -> /var/www/version/02

This technique integrate perfectly into a revision control system (svn, git, mercurial, ...) as you can checkout branches & tags, changes the symbolic link and reload Apache. The downtime is minimal using this technique and it allows very easy rollback.

It also integrate well with more complex deployment system such as RPM packages, or configuration change management (chef, puppet, etc) infrastructure.

CloudWeavers
  • 2,561
  • 1
  • 16
  • 17
14

Renaming the directories without shutting Apache down should work as well. That will shorten the window significantly. mv public_html public_html_old && mv public_html_new public_html should finish in a fraction of a second.

A couple of drawbacks are that this approach will give a 404 to any request that still manage to happen during the window. And if you run the above command without having a public_html_new directory it will fail and leave you with a site giving 404 on every request.

Doing it atomically with directories is not support. But you could do it with symlinks. Instead of having a directory named public_html, have a directory named public_html.version-number and a symlink called public_html pointing to that directory. Now you can create a directory called public_html.new-version-number and a new symlink called public_html.new.

Then you can rename public_html.new to public_html to switch atomically. Notice that mv is "too intelligent" to perform that rename, but it could be done using os.rename from python or anything else which will call the rename system call without trying to be smart.

What to do with the data base depends on what database you are using, and what you are using it for. You need to provide a lot more details about the database before we can give you a good answer to that part of your question.

kasperd
  • 31,086
13

Using a load balancer is a good idea. If the site is important enough to worry about a few seconds of downtime, it's important enough to worry about fault tolerance.

That aside, if this is on a UNIX system you can put Apache on hold during the rename (or symlink update, etc.):

killall -STOP httpd  # Pause all httpd processes
mv public_html public_html_orig
mv public_html_new public_html
killall -CONT httpd  # Resume all httpd processes

This will keep Apache from accepting new requests during the rename. If you prefer symlinks or some other approach, the same idea can be used:

killall -STOP httpd  # Pause all httpd processes
rm /var/www/html
ln -s /var/www/version/03 /var/www/html
killall -CONT httpd  # Resume all httpd processes

Note that any pending connections or packets will queue up in the OS. For an extremely busy site, consider tuning ListenBacklog if appropriate for your httpd worker type, and check your OS settings related to TCP listen backlog.

You could also change DocumentRoot in httpd.conf and do a graceful restart (apachectl graceful). The drawback here is the increased risk of errors, since you'd have to update any Directory configuration as well.

GargantuChet
  • 314
  • 1
  • 8
11

Symlinks and mv are your friends, however, if you really need to avoid end-users getting an error page while deploying a new version, you should have a reverse-proxy or a load-balancer in front of at least 2 backend servers (apache in your case).

During the deploy, you just need to stop one backend at a time, deploy the new code, restart it and then iterate on the remaining backends.

The end-users will be always directed to good backends by the proxy.

Giovanni Toraldo
  • 2,607
  • 21
  • 27
9

If you are applying changes regularly on a production system, I would take care of a structured life cycle. A good practice is Capistrano http://capistranorb.com/. This is a an open source solution for deploying software on one or more servers on several platforms and configurations.

For Magento there is even a plugin: https://github.com/augustash/capistrano-ash/wiki/Magento-Example

For single server and almost seamless transitions, I recommend to use symlinks.

Skiaddict
  • 116
4

The way I do it is do commit my changes from my local dev environment to an online Git repository such as Github. My production environment runs off a remote repository so all I need to do is ssh to the server and run git pull to bring down the latest changes. No need to stop your webserver.

If you have files in your project whose settings and/or content differ from your local version (such as configuration files and media uploads) you can use environment variables and/or add these files/directories to a .gitignore file to prevent syncing with the repository.

harryg
  • 971
3

My first idea is:

# deploy into public_html_new, and then:
rsync -vaH --delete public_html_new/ public_html/

A good solution were to use rsync. It changed only the really changed files. Beware, the slashes at the end ot the pathes are here important.

Normally apache don't need a restart, it is not the java world. It checks for the change of every php file on request, and rereads (and re-tokenizes) on change automatically.

Git pull were similar efficient, although it were a little bit harder to script. Of course it enabled a wide spectrum of different merging/change detection possibilities.

This solution will seamlessly only if there are no really major changes - if there are big changes in the deployment, a little bit of hazard can't be closed out, because there is a not negligible time interval, when the code will be partially changed and partically not.

If there are big changes, my suggestion were your initial solution (two rename).

Here is a little bit hardcore, but 100% atomic solution:

(1) do an alternate mount some of your filesystem, where your magento takes place:

mount /dev/sdXY /mnt/tmp

(2) do a --bind mount of your public_html_new to public_html:

mount --bind /path/to/public_html_new /path/to/public_html

From this point, the apache will see your new deployment. Any change of a 404 is impossible.

(3) do the synhcronistation with rsync, but on the alternate mount point):

rsync -vaH --delete /mnt/tmp/path/to/public_html_new/ /mnt/tmp/path/to/public_html/

(4) remove the bind mount

umount /path/to/public_html
peterh
  • 5,017
2

Moving/replacing the http_public folder can be achieved with simple mv or ln -s commands or equivalent while your http server keeps running. You can do some scripting to significantly reduce the downtime, but check carefully the return codes of your commands in the script if you automate the process.

That said, if you want to achieve no downtime, you application must also support it. Most application uses a database for persistency. Having version N of your application messing with version N+1 (or the reverse) of your datamodel may break things if not foreseen by the development team.

From experience, maintaining such a consistency through upgrades is not a given for most applications. A proper shutdown, despite the downtime, is a good way to avoid consistency issues.

Uriel
  • 123