Thursday, July 5, 2007

Subversion, SSH and Capistrano for a fully automated deployment

Deployment could be defined as; uploading an application to the hosting server and make any required modification or configuration to the file system, web server, database server, or whatever for the application to run properly.

For deployment, I use Capistrano, Subversion and Putty. Notice that I did not mention an FTP client. My minimal typical hosting environment consists of at least one instance of production web server and another instance as a staging web server, and each of them has its own file structure spanning to more than 3 levels and all the web servers are behind a reverse proxy server as the front end for web request, and I may run clusters of these instances. It could take me more than 10 minutes to do but the simplest tasks of deployment, like updating 3 files in 2 different folders, for example. As simple as it seems, but that kind of task is troublesome, especially that after uploading those files in their respective paths, and preview the updates on the staging server, and iterating a little bit in a preview, comments, fix, deploy loop, and finally redeploy in the production server, and restart the cluster instances. Well, at best, if half of your brain is actually a computer machine and you didn't make any silly overwrite or other nasty stuff to your application, your other brain half will get bored of these brain dead repetitive tasks, or you will simply screw yourself bad when you break your production hosting environment or worse destroy your production database.

We have to admit it; a programmer should always train himself to automate everything repetitive and boring he does, in order to spare his brain power on otherwise useful stuff that needs his attention like application architecture design and algorithms. Or simply to spare the world the kind of destruction our minds able to do when faced with boring repetitive tasks. This is a very popular concept which is called DRY, abbreviation of Don't Repeat Yourself.

Subversion

I believe that the centerpiece of any successful and decently automated deployment lies in the Version Control tool you use. Coupled with a decent SSH client and you're up to the deployment task. I use Subversion. It really makes my life easier. Imagine the following scenario;
• Write your application
• Import it to your online code repository.
• Continuously commit any changes you do in your application to your code repository.
• Need to deploy a development server; just check out a version of your application from the repository
• Changed anything in your application; just update your development server with any recent changes, and it will be in sync with your latest application code.
• Satisfied with your current development version, make a release tag in your code repository, and check out this version to a new production server.
• Need fixing, change the code, commit it, update your hosting server.
• Did a bad decision and decided to go back to a previous version; just update your application to the required version.

It is even simpler than it sounds. All these steps are done using simple commands like;
On local system:
$ mkdir myapp
$ cd myapp
$ svn import http://svn.myrepository.com/myapp
$ echo "puts 'hello'" > hello.rb
$ svn add hello.rb
$ svn commit hello.rb -m "Added Hello module that says hello"
On hosting server using secure shell:
$ svn checkout hello
$ svn update
$ svn update –r $required_revision_number

The beauty of this process of check in, check out, commit and update, is that you do not have to keep track which files are changed and need to be uploaded, or worse upload all your application folder each time you make an update.

Useful link: A Novices Tutorial on Subversion


Capistrano
Capistrano is the deployment tool i use for deploying my Rails applications on the hosting server. As quoted from Capistrano manual; Application deployment is one of those things that becomes more and more complicated as the scale of your application increases. With just a single box running your database and your application, it’s quite simple. But when you start putting your database on a different server, and then separating your web servers from your app servers, and eventually splitting your database into master and slave servers… It can get to where you almost don’t want to deploy your application any more.

Simply put, Capistrano makes it even easier to do tasks like fire up your ssh client, type some commands on the remote server to move, update, restart, or whatever task you would normally do by hand, you can write your commands in a script called "recipe" for Capistrano and as the next decent software, it will submit to your command and do every task you tell it to do.

Tasks like; login to remote server, update your application code, migrate your database, restart your server , are very likely will be repeated over and over, and that should notify you to start a new recipe for Capistrano to cook at your command.

Useful link: Capistrano: Automating Application Deployment

Putty
I like Putty, it is a secure shell client. Nothing bad with openssh client, but when i use Windows, it's pathetic command prompt console leave no choice but to use Putty with its own terminal window. By the way, I wonder why for God sake, there is no any alternatives for the bloody windows command prompt console, is there any out there? And I mean terminal or console not just another shell, cause i found many (Cygwin, bash, sh). [update: I found a great console for Windows, it is called Console. It is a great tool, it even support transparent background! when used with unix tools like sh, tar, cp, rm...etc and all you unix geeks will find yourself almost at home.]

Among the servers of the hosting data center we have at work, there is only one server that open a Secure Shell port to the internet. For a hosting data center, this is not a good idea. But we have to have a remote shell to login to, in case of emergencies or remote administration. My system admin collegue at work, had generated a pair key for my account on the hosting server. And that was a good idea. Because, it requires a private key for your account on the linux box for you to be able to login. No username/password login prompt anymore. This technique is both secure and easier than the regular login procedure.

Using key authorization is very useful if you intend to use any kind of automated deployment tools. Because such tools will almost always use key authenticated secure shell for login to the deployment server. And scattering your username/password all over your deployment scripts is not something you should do if you care about your hosting server security.


1 comment:

Admin said...

Thanks for the great article. I've noticed it is dated and you are now talking about git, but still, i would like to ask, do you have any experience on deploy PHP applications using something similar to described above?

How can you work with a JSON value if you know nothing about it?

I have talked about the difficulty of typing certain JSON values coming from some APIs. The JSON is just very complicated. When I do that,...