In my previous post about unit testing for django, I laid the groundwork for how to unit test any django application. One nice feature that django includes with its test framework is the test database syncing. Even better is if you are using South to do database migrations - it will run the migrations in  your test environment for you.

However, what if you are using a NoSQL database backend like MongoDB, Cassandra, CouchDB or something similar and you aren’t using the Django ORM? How do you handle setting up and tearing down the database environments?

One of the big problems with hosting your own database solution is that you have to do backups for it on a regular basis. Not only do you need to do backups for it, but you need to also keep backups offsite. Luckily, Amazon S3 allows a cheap and easy solution for your offsite backups.

I found a shell script solution for handling MongoDB backups, but it only does local backups. It keeps a nice history of recent backups, and rotates off the oldest ones when the threshold for age is reached. I modified the code to call a Python script that then synchronizes the newly created backup file to S3. I haven’t wired up any purging functionality yet, and I don’t know if I am going to. S3 storage is so cheap that it really doesn’t matter much. A complete solution would, of course, keep your local files and your remote off-site backups in S3 in sync, but there is also a case to be made for keeping a rich history of backups in the “cloud” so as to be able to revert to any point in history if necessary.