Backing up Verdad

Since there is a lot of critical information in Verdad, it's critical to back it up and be able to restore it.

Backup

Because all of the data in Verdad is stored in a MySQL database, you can simply use MySQL backup utilities to get a good backup of the database and you're done.

There is a script called backup-verdad that comes with Verdad that you can use as a template to do backups at your site. You will want to edit it to change where email gets sent, and to set pathnames that make sense for your installation.

It does three jobs. First, it uses mysqlhotcopy to create a copy of the database called ``verdad_copy''. Next, it uses mysqldump to create a text dump of the copy of the database, which it stores in a compressed format.

Finally, it runs check table on each table in the database. This can repair MySQL problems before they become serious, and can also optimize query speed in some cases. See the MySQL documentation for more information on check table.

You should probably be running backup-verdad from cron.

Restore

When you need to restore a Verdad database, you have two choices. You can either rename the ``verdad_copy'' database to ``verdad'' or you can create a new database and import the text dump into it.

Please note that if you are using MySQL replication, you'll need to take extra steps, summarized below.

Restore from a Backup Database

Say you've just written a clever script which will renumber a subnet. Upon running it, you discover that all the IP addresses stored in Verdad are ``0.0.0.0''. You know that the only changes to the database since you ran your script are easy to redo. So you decide to revert the database to the way it was last night at midnight when ``backup-verdad'' ran from cron.

Shut down the webserver

This will prevent users from interacting with Verdad while you are doing maintenance.

Shut down MySQL

I use ``/etc/init.d/mysql.server stop'' to do so. Your mileage may vary.

Rename the databases

Change directory into the MySQL data directory. Move the ``verdad'' directory to ``verdad_broken'', and move the ``verdad_copy'' database to ``verdad''.

Put things back

Start MySQL again and then start Apache.

Restore from Text Dump

You'd want to do this if the database (and the backup one) are unusable for some reason. Perhaps they were on the disk that's now spewing black smoke, or on a partition that fsck ``fixed'' a little too well. Whatever the case is, you've lost the MySQL database entirely at this point, and you want to get going again.

Shut down the webserver

This will prevent users from interacting with Verdad while you are doing maintenance.

Consider locking out the Verdad user

By changing the password temporarily on the MySQL account for the user Verdad logs in as, you can prevent all access to the database. If most access is via the webserver, this is probably not necessary.

Take a backup just in case

Maybe the cure is worse than the disease... maybe restoring the database isn't really what you want to be doing. Take a backup of the database right now before going any farther, using the normal system you always use to back it up.

Make a list of changes you'll lose

This bit of SQL will help you identify changes that will be lost by restoring this backup.

        select * from txns where when_dt > 'YYYY-MM-DD';

Replace YYYY-MM-DD with the year, month, and day of the backup file you are using.

Drop the old database

Use ``mysqladmin drop verdad'' to drop the database named ``verdad''. After this point, there's no going back.

Create the new database

Use ``mysqladmin create verdad'' to create the new database.

Load the database

Use a command like gunzip -c $backup.gz | mysql -u verdad verdad to uncompress and execute the SQL data in the backup file in one command.

Put things back to normal

Fix the password on the database account. Turn on the webserver. Notify your users that changes between the time of the backup and now have been lost due to the restore operation.

Replication

It is beyond the scope of this section to make you into an expert on MySQL replication. However, if you use replication, you need to be aware that there are additional concerns involved with doing a restore.

The basic difficulty is that a restore moves the database ``backwards in time'', putting the master ``behind'' the slaves. Unless all of the actions you take on the master feed through to the slaves via the replication system, you'll end up with very confused slaves, and a master that can't talk to them.

If all the actions you take during the restore are regular SQL actions, there's a good chance the slaves will follow along with the master. To do that, you'd use truncate to clean out the old master database before reloading it with only insert commands. Presumeably you'd do the truncates by hand, and grep out only inserts from the text dump file, then send them in to mysql. You must not drop the tables and recreate them (as suggested above), since that action will leave the slaves confused.

If the slaves get confused, or you don't want to bother keeping them from getting confused, then you need to nuke the slaves and repeat the instructions in the MySQL manual for starting replication. Briefly, that involves cleaning out the master's directory to have no bin logs, taking a snapshot of the master while it is quiescent, starting the master, sending the snapshot out the the slaves, then starting the slaves. At that point, your log number reverts to 001, and all new transactions show up on the slaves.


Author

Jeff R. Allen <jra@nella.org>