• BACKUPS
  • DATABASE
    • MySQL
      • Restoring from backups
    • PostgreSQL
      • Restoring from backups
  • FILESYSTEM

BACKUPS

RT is often a critical piece of businesses and organizations. Backups are absolutely necessary to ensure you can recover quickly from an incident.

Make sure you take backups. Make sure they work.

There are many issues that can cause broken backups, such as a max_allowed_packet too low for MySQL (in either the client or server), or encoding issues, or running out of disk space.

Make sure your backup cronjobs notify someone if they fail instead of failing silently until you need them.

Test your backups regularly to discover any unknown problems before they become an issue. You don't want to discover problems with your backups while tensely restoring from them in a critical data loss situation.

DATABASE

You should backup the entire RT database, although for improved speed and space you can ignore the data in the sessions table. Make sure you still get the sessions schema, however.

Database specific notes and example backup commands for each database are below. Adjust the commands as necessary for connection details such as database name (rt4 is the placeholder below), user, password, host, etc. You should put the example commands into a shell script for backup and setup a cronjob. Make sure output from cron goes to someone who reads mail! (Or into RT. :)

MySQL

    ( mysqldump rt4 --tables sessions --no-data --single-transaction; \
      mysqldump rt4 --ignore-table rt4.sessions --single-transaction ) \
        | gzip > rt-`date +%Y%m%d`.sql.gz

The dump will be much faster if you can connect to the MySQL server over localhost. This will use a local socket instead of the network.

If you find your backups taking far far too long to complete (this point should take quite a long time to get to on an RT database), there are some alternate solutions. Percona maintains a highly regarded hot-backup tool for MySQL called XtraBackup. If you have more resources, you can also setup replication to a slave using binary logs and backup from there as necessary. This not only duplicates the data, but lets you take backups without putting load on your production server.

Restoring from backups

New Database Server (Catastrophic Failure)

If you are starting fresh with a new database server (because your old one no longer works or because you want to set up a dev machine to test on) you will need to create a fresh database and database user for RT to use. RT can do that for you using:

    /opt/rt4/sbin/rt-setup-database --action create,acl

By default, this will create an rt4 database and an rt_user user. If you've specified a custom password in RT_SiteConfig.pm, RT will use that. Once the database and user exist, you can restore from your backup using:

    gunzip -c rt-20141014.sql.gz | mysql -uroot -p rt4

Changing -uroot -p as needed to access the database as a user with enough rights to handle creating tables.

Restore over an existing database

If something terrible happened this morning and you want to roll back to your backups, or if you want to update a dev server using your backups, this is straightforward on MySQL.

    gunzip -c rt-20141014.sql.gz | mysql -uroot -p rt4

MySQL will drop any existing tables before recreating and repopulating them. It will leave the database and the rt_user untouched. This is not suitable for restoring on a fresh database install since there will be no rt4 database or rt_user user.

PostgreSQL

    ( pg_dump rt4 --table=sessions --schema-only; \
      pg_dump rt4 --exclude-table=sessions ) \
        | gzip > rt-`date +%Y%m%d`.sql.gz

Restoring from backups

New Database Server (Catastrophic Failure)

If you are starting fresh with a new database server (because your old one no longer works or because you want to set up a dev machine to test on) you will need to create a fresh database and database user for RT to use. RT can do part of that for you using:

    /opt/rt4/sbin/rt-setup-database --action create

You will need to create the rt_user separately.

    createuser -P rt_user

This will prompt you for a password. You should ensure that it is the same password you have configured in RT_SiteConfig.pm or RT_Config.pm using $DatabasePassword.

Once the database and user exist, you can restore from your backup which will create tables, insert data and configure rights for your rt_user user.

    gunzip -c rt-20141014.sql.gz | psql rt4

This may need to be run as the postgres user or some other admin level user who can create tables.

Restore over an existing database

If something terrible happened this morning and you want to roll back to your backups, or if you want to update a dev server using your backups, you will need to drop your database and recreate a fresh one to restore into. RT can drop and recreate the database for you using:

    /opt/rt4/sbin/rt-setup-database --action drop
    /opt/rt4/sbin/rt-setup-database --action create

Remember that this will completely destroy the existing data and create a fresh database. Your rt_user user will remain untouched. Once this is complete, you can restore from your backup which will create tables and insert data and configure rights for the rt_user.

    gunzip -c rt-20141014.sql.gz | psql rt4

After Restoring

Postgres will generally perform poorly after restoring from backups because it has outdated index statistics. You should run analyze after your restore is complete. If you'd like to watch the progress, you can run analyze verbose.

FILESYSTEM

You will want to back up, at the very least, the following directories and files:

/opt/rt4

RT's source code, configuration, GPG data, and plugins. Your install location may be different, of course.

You can omit var/mason_data and var/session_data if you'd like since those are temporary caches. Don't omit all of var/ however as it may contain important GPG data.

Webserver configuration

Often /etc/httpd or /etc/apache2. This will depend on your OS, web server, and internal configuration standards.

/etc/aliases

Your incoming mail aliases mapping addresses to queues.

Mail server configuration

If you're running an MTA like Postfix, Exim, SendMail, or qmail, you'll want to backup their configuration files to minimize restore time. "Lightweight" mail handling programs like fetchmail, msmtp, and ssmtp will also have configuration files, although usually not as many nor as complex. You'll still want to back them up.

The location of these files is highly dependent on what software you're using.

Crontab containing RT's cronjobs

This may be /etc/crontab, /etc/cron.d/rt, a user-specific crontab file (crontab -l $USER), or some other file altogether. Even if you only have the default cronjobs in place, it's one less piece to forget during a restore. If you have custom rt-crontool invocations, you don't want to have to recreate those.

External storage

If you use RT::ExternalStorage, you will want to backup the attachments in your chosen storage engine.

If you're using RT::ExternalStorage::Disk, then you need only back up the files under the Path option under %ExternalStorage in your RT_SiteConfig.pm.

If you're using a cloud storage engine like RT::ExternalStorage::AmazonS3, consult that service's documentation regarding backups.

Simply saving a tarball should be sufficient, with something like:

    tar czvpf rt-backup-`date +%Y%m%d`.tar.gz /opt/rt4 /etc/aliases /etc/httpd ...

Be sure to include all the directories and files you enumerated above!