• NAME
• SYNOPSIS
  • OPTIONS
• GARBAGE COLLECTION

NAME

rt-serializer - Serialize an RT database to disk

SYNOPSIS

    rt-validator --check && rt-serializer

This script is used to write out the entire RT database to disk, for later import into a different RT instance. It requires that the data in the database be self-consistent, in order to do so; please make sure that the database being exported passes validation by rt-validator before attempting to use rt-serializer.

While running, it will attempt to estimate the number of remaining objects to be serialized; these estimates are pessimistic, and will be incorrect if --no-users, --no-groups, or --no-tickets are used.

If the controlling terminal is large enough (more than 25 columns high) and the gnuplot program is installed, it will also show a textual graph of the queue size over time.

OPTIONS

--directory name

The name of the output directory to write data files to, which should not exist yet; it is a fatal error if it does. Defaults to ./$Organization:Date/, where $Organization is as set in RT_SiteConfig.pm, and Date is today's date.

--force

Remove the output directory before starting.

--size megabytes

By default, rt-serializer chunks its output into data files which are around 32Mb in size; this option is used to set a different threshold size, in megabytes. Note that this is the threshold after which it rotates to writing a new file, and is as such the lower bound on the size of each output file.

--no-users

By default, all privileged users are serialized; passing --no-users limits it to only those users which are referenced by serialized tickets and history, and are thus necessary for internal consistency.

--no-groups

By default, all groups are serialized; passing --no-groups limits it to only system-internal groups, which are needed for internal consistency.

--no-assets

By default, all assets are serialized; passing --no-assets skips assets during serialization.

--no-disabled

By default, all queues, custom fields, etc, including disabled ones, are serialized; passing --no-disabled skips such disabled records during serialization.

--no-deleted

By default, all tickets and assets, including deleted ones, are serialized; passing --no-deleted skips deleted tickets and assets during serialization.

--scrips

No scrips or templates are serialized by default; this option forces all scrips and templates to be serialized.

--acls

No ACLs are serialized by default; this option forces all ACLs to be serialized.

--no-tickets

Skip serialization of all ticket data.

--limit-queues

Takes a list of queue IDs or names separated by commas. When provided, only that set of queues (and the tickets in them) will be serialized.

--limit-cfs

Takes a list of custom field IDs or names separated by commas. When provided, only that set of custom fields will be serialized.

--hyperlink-unmigrated

Replace links to local records which are not being migrated with hyperlinks. The hyperlinks will use the serializing RT's configured URL.

Without this option, such links are instead dropped, and transactions which had updated such links will be replaced with an explanatory message.

--no-transactions

Skip serialization of all transactions on any records (not just tickets).

--clone

Serializes your entire database, creating a clone. This option should be used if you want to migrate your RT database from one database type to another (e.g. MySQL to Postgres). It is an error to combine --clone with any option that limits object types serialized. No dependency walking is performed when cloning. rt-importer will detect that your serialized data set was generated by a clone.

--all

Serializes your entire database, creating a clone-like data. Both --all and --clone do not check dependencies, the difference is --all generates UIDs: it means the ids in source instance do not necessirily be synced to target instance, which makes it quite useful to fully merge multiple RT instances. Use --clone instead if you really want to keep ids in source instance.

--incremental

Will generate an incremenal serialized dataset using the data stored in your IncrementalRecords database table. This assumes that you have created that table and run RT using the Record_Local.pm shim as documented in docs/incremental-export/.

--gc n

Adjust how often the garbage collection sweep is done; lower numbers are more frequent. See "GARBAGE COLLECTION".

--page n

Adjust how many rows are pulled from the database in a single query. Disable paging by setting this to 0. Defaults to 100.

Keep in mind that rows from RT's Attachments table are the limiting factor when determining page size. You should likely be aiming for 60-75% of your total memory on an otherwise unloaded box.

--quiet

Do not show graphical progress UI.

--verbose

Do not show graphical progress UI, but rather log was each row is written out.

GARBAGE COLLECTION

rt-serializer maintains a priority queue of objects to serialize, or searches which may result in objects to serialize. When inserting into this queue, it does no checking if the object in question is already in the queue, or if the search will contain any results. These checks are done when the object reaches the front of the queue, or during periodic garbage collection.

During periodic garbage collection, the entire queue is swept for objects which have already been serialized, occur more than once in the queue, and searches which contain no results in the database. This is done to reduce the memory footprint of the serialization process, and is triggered when enough new objects have been placed in the queue. This parameter is tunable via the --gc parameter, which defaults to running garbage collection every 5,000 objects inserted into the queue; smaller numbers will result in more frequent garbage collection.

The default of 5,000 is roughly tuned based on a database with several thousand tickets, but optimal values will vary wildly depending on database configuration and size. Values as low as 25 have provided speedups with smaller databases; if speed is a factor, experimenting with different --gc values may be helpful. Note that there are significant boundary condition changes in serialization rate, as the queue empties and fills, causing the time estimates to be rather imprecise near the start and end of the process.

Setting --gc to 0 turns off all garbage collection. Be aware that this will bloat the memory usage of the serializer. Any negative value for --gc turns off periodic garbage collection and instead objects already serialized or in the queue are checked for at the time they would be inserted.