• Summary of initialdata files
• Examples
• What can be in an initialdata file?
  • @Users
  • @Groups
  • @CustomRoles
  • @Queues
  • @CustomFields
  • @ACL
    • Pick one or more Rights
    • Pick a level: on a queue, on a CF, or globally
    • Pick a Principal: User or Group or Role
    • Common cases
    • Troubleshooting
  • @Scrips
  • @ScripActions
  • @ScripConditions
  • @Templates
  • @Attributes
  • @Initial
  • @Final
• Running an initialdata file
• Implementation details
• JSON initialdata
  • Example JSON File
  • Limitations
• Automatically Generating Initialdata to Migrate Changes
  • Validate Your Starting Data
  • Create a Baseline Checkpoint
  • Make Configuration Changes
  • Export Changes as Initialdata
  • Importing Data

Summary of initialdata files

It's often useful to be able to test configuration/database changes and then apply the same changes in production without manually clicking around. It's also helpful if you're developing customizations or extensions to be able to get a fresh database back to the state you want for testing/development.

This documentation applies to careful and thorough sysadmins as well as extension authors who need to make database changes easily and repeatably for new installs or upgrades.

Examples

RT ships with many initialdata files, only one of which is used to configure a fresh install; the rest are used for upgrades, but function the same despite being named differently.

    etc/initialdata
    etc/upgrade/*/content

The upgrade "content" files are meant to be incremental changes applied on top of one another while the top level initialdata file is for fresh RT installs.

Extensions may also ship with database changes in such files. You may find some in your install with:

    find local/plugins -name initialdata -or -name content

What can be in an initialdata file?

By default, initialdata files are Perl, but often consist primarily of a bunch of data structures defining the new records you want and not much extra code. There's nothing stopping you from writing a bunch of code, however!

RT's initialdata importer is also pluggable and you can add handlers for other formats using the option "InitialdataFormatHandlers" in RT_Config. Starting in RT 5.0, RT supports JSON in addition to Perl. See below for details.

The basic template of a new Perl initialdata file should look something like this:

    use strict;
    use warnings;

    our @Queues = (
        # some definitions here
    );

    our @Groups = (
        # some other definitions here
    );

    1;

The @Queues and @Groups arrays are expected by RT and should contain hashref definitions. There are many other arrays RT will look for and act on, described below. None are required, all may be used. Keep in mind that since they're just normal Perl arrays, you can push onto them from a loop or grep out definitions based on conditionals or generate their content with map, etc.

The complete list of possible arrays which can be used, along with descriptions of the values to place in them, is below.

@Users

    push @Users, {
        Name        => 'john.doe',
        Password    => 'changethis',
        Language    => 'fr',
        Timezone    => 'America/Vancouver',
        Privileged  => 1,
        Disabled    => 0,
    };

Each hashref in @Users is treated as a new user to create and passed straight into RT::User->Create. All of the normal user fields are available, as well as Privileged and Disabled (both booleans) which will do the appropriate internal group/flag handling. Also accepts an Attributes key, which is equivalent to pushing its arrayref of values onto @Attributes, below, with Object set to the new user.

For a full list of fields, read the documentation for "Create" in RT::User.

@Groups

    push @Groups, {
        Name        => 'Example Employees',
        Description => 'All of the employees of my company',
        Members     => { Users =>  [ qw/ alexmv trs falcone / ],
                         Groups => [ qw/ extras / ] },
    };

Creates a new RT::Group for each hashref. In almost all cases you'll want to follow the example above to create a group just as if you had done it from the admin interface.

In addition to the Members option shown above, which can take both users and groups, the MemberOf field may be a single value or an array ref. Each value should be a user-defined group name or hashref to pass into "LoadByCols" in RT::Group. Each group found will have the new group added as a member.

It also accepts an Attributes key, which is equivalent to pushing its arrayref of values onto @Attributes, below, with Object set to the new group.

@CustomRoles

    push @CustomRoles, {
        Name        => 'Support Rep',
        Description => 'Support representative assigned for this ticket',
        ApplyTo     => 'General',
    };

Creates a new RT::CustomRole for each hashref. Custom roles must be applied per queue; they cannot be applied globally. See the ApplyTo entry as described in "@CustomFields" below for more information.

@Queues

    push @Queues, {
        Name                => 'Helpdesk',
        CorrespondAddress   => 'help@example.com',
        CommentAddress      => 'help-comment@example.com',
    };

Creates a new RT::Queue for each hashref. Refer to the documentation of "Create" in RT::Queue for the fields you can use. It also accepts an Attributes key, which is equivalent to pushing its arrayref of values onto @Attributes, below, with Object set to the new queue.

@CustomFields

    push @CustomFields, {
        Name        => 'Favorite color',
        Type        => 'FreeformSingle',
        LookupType  => 'RT::Queue-RT::Ticket',
    };

Creates a new RT::CustomField for each hashref. It is the most complex of the initialdata structures. The most commonly used fields are:

Name

The name of this CF as displayed in RT.

Description

A short summary of what this CF is for.

ApplyTo

May be a single value, or an array reference of such; each should be either an ID or Name. If omitted, the CF is applied globally. This should not be used for User or Group custom fields.

This argument may also be passed via Queue, for backwards compatibility, which also defaults the LookupType to RT::Queue-RT::Ticket.

Type

One of the following on the left hand side:

    SelectSingle            # Select one value
    SelectMultiple          # Select multiple values

    FreeformSingle          # Enter one value
    FreeformMultiple        # Enter multiple values

    Text                    # Fill in one text area
    HTML                    # Fill in one HTML area
    Wikitext                # Fill in one wikitext area

    BinarySingle            # Upload one file
    BinaryMultiple          # Upload multiple files

    ImageSingle             # Upload one image
    ImageMultiple           # Upload multiple images

    Combobox                # Combobox: Select or enter one value

    AutocompleteSingle      # Enter one value with autocompletion
    AutocompleteMultiple    # Enter multiple values with autocompletion

    Date                    # Select date
    DateTime                # Select datetime

    IPAddressSingle         # Enter one IP address
    IPAddressMultiple       # Enter multiple IP addresses

    IPAddressRangeSingle    # Enter one IP address range
    IPAddressRangeMultiple  # Enter multiple IP address ranges

If you don't specify "Single" or "Multiple" in the type, you must specify MaxValues.

LookupType

Labelled in the CF admin page as "Applies to". This determines whether your CF is for Tickets, Transactions, Users, Groups, or Queues. Possible values:

    RT::Queue-RT::Ticket                    # Tickets
    RT::Queue-RT::Ticket-RT::Transaction    # Transactions
    RT::User                                # Users
    RT::Group                               # Groups
    RT::Queue                               # Queues
    RT::Class-RT::Article                   # Articles

Ticket CFs are the most common, meaning RT::Queue-RT::Ticket is the most common LookupType.

RenderType

Only valid when Type is "Select". Controls how the CF is displayed when editing it. Valid values are: Select box, List, Dropdown, and Checkbox.

See "SetRenderType" in RT::CustomField for details.

MaxValues

Determines whether this CF is a Single or Multiple type. 0 means multiple. 1 means single.

Make sure to set the MaxValues field appropriately, otherwise you can end up with unsupported CF types like a "Select multiple dates" (it doesn't Just Work).

You can also use old-style Types which end with "Single" or "Multiple", for example: SelectSingle, SelectMultiple, FreeformSingle, etc.

Values

Values should be an array ref (never a single value!) of hashrefs representing new RT::CustomFieldValue objects to create for the new custom field. This only makes sense for "Select" CFs. An example:

    my $i = 1;
    push @CustomFields, {
        LookupType  => 'RT::Queue-RT::Ticket',  # for Tickets
        Name        => 'Type of food',
        Type        => 'SelectSingle',  # SelectSingle is the same as: Type => 'Select', MaxValues => 1
        RenderType  => 'Dropdown',
        Values      => [
            { Name => 'Fruit',      Description => 'Berries, peaches, tomatos, etc', SortOrder => $i++ },
            { Name => 'Vegetable',  Description => 'Asparagus, peas, lettuce, etc',  SortOrder => $i++ },
            # more values as such...
        ],
    };

In order to ensure the same sorting of Values, set SortOrder inside each value. A clever way to do this easily is with a simple variable you increment each time (as above with $i). You can use the same variable throughout the whole file, and don't need one per CF.

BasedOn

Name or ID of another Select Custom Field. This makes the named CF the source of categories for your values.

Pattern

The regular expression text (not qr//!) used to validate values.

It also accepts an Attributes key, which is equivalent to pushing its arrayref of values onto @Attributes, below, with Object set to the new custom field.

Refer to the documentation and implementation of "Create" in RT::CustomField and "Create" in RT::CustomFieldValue for the full list of available fields and allowed values.

@ACL

@ACL is very useful for granting rights on your newly created records or setting up a standard system configuration. It is one of the most complex initialdata structures.

Pick one or more Rights

All ACL definitions expect a key named Right with the internal right name you want to grant; alternately, it may contain an array reference of right names. The internal right names are visible in RT's admin interface in grey next to the longer descriptions.

Pick a level: on a queue, on a CF, or globally

After picking a Right, you need to specify on what object the right is granted. This is different than the user/group/role receiving the right.

Granted on a custom field by name (or ID), potentially a global or queue

    CF => 'Name',
    LookupType => 'RT::User',  # optional, in case you need to disambiguate

Granted on a queue

    Queue => 'Name',

Granted on a custom field applied to a specific queue

    CF      => 'Name',
    Queue   => 'Name',

Granted on a custom field applied to some other object

    # This finds the CF named "Name" applied to Articles in the
    # "Responses" class
    CF         => 'Name',
    LookupType => RT::Article->CustomFieldLookupType,
    ObjectId   => 'Responses',

Granted on some other object (article Classes, etc)

    ObjectType => 'RT::Class',
    ObjectId   => 'Name',

Granted globally

Specifying none of the above will get you a global right.

There is currently no way to grant rights on a group or article class level. Note that you can grant rights to a group; see below. If you need to grants rights on a group or article class level, you'll need to write an @Final subref to handle it using the RT Perl API.

Pick a Principal: User or Group or Role

Finally you need to specify to what system group, system/queue role, user defined group, or user you want to grant the right to.

An internal user group

    GroupDomain => 'SystemInternal',
      GroupType => 'Everyone, Privileged, or Unprivileged'

A system-level role

    GroupDomain => 'RT::System-Role',
      GroupType => 'Requestor, Owner, AdminCc, or Cc'

A queue-level role

    GroupDomain => 'RT::Queue-Role',
      Queue     => 'Name',
      GroupType => 'Requestor, Owner, AdminCc, or Cc',

A system-level custom role

    GroupDomain => 'RT::System-Role',
    CustomRole  => 'Supervisor',

A queue-level custom role

    Queue       => 'Customer Support',
    GroupDomain => 'RT::Queue-Role',
    CustomRole  => 'Customer',

A group you created

    GroupDomain => 'UserDefined',
      GroupId   => 'Name'

Individual user

    UserId => 'Name or email or ID'

Common cases

You're probably looking for definitions like these most of the time.

Grant a global right to a group you created

    { Right       => '...',
      GroupDomain => 'UserDefined',
      GroupId     => 'Name' }

Grant a queue-level right to a group you created

    { Queue       => 'Name',
      Right       => '...',
      GroupDomain => 'UserDefined',
      GroupId     => 'Name' }

Grant a CF-level right to a group you created

    { CF          => 'Name',
      Right       => '...',
      GroupDomain => 'UserDefined',
      GroupId     => 'Name' }

Since you often want to grant a list of rights on the same object/level to the same role/group/user, we generally use Perl loops and operators to aid in the generation of @ACL without repeating ourselves.

    # Give Requestors globally the right to see tickets, reply, and see the
    # queue their ticket is in
    push @ACL, map {
        {
            Right       => $_,
            GroupDomain => 'RT::System-Role',
            GroupType   => 'Requestor',
        }
    } qw(ShowTicket ReplyToTicket SeeQueue);

Troubleshooting

The best troubleshooting is often to see how the rights you define in @ACL show up in the RT admin interface.

@Scrips

Creates a new RT::Scrip for each hashref. Refer to the documentation of "Create" in RT::Scrip for the fields you can use.

Additionally, the Queue field is specially handled to make it easier to setup the same Scrip on multiple queues:

Globally

    Queue => 0,

Single queue

    Queue => 'General', # Name or ID

Multiple queues

    Queue => ['General', 'Helpdesk', 13],   # Array ref of Name or ID

@ScripActions

Creates a new RT::ScripAction for each hashref. Refer to the documentation of "Create" in RT::ScripAction for the fields you can use.

@ScripConditions

Creates a new RT::ScripCondition for each hashref. Refer to the documentation of "Create" in RT::ScripCondition for the fields you can use.

@Templates

Creates a new RT::Template for each hashref. Refer to the documentation of "Create" in RT::Template for the fields you can use.

@Attributes

An array of RT::Attributes to create. You likely don't need to mess with this. If you do, know that the key Object is expected to be an RT::Record object or a subroutine reference that returns an object on which to call AddAttribute. If you don't provide Object or it's undefined, RT->System will be used.

Here is an example of using a subroutine reference as a value for Object:

    @Attributes = ({
        Name        => 'SavedSearch',
        Description => 'New Tickets in SomeQueue',
        Object      => sub {
            my $GroupName = 'SomeQueue Group';
            my $group     = RT::Group->new( RT->SystemUser );

            my( $ret, $msg ) = $group->LoadUserDefinedGroup( $GroupName );
            die $msg unless $ret;

            return $group;
        },
        Content     => {
            Format =>  <<'        END_OF_FORMAT',
    ....
            END_OF_FORMAT
            Query   => "Status = 'new' AND Queue = 'SomeQueue'",
            OrderBy => 'id',
            Order   => 'DESC'
        },
    });

@Initial

See @Final below.

@Final

@Initial and @Final are special and let you write your own processing code that runs before anything else or after everything else. They are expected to be arrays of subrefs (usually anonymous) like so:

    our @Final = (sub {
        RT->Logger->info("Finishing up!");
    });

You have the full power of RT's Perl libraries at your disposal. Be sure to do error checking and log any errors with RT->Logger->error("...")!

Running an initialdata file

    /opt/rt5/sbin/rt-setup-database --action insert --datafile /path/to/your/initialdata

This may prompt you for a database password.

Implementation details

All the handling of initialdata files is done in RT::Handle->InsertData. If you want to know exactly what's happening with each array, your best bet is to start reading the code there.

RT takes care of the ordering so that your new queues are created before it processes the new ACLs for those queues. This lets you refer to new queues you just created by Name.

JSON initialdata

To configure RT to load JSON-formatted initialdata, add this option:

    Set( $InitialdataFormatHandlers,
         [
            'perl',
            'RT::Initialdata::JSON',
         ]
       );

There is a direct one-to-one mapping between the Perl initialdata structures and the JSON file data structures, with the exception of how the top-level elements are composed. In the Perl file, each array is named separately, like this:

    @Queues = ( {...}, {...} );
    @Scrips = ( {...}, {...} );

To represent this in JSON, the root-level element is a JSON object--a key/value structure that is analogous to a perl hash. The key is the name of the array you would normally type as @Queues, and the value is a JSON array:

    {
        "Queues":[ {...}, {...} ],
        "Scrips":[ {...}, {...} ]
    }

You can find details on JSON formatting rules at http://json.org.

Example JSON File

There is a JSON file with examples in the RT test files here:

    F<t/data/initialdata/initialdata.json>

Limitations

The JSON initialdata format cannot support the full functionality of the perl format, as the perl format allows executable code. Specifically, these elements cannot be used, and if present, will be ignored:

@Initial

No Initial elements will be used.

@Final

No Final elements will be used.

Automatically Generating Initialdata to Migrate Changes

If you have RT instances running in multiple environments like dev, test, and production, you may want to pilot new configuration in dev, then move that configuration to production once it is validated. RT provides tooling to help export just new configuration changes as Initialdata and then load it into other environments.

Note that for systems with very large databases, the generated initialdata files can be extremely large. With such systems, the process described below can take much longer and require more resources than on a typical RT system.

Validate Your Starting Data

It is good practice to run the rt-validator tool to identify and resolve any errors in the RT database before starting:

    /opt/rt5/sbin/rt-validator --check
    /opt/rt5/sbin/rt-validator --check --resolve # If necessary

Create a Baseline Checkpoint

You can then create an initial export of your current configuration using the rt-dump-initialdata utility. The generated Initialdata file will contain all of your current configuration, but no ticket, asset, or article data.

    /opt/rt5/sbin/rt-dump-initialdata --sync

When finished, you'll have a directory and file in the format yoursite.com:YYYY-MM-DD/initialdata.json.

Make Configuration Changes

Make changes to your RT configuration through the web interface. You might add a new queue, create a new group, and add rights for that group on your new queue. All of these will be saved in the dev system database as you work.

Export Changes as Initialdata

When you've completed the changes you want to migrate to another RT, re-run rt-dump-initialdata, providing the first export file as an argument.

    /opt/rt5/sbin/rt-dump-initialdata --sync --directory changes/ \
        --base yoursite.com:YYYY-MM-DD/initialdata.json

A new output directory must be provided if you are making a diff file on the same day as the original configuration dump file was produced.

You will now have a new directory, changes/, with two files. initialdata.json is the full dump of configuration data, and changes.json has only that data which changed since the original dump. The initialdata file is JSON, so you can view it with a regular text editor. Before migrating changes to the target RT instance, you can review the contents. You can even edit the file directly if you need to add or remove some data.

Importing Data

By default, RT is not configured to import JSON data. The JSON format handler needs to be enabled before the change file produced in the prior step can be imported. See "@InitialdataFormatHandlers" in RT_Config for details on enabling the JSON format.

Copy changes.json to your target server and run the following:

    /opt/rt5/sbin/rt-setup-database --action insert \
        --datafile path/to/changes.json

The new configurations you made in development will now be available in your new environment.

It is recommended that you make a database backup of your target RT before importing configuration changes. See Backups in Database for more information.

← Back to index