In today’s article we are going to provide a reference of all configuration options that can be set in migration definition files. Additional configuration options available for migrations defined as configuration will also be listed. Finally, we present the configuration options for migrations groups.

List of configuration options in YAML definition files

General configuration keys

The following keys can be set for any Drupal migration.

id key

A required string value. It serves as the identifier for the migration. The value should be a machine name. That is, lowercase letters with underscores instead of spaces. The value is for creating mapping and messages tables. For example, if the id is ud_migrations, the Migrate API will create the following migrations migrate_map_ud_migrations and migrate_message_ud_migrations.

label key

A string value. The human-readable label for the migration. The value is used in different interfaces to refer to the migration.

audit key

A boolean value. Defaults to FALSE. It indicates whether the migration is auditable. When set to TRUE, a warning is displayed if entities might be overridden when the migration is executed. For example, when doing an upgrade from a previous version of Drupal, nodes created in the new site before running the automatic upgrade process would be overridden and a warning is logged. The Migrate API checks if the highest destination ID is greater than the highest source ID.

migration_tags key

An array value. It can be set to an optional list of strings representing the tags associated with the migration. They are used by the plugin manager for filtering. For example, you can import or rollback all migrations with the Content tag using the following Drush commands provided by the Migrate Tools module:

$ drush migrate:import --tag='Content'
$ drush migrate:rollback --tag='Content'

source key

A nested array value. This represents the configuration of the source plugin. At a minimum, it contains an id key which indicates which source plugin to use for the migration. Possible values include embedded_data for hardcoded data; csv for CSV files; url for JSON feeds, XML files, and Google Sheets; spreadsheet for Microsoft Excel and LibreOffice Calc files; and many more. Each plugin is configured differently. Refer to our list of configuration options for source plugins to find out what is available for each of them. Additionally, in this section you can define source contents that can be later used in the process pipeline.

process key

A nested array value. This represents the configuration of how source data will be processed and transformed to match the expected destination structure. This section contains a list of entity properties (e.g. nid for a node) and fields (e.g. field_image in the default article content type). Refer to our list of properties for content entities including Commerce related entities to find out which properties can be set depending on your destination (e.g. nodes, users, taxonomy terms, files and images, paragraphs, etc.). For field mappings, you use the machine name of the field as configured in the entity bundle. Some fields have complex structures so you migrate data into specific subfields. Refer to our list of subfields per field type to determine which options are available. When migrating multivalue fields, you might need to set deltas as well. Additionally, you can have pseudofields to store temporary values within the process pipeline.

For each entity property, field, or pseudofield, you can use one or more process plugins to manipulate the data. Many of them are provided by Drupal core while others become available when contributed modules are installed on the site like Migrate Plus and Migrate Process Extra. Throughout the 31 days of migrations series, we provided examples of how many process plugins are used. Most of the work for migrations will be devoted to configuring the right mappings in the process section. Make sure to check our debugging tips in case some values are not migrated properly.

destination key

A nested array value. This represents the configuration of the destination plugin. At a minimum, it contains an id key which indicates which destination plugin to use for the migration. Possible values include entity:node for nodes, entity:user for users, entity:taxonomy_term for taxonomy terms, entity:file for files and images, entity_reference_revisions:paragraph for paragraphs, and many more. Each plugin is configured differently. Refer to our list of configuration options for destination plugins to find out what is available for each of them.

This is an example migration from the ud_migrations_csv_source module used in the article on CSV sources.

id: udm_csv_source_paragraph
label: 'UD dependee paragraph migration for CSV source example'
migration_tags:
  - UD CSV Source
  - UD Example
source:
  plugin: csv
  path: modules/custom/ud_migrations/ud_migrations_csv_source/sources/udm_book_paragraph.csv
  ids: [book_id]
  header_offset: null
  fields:
    - name: book_id
    - name: book_title
    - name: 'Book author'
process:
  field_ud_book_paragraph_title: book_title
  field_ud_book_paragraph_author: 'Book author'
destination:
  plugin: 'entity_reference_revisions:paragraph'
  default_bundle: ud_book_paragraph

migration_dependencies key

A nested array value. The value is used by the Migrate API to make sure the listed migrations are executed in advance of the current one. For example, a node migration might require users to be imported first so you can specify who is the author of the node. Also, it is possible to list optional migrations so that they are only executed in case they are present. The following example from the d7_node.yml migration shows how key can be configured:

migration_dependencies:
  required:
    - d7_user
    - d7_node_type
  optional:
    - d7_field_instance
    - d7_comment_field_instance

To configure the migration dependencies you specify required and optional subkeys whose values are an array of migration IDs. If no dependencies are needed, you can omit this key. Alternatively, you can set either required or optional dependencies without having to specify both keys. As of Drupal 8.8 an InvalidPluginDefinitionException will be thrown if the migration_dependencies key is incorrectly formatted.

class key

A string value. If set, it should point to the class used as the migration plugin. The MigrationPluginManager sets this key to \Drupal\migrate\Plugin\Migration by default. Whatever class specified here should implement the MigrationInterface. This configuration key rarely needs to be set as the default value can be used most of the time. In Drupal core there are few cases where a different class is used as the migration plugin:

deriver key

A string value. If set, it should point to the class used as a plugin deriver for this migration. This is an advanced topic that will be covered in a future entry. In short, it is a mechanism in which new migration plugins can be created dynamically from a base template. For example, the d7_node.yml migration uses the D7NodeDeriver to create one node migration per content type during a Drupal upgrade operation. In this case, the configuration key is set to Drupal\node\Plugin\migrate\D7NodeDeriver. There are many other derivers used by the Migrate API including D7NodeDeriver, D7TaxonomyTermDeriver, EntityReferenceTranslationDeriver, D6NodeDeriver, and D6TermNodeDeriver.

field_plugin_method key

A string value. This key must be set only in migrations that use Drupal\migrate_drupal\Plugin\migrate\FieldMigration as the plugin class. They take care of importing fields from previous versions of Drupal. The following is a list of possible values:

  • alterFieldMigration as set by d7_field.yml.
  • alterFieldFormatterMigration as set by d7_field_formatter_settings.yml.
  • alterFieldInstanceMigration as set by d7_field_instance.yml.
  • alterFieldWidgetMigration as set by d7_field_instance_widget_settings.yml

There are Drupal 6 counterparts for these migrations. Note that the field_plugin_method key is a replacement for the deprecated cck_plugin_method key.

provider key

An array value. If set, it should contain a list of module machine names that must be enabled for this migration to work. Refer to the d7_entity_reference_translation.yml and d6_entity_reference_translation.yml migrations for examples of possible values. This key rarely needs to be set. Usually the same module providing the migration definition file is the only one needed for the migration to work.

Deriver specific configuration keys

It is possible that some derivers require extra configuration keys to be set. For example, the EntityReferenceTranslationDeriver the target_types to be set. Refer to the d7_entity_reference_translation.yml and d6_entity_reference_translation.yml migrations for examples of possible values. These migrations are also interesting because the source, process, and destination keys are not configured in the YAML definition files. They are actually set dynamically by the deriver.

Migration configuration entity keys

The following keys should be used only if the migration is created as a configuration entity using the Migrate Plus module. Only the migration_group key is specific to migrations as configuration entities. All other keys apply for any configuration entity in Drupal. Refer to the ConfigEntityBase abstract class for more details on how they are used.

migration_group key

A string value. If set, it should correspond to the id key of a migration group configuration entity. This allows inheriting configuration values from the group. For example, the database connection for the source configuration. Refer to this article for more information on sharing configuration using migration groups. They can be used to import or rollback all migrations within a group using the following Drush commands provided by the Migrate Tools module:

$ drush migrate:import --group='udm_config_group_json_source'
$ drush migrate:rollback --group='udm_config_group_json_source'

uuid key

A string value. The value should be a UUID v4. If not set, the configuration management system will create a UUID on the fly and assign it to the migration entity. Refer to this article for more details on setting UUIDs for migrations defined as configuration entities.

langcode key

A string value. The language code of the entity's default language. English is assumed by default. For example: en.

status key

A boolean value. The enabled/disabled status of the configuration entity. For example: true.

dependencies key

A nested array value. Configuration entities can declare dependencies on modules, themes, content entities, and other configuration entities. These dependencies can be recalculated on save operations or enforced. Refer to the ConfigDependencyManager class’ documentation for details on how to configure this key. One practical use of this key is to automatically remove the migration (configuration entity) when the module that defined it is uninstalled. To accomplish this, you need to set an enforced module dependency on the same module that provides the migration. This is explained in the article on defining Drupal migrations as configuration entities. For reference, below is a code snippet from that article showing how to configure this key:

uuid: b744190e-3a48-45c7-97a4-093099ba0547
id: udm_config_json_source_node_local
label: 'UD migrations configuration example'
dependencies:
  enforced:
    module:
      - ud_migrations_config_json_source

Migration group configuration entity keys

Migration groups are also configuration entities. That means that they can have uuid, langcode, status, and dependencies keys as explained before. Additionally, the following keys can be set. These other keys can be set for migration groups:

id key

A required string value. It serves as the identifier for the migration group. The value should be a machine name.

label key

A string value. The human-readable label for the migration group.

description key

A string value. More information about the group.

source_type key

A string value. Short description of the type of source. For example: "Drupal 7" or "JSON source".

module key

A string value. The machine name of a dependent module. This key rarely needs to be set. A configuration entity is always dependent on its provider, the module defining the migration group.

shared_configuration key

A nested array value. Any configuration key for a migration can be set under this key. Those values will be inherited by any migration associated with the current group. Refer to this article for more information on sharing configuration using migration groups. The following is an example from the ud_migrations_config_group_json_source module from the article on executing migrations from the Drupal interface.

uuid: 78925705-a799-4749-99c9-a1725fb54def
id: udm_config_group_json_source
label: 'UD Config Group (JSON source)'
description: 'A container for migrations about individuals and their favorite books. Learn more at https://understanddrupal.com/migrations.'
source_type: 'JSON resource'
shared_configuration:
  dependencies:
    enforced:
      module:
        - ud_migrations_config_group_json_source
  migration_tags:
    - UD Config Group (JSON Source)
    - UD Example
  source:
    plugin: url
    data_fetcher_plugin: file
    data_parser_plugin: json
    urls:
      - modules/custom/ud_migrations/ud_migrations_config_group_json_source/sources/udm_data.json

What did you learn in today’s article? Did you know there were so many configuration options for migration definition files? Were you aware that some keys apply only when migrations are defined as configuration entities? Have you used migrations groups to share configuration across migrations? Share your answers in the comments. Also, I would be grateful if you shared this blog post with friends and colleagues.

Read more and discuss at agaric.coop.

https://agaric.coop/blog/drupal-migrations-reference-list-configuration-options…