Agaric Collective: What is the difference between migration tags and migration groups in Drupal?

In the previous post we talked about migration groups provided by the Migrate Plus module. Today, we are going to compare them to migration tags. Along the way, we are going to dive deeper into how they work and provide tips to avoid problems when working with them. Let’s get started.

Example migration group definition containing migration tags.

What is the difference between migration tags and migration groups?

In the article on declaring migration dependencies we briefly touched on the topic of tags. Here is a summary of migration tags:

They are a feature provided by the core Migrate API.
Multiple tags can be assigned to a single migration.
They are defined in the migration definition file alone and do not require creating a separate file.
Both Migrate Tools and Migrate Run provide a flag to execute operations by tag.
They do not allow you to share configuration among migrations tagged with the same value.

Here is a summary of migration groups:

You need to install the Migrate Plus module to take advantage of them.
Only one group can be assigned to any migration.
You need to create a separate file to declare group. This affects the readability of migrations as their configuration will be spread over two files.
Only the Migrate Tools provides a flag to execute operations by group.
They offer the possibility to share configuration among members of the same group.
Any shared configuration could be overridden in the individual migration definition files.

What do migration tags and groups have in common?

The ability to put together multiple migrations under a single name. This name can be used to import or rollback all the associated migrations in one operation. This is true for the migrate:import and migrate:rollback Drush commands provided by Migrate Plus. What you have to do is use the --group or --tag flags, respectively. The following snipped shows an example of importing and rolling back the migrations by group and tag:

$ drush migrate:import --tag='UD Config Group (JSON Source)'
$ drush migrate:rollback --tag='UD Config Group (JSON Source)'

$ drush migrate:import --group='udm_config_group_json_source'
$ drush migrate:rollback --group='udm_config_group_json_source'

Note: You might get errors indicating that the "--tag" or "--group" options do not accept a value. See this issue if you experience that problem.

Neither migration tags nor groups replace migration dependencies. If there are explicit migration dependencies among the members of a tag or group, the Migrate API will determine the order in which the migrations need to be executed. But if no dependencies are explicitly set, there is no guarantee the migrations will be executed in any particular order. Keep this in mind if you have separate migrations for entities that you later use to populate entity reference fields. Also note that migration dependencies are only executed automatically for import operations. Dependent migrations will not be rolled back automatically if the main migration is rolled back individually.

Can groups only be used for migrations defined as configuration entities?

Technically speaking, no. It is possible to use groups for migrations defined as code. Notwithstanding, migration groups can only be created as configuration entities. You would have to rebuild caches and sync configuration for changes in the migrations and groups to take effect, respectively. This is error prone and can lead to hard to debug issues.

Also, things might get confusing when executing migrations. The user interface provided by Migrate Plus works exclusively with migrations defined as configuration entities. The Drush commands provided by the same module work for both types of migrations: code and configuration. The default and null values for the migration_group key are handled a bit different between the user interface and the Drush commands. Moreover, the ability to execute operations per group using Drush commands is provided only by the Migrate Tools module. The Migrate Run module lacks this functionality.

Managing migrations as code or configuration should be a decision to take at the start of the project. If you want to use migration groups, or some of the other benefits provided by migrations defined as configuration, stick to them since the very beginning. It is possible to change at any point and the transition is straightforward. But it should be avoided if possible. In any case, try not to mix both workflows in a single project.

Tip: It is recommended to read this article to learn more about the difference between managing migrations as code or configuration.

Setting migration tags inside migration groups

As seen in this article, it is possible to use set migration tags as part of the shared configuration of a group. If you do this, it is not recommended to override the migration_tags key in individual migrations. The end result might not be what you expect. Consider the following snippets as example:

# Migration group configuration entity definition.
# File: migrate_plus.migration_group.udm_config_group_json_source.yml
uuid: 78925705-a799-4749-99c9-a1725fb54def
id: udm_config_group_json_source
label: 'UD Config Group (JSON source)'
description: 'A container for migrations about individuals and their favorite books. Learn more at https://understanddrupal.com/migrations.'
source_type: 'JSON resource'
  migration_tags:
    - UD Config Group (JSON Source)
    - UD Example

# Migration configuration entity definition.
# File: migrate_plus.migration.udm_config_group_json_source_node.yml
uuid: def749e5-3ad7-480f-ba4d-9c7e17e3d789
id: udm_config_group_json_source_node
label: 'UD configuration host node migration for migration group example (JSON source)'
migration_tags:
  - UD Lorem Ipsum
migration_group: udm_config_group_json_source
source: ...
process: ...
destination: ...
migration_dependencies: ...

The group configuration declares two tags: UD Config Group (JSON Source) and UD Example. The migration configuration overrides the tags to a single value UD Lorem Ipsum. What would you expect the final value for the migration_tags key be? Is it a combination of the three tags? Is it only the one key defined in the migration configuration?

The answer in this case is not very intuitive. The final migration will have two tags: UD Lorem Ipsum and UD Example. This has to do with how Migrate Plus merges the configuration from the group into the individual migrations. It uses the array_replace_recursive() PHP function which performs the merge operation based on array keys. In this example, UD Config Group (JSON Source) and UD Lorem Ipsum have the same index in the migration_tags array. Therefore, only one value is preserved in the final result.

The examples uses the migration_tags key as it is the subject of this article, but the same applies to any nested structure. Some configurations are more critical to a migration than a tag or group. Debugging a problem like this can be tricky. But the same applies to any configuration that has a nested structure. If the end result might be ambiguous, it is preferred to avoid the situation in the first place. In general, nested structures should only be set in either the group or the migration definition file, but not both. Additionally, all the recommendations for writing migrations presented in this article also apply here.

What did you learn in today’s blog post? Did you know the difference between migration tags and groups? Share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.