Dev Series: Migrating Data From Magento 1 To Magento 2 Using The Data Migration Tool

A few days ago I was asked to migrate orders and customers from a very old Magento 1 website to Magento 2.1.4. The website was on version 1.5.1.0 so it felt like an impossible task at the beginning. However, most of the Magento 2 website was already completed and we only needed to get orders and customers migrated.

Magento released a very powerful data migration tool, but this tool is meant to migrate an entire Magento 1 website to Magento 2, so it would overwrite whatever is already in there. As a result, the official recommendation is to run the tool on a brand new Magento 2 instance. However, the tool can also be setup to do a partial migration. In other words, to migrate only certain entities, such as Orders, Products, Categories or Customers, but this is not explained in any of the Magento docs, and there is a lot of configuration files in the tool, so I had to figure this out myself.

In this article I will share my experience and I will explain how to do a partial migration in order to get only orders and customers migrated.

The first step is to make sure your Magento 1 instance can be migrated. The data migration tool supports only Magento 1.6+ versions, so if you have an earlier version you’ll need to first upgrade it to the latest Magento 1 version. In my case I had to upgrade the website from version 1.5.1.0 to 1.9.3.2 and I have to say it was really easy, but you might encounter some issues blocking the upgrade that will need to be sorted on a case by case basis. Only the database needs to be upgraded since the tool accesses the database directly so you don’t need to worry about making the actual website work.

It is highly recommended to create a backup of the Magento 1 database and to truncate all the log tables before upgrading the website:

• log_customer
• log_visitor
• log_visitor_info
• log_url
• log_url_info
• log_quote
• report_viewed_product_index
• report_compared_product_index
• report_event
• catalog_compare_item

The second step is to install the data migration tool on the Magento 2 website. The recommended way for doing so is by adding it as a dependency to composer for the Magento 2 website. This can be done as follows:

cd <your Magento 2 install dir>
composer require magento/data-migration-tool:<version> 
compose install

Where <version> must match the version of the Magento 2 codebase.

Then you need to make sure your Magento 2 server can access the Magento 1 database, since the tool will connect to the database directly. You will get optimum performance if both the Magento 1 and Magento 2 databases are in the same Mysql server, so this is highly recommended.

After you install the data migration tool, the following directories will contain mapping and configuration files. Note that both community and enterprise editions can be migrated using this tool:

Magento CE:

• <your Magento 2 install dir>/vendor/magento/data-migration-tool/etc/ce-to-ce: Configuration and mapping files for migrating from Magento 1 CE to Magento 2 CE

Magento EE:

• <your Magento 2 install dir>/vendor/magento/data-migration-tool/etc/ce-to-ee: Configuration and mapping files for migrating from Magento 1 CE to Magento 2 EE
• <your Magento 2 install dir>/vendor/magento/data-migration-tool/etc/ee-to-ee: Configuration and mapping files for migrating from Magento 1 EE to Magento 2 EE

Before you migrate any data, you must create a config.xml configuration file from the provided sample. The sample configuration file (config.xml.dist) can be found at:

<your Magento 2 install dir>/vendor/magento/data-migration-tool/etc/<migration edition>/<ce or version>

In this file you need to enter your <crypt_key> and all the details for accessing both your source and destination databases. Where <source> is the Magento 1 instance and <destination> is the Magento 2 instance.

<source>
    <database host="db" name="magento1" user="root" password="root"/>
</source>
<destination>
    <database host="db" name="magento2" user="root" password="root"/>
</destination>
<options>
    <crypt_key>sample123456</crypt_key>
</options>

There’s several other options you can set such as database prefix and the steps you want to run. There’s three step categories: settings, data and delta. Settings contains the steps for migrating only the store settings and definitions, data is used for migrating everything and delta is used for migrating only the changes made in the Magento 1 instance since the last time the tool was executed. We are only interested in the data category and we only need to run the following steps:

• Customer Attributes Step: This step is for migrating the customer attributes
• Map Step: This step is for mapping the Magento 1 database structure to Magento 2. Most of the data is either copied or transformed in this step.
• OrderGrids Step: This step is for migrating the order grids.
• SalesIncrement Step: This step is for migrating the actual orders.

So we can delete all the other steps and groups. The config.xml file would look as follows:

<config xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xs:noNamespaceSchemaLocation="../../config.xsd">
    <steps mode="data">
        <step title="Customer Attributes Step">
            <integrity>Migration\Step\Customer\Integrity</integrity>
            <data>Migration\Step\Customer\Data</data>
            <volume>Migration\Step\Customer\Volume</volume>
        </step>
        <step title="Map Step">
            <integrity>Migration\Step\Map\Integrity</integrity>
            <data>Migration\Step\Map\Data</data>
            <volume>Migration\Step\Map\Volume</volume>
        </step>
        <step title="OrderGrids Step">
            <integrity>Migration\Step\OrderGrids\Integrity</integrity>
            <data>Migration\Step\OrderGrids\Data</data>
            <volume>Migration\Step\OrderGrids\Volume</volume>
        </step>
        <step title="SalesIncrement Step">
            <integrity>Migration\Step\SalesIncrement\Integrity</integrity>
            <data>Migration\Step\SalesIncrement\Data</data>
            <volume>Migration\Step\SalesIncrement\Volume</volume>
        </step>
    </steps>
     .
     .

Next, we need to update the mapping files. There’s quite a few mapping files, but for this example we are only interested in map.xml and map-customer.xml. Here the terminology could be misleading: Tables are known as documents in the mapping files, and columns are known as fields. In map-customer.xml we need to specify which customer attributes should be skipped or ignored. The map.xml file describes the differences between data structures of source (Magento 1) and destination (Magento 2), so we need to specify which tables (and columns) should be ignored and which tables should be renamed or transformed. The tool requires you to explicitly ignore tables or fields you are not interested in migrating. Otherwise, you would get errors such as the one below:

Destination documents are not mapped: custom_table_name

However, the easiest way to find which tables and columns we should be ignoring is by running the actual tool, since the first step is an integrity check. It’s very important to check which tables will be mapped as the information on the Magento 2 database could be overwritten. I ignored several tables that were not needed; such as core_store, core_website, core_store_group, because I didn’t want to migrate websites and stores. The product, cms, stock and category tables should also be ignored, since we do not want to migrate them. You can use wildcards(*) in the mapping files for ignoring multiple documents or fields. There were only two tables that I couldn’t skip: design_change and customer_form_attribute, so I recommend backing these tables up separately before doing the migration, and restoring them after completing the migration.

Now it’s time to run the tool and you should run it first on a development environment, and it is highly recommended to backup your Magento 2 database in case anything goes wrong. This can be done as follows:

php bin/magento setup:backup --db

Now we will cross our fingers and run the data migration tool as follows:

php bin/magento migrate:data --reset vendor/magento/data-migration-tool/etc/ce-to-ce/<version>/config.xml

Where the –reset parameter starts the migration process from the beginning which is very handy for testing purposes.

Finally, you will need to run the customer_grid reindexer manually as follows:

php bin/magento indexer:reindex customer_grid

This concludes the migration process. You should hopefully have migrated all customers and orders correctly, but it is recommended to double check all data and go through both the frontend and backend manually before doing the process on a live environment. In my particular case this worked beautifully so I hope this works fine for you too.