4 minute read 6 Dec 2019
Aerial view oil refinery storage tanks illuminated at night Los Angeles USA

Why data migration is about risk mitigation not technology

4 minute read 6 Dec 2019

Show resources

Data migrations are fundamental to any business transformation that involves technology change and enterprises must not underestimate their complexity.

When a technology upgrade is underway, data will likely need to be moved from one system to another. Or, in circumstances where an organization has acquired, or is merging with another organization, multiple systems will need to be consolidated. The technology systems are vital for success, but without the right information flowing through them, it is difficult to meet the needs of customers or citizens, executive management and other key stakeholders.

Enterprise teams must shift their thinking about data migration to be truly successful, taking it from a technology project, to a business-critical risk mitigation program.

Why the problem is always bigger than expected

In its simplest form, data migration is about taking data from a legacy system and putting into a new one. It is essential to extract the data, determine what can be migrated, what’s missing, what needs to be cleansed and what transformation is required to fit the new environment.

Sounds like a simple technology project, doesn’t it?

Where things get complex is at the enterprise level, with legacy systems that have grown organically with questionable data quality. Coupled with that, is often a constantly evolving new system which needs to respond to the changing needs of the organization even as the project runs. There are likely to be millions of data points across these systems and every single one needs to be assessed to determine which are required for operational, regulatory or analytical needs. They must then be migrated to the new system without disrupting business as usual.

Things are no longer quite so simple.

On one hand there is disparate, messy and incomplete data and on the other is a target state that is constantly changing.

How can enterprises successfully undertake a data migration?

1. Think of it as a business-critical risk mitigation project, not a technical one

It is vital to understand the data migration through a business lens such as regulatory impacts and what is needed for business-as-usual operations. Engage the business and get them involved as sponsors and project team members. Make sure the business understands how critical their input is to achieve the outcomes required. For example, customers must not feel the change. Ensuring their balances are correctly transferred, enabling regular direct debits to continue unhindered, and giving the customer service team the right data to quickly and accurately resolve customer issues are all business-critical. And, knowing how customers behave and using that information to determine common customer requests so that the right information can improve engagement and loyalty.

2. Understand the scale of the problem

The word “problem” is used deliberately here; we’ve never seen an enterprise-level migration that didn’t have some (ok, many!). Time and again clients don’t see the true depth of the challenge. Instead of data being a stand-alone line item in the transformation program and budget – we use 20% as a rule of thumb – it is lumped in with technology and whatever is left over after the system is integrated is given to the migration.

3. Understand the risk of impact to business as usual

Successful data migrations occur when organizations understand how to - and are in a position to - move data from legacy systems to the new system while minimizing the risk to business as usual. Risk, whether regulatory, brand- or cost-related, can flow through an organization if the data is incorrect. Operational risk is also an issue such as being unable to answer customer questions or deliver the right information. Typically, the most challenging, and therefore riskiest, part of a migration is the data mapping; understanding how each element in the legacy systems transfers into the new system.

We have seen situations where data migrations have not accurately reflected a customer’s history in the new system, making it impossible to service them correctly.

4. Actively manage how migration fits into the transformation program

Migration activities need to work in collaboration with the rest of a transformation program so that data migration supports it rather than being on the critical path. It is also important to get the timing right; consider the migration at the beginning of the transformation journey as transitioning to the new system is in danger of delay if the migration isn’t ready to go. Start the migration while the target system is being built and build in full auditability of the data to ensure absolute accuracy in the new systems.

5. De-risk the technical aspects

Establishing robust reconciliation processes throughout the migration will help mitigate technical risk. A continuous integration approach with automated reconciliation processes running every night throughout the project will identify problems literally as they occur. As you build more of the migration routines each day, test the entire routine overnight, including the new elements, to identify negative impact on the output. This helps avoid breaking one part of your migration while fixing another.

6. Consider using AI as part of the migration

Although not applied widely yet for data migration, there are still a number of real opportunities for streamlining and improving with artificial intelligence (AI). AI can help determine the business rules associated with data to address data quality and matching issues. For example, merging records and removing duplications, both currently manual activities, could be handled by AI to save time and effort, and to reduce cost. When there is a significant amount of information captured in ‘free text’ fields, AI techniques can be used to turn this ‘unstructured’ information into structured data to meet the governance and quality standards of the new environment.

Data migration is detailed, challenging work and fraught with business-crippling outcomes if not done successfully.

If the environment is complex, as enterprises tend to be, where there are multiple source systems, a constantly changing target system and data quality issues, it is vital to take a business-critical, risk mitigation approach to the project.

There is serious upside if the migration is done well. Not only will the business have new technology systems, it will have the right information available when required to meet and exceed the needs of customers and other key stakeholders.


Data migrations are fundamental to any business transformation that involves technology change – which is the vast majority. Enterprises encounter issues when they underestimate the scale of a data migration project, lumping it with technology, rather than recognizing it as a “business-critical risk mitigation program.”

About this article