DATA QUALITY IMPERATIVES FOR DATA MIGRATION INITIATIVES :
A GUIDE FOR DATA PRACTITIONERS
- Anastasia Berzhanin
-
Published: 18 October 2019
In a data migration project, risk usually surfaces very late in the form of target system (load) failures, which are often the result of poor data quality and poor understanding of the data. Most data migration projects rely on documentation about the current state of data landscape or conversations with source data owners and business experts. This approach is insufficient, as data transformations based on assumptions that may only on average be valid will result in data issues and load failures even with minimal deviation from these assumptions.
An early profiling exercise helps gain a thorough understanding of data nuances and prevents system failures later on.
We have compiled a best practice guide for data profiling, reflecting on pitfalls and meaningful discoveries from a recent major data migration project and covering the methods that worked after testing the ones that failed.
