One relatively frequent pattern for using data matching is what I call a “contribution model”, which mixes “App Data Consolidation” and “User Collaboration”. This pattern applies when you want data coming from applications to merge in a consolidation hub style (initial/batch data loads). This data needs to combine with information directly authored in a centralized hub style in MDM workflows and UI. This requires data matching and merging at record and field level with complex survivorship rules (field-level, mostly).

FX,

Thank you for your comment. Very appropriate observation and I do see this often in implementation phases where consolidation continues to occur while master data management directly servers the pilot applications which pioneer to leverage the hub.

Coming to merging the record with complex field level survivorship rules, its a big topic in itself. There are only few vendors who do a great job at this by providing a trust framework which is flexible and allows customers to use it in a way which best suits there specific need. I will write more on this in coming months.

Thank You
-Prash

There’s another issue with data matching and linking in particular, when do you know you’re done? If you have 90M records that came from 4 or 5 different sources and you’re trying to link and/or dedup them, how often do you go through the process? Especially considering the cost, i.e. if you process in batches of 6M, it takes about 3 days to process.

I’d be interested if you have any rules-of-thumb or if you’ve done any analysis about how to make that decision.

Good article. De-duplication itself is a project at many places and never ends. Please do write more about trust framework and how successful these frameworks are.

Thanks

This is always one of the challenges of mergers and acquisitions and developing an ecosystem of companies where connecting disparate yet potentially duplicate data sources is critical. The sensitivity of the data need be taken into consideration. If for example you have a high probability that you have duplicate records, relationships and volume discounts for the same provider than by all mean try to renegotiate one (larger) volume discount. Part of the business case for MDM. However with customers it is different. For example when one bank recognized in their merged CIF identical names, dates of birth, and SSN numbers but different addresses for seven accounts it seemed sensible that the one outlier address was outdated and a combed CIF and statement made sense to demonstrate the importance to the bank of the overall relationship with the customer. . . Unfortunately the one outlier was a charge card for the mistress and combining its statement with the wife’s statement ended up in a legal quagmire and a lost customer relationship. With duplicate customer data it is often smart to incentivize the customer to connect the disparate dots.

One method of MDM implementation you touched was Searching, Cloud MDM tools give us the perfect way to search for duplicates at the point of entry.

It basically recognizes our Matching settings for the batch job to identify the duplicate and puts that logic into Search of records and gives potential duplicate at the Point of Entry. One more important thing to consider here is we can actually toggle between the matching settings we would use for searching and the one which we would use for finding duplicates in the system to merge. Looking forward for more cloud based MDM implementation information.

We recommend that a full batch process be run annually due to changes in address coding guides. For example, a zipcode change may cause a split if the old zip resides in the data warehouse and the new zip appears on an incoming record. Many times, we also recommend an annual NCOA process to run concurrent to this to avoid a similar problem. An old address in the database that will not match to a new address for the same person on an incoming feed. Of course if the source has a customer number, then the data can be changed but in most instances, there are external feeds that will not.

Good post. There is so much great work being done with data matching tools in various industries such as financial services and health care, especially with data-driven decision making becoming a bigger trend. It will be interesting to see the impact of these changes down the road.

Linda Boudreau
Data Ladder

[…] Data Management tools on the market, you will find that most of them provide functionality for matching records from various source systems and blending or merging them into a single, trusted data set, a practice also dubbed “Golden […]

Really useful inputs. Thanks for keeping Data matching scenarios together.

Hey,
Sir i am a beginner in MDM. I am using Informatica Hub.
I have gone through various documents and videos and now i have a good idea of what is the purpose is.
I have performed the following process,
Created Landing Tables, Base Objects, Staging Tables, Mapping , Cleanse functions ,
stage and load process.
Now i want to do match merge and tokenize, which im not able to understand that how it is done, I am trying by and am not able to do so as i dont have that much knowledge of Informatica hub or how these process work.
Any guidance or help would be appreciated.
Thanks

Hi,
We are using MDM in our bank project, it’s totally outdated and inappropriate framework, even for matching functions…
All team members only wish to get out of MDM now!