Part 1: Reference Data Management – An Overview
Lately, I have been working with few clients who’ve been implementing reference data management (RDM) systems. I will be writing a series of blogs sharing my experience working with reference data including challenges involved while implementing RDM solutions, architectural best practices and reference data integration considerations.
This post covers some of the basics.
What is Reference Data?
Reference Data as we know is the data used to categorize other data within applications and data bases. We usually refer to this data as look-up, code table or domain values. These code tables are usually characterized by code and value pairs. Think of it as set of allowed values for a given field.
Some of the best examples are – state codes, country codes, gender codes, marital status type codes etc.
NAICS codes are great example of industry standard reference data sets used for classifying business establishments. For example, according to Federal Statistical Agency in North America, NAICS code for Pizza delivery shops is 722513. Similarly, every business has been given a code and this standardized coding helps in collecting, analyzing and publishing statistical data related to US economy.
Why is it important to manage reference data?
Typically, each application in an enterprise has its own representation of code sets defining the same thing. During integration of master data (or any data) across applications, it’s necessary to translate between the different code table representations in order to categorize data in a consistent way.
Mapping between the different representations and keeping track of changes across all the different code table variations on an ongoing basis can be a major challenge. Many enterprises struggle with this challenge and often use error-prone manual processes to record and manage changes to reference data sets.
Errors in the reference data can have a major business impact. Quality issues in reference data have ripple effect and can cause major issues in the downstream applications. Integrity of the reports you generate are directly proportional to how good is your quality of reference data. More than everything, bad quality of reference data is the most common source of system integration failure.
Key features of Reference Data Management Systems
Earlier I wrote about key functionalities of a Master Data Management System. Here are some of the key features of a Reference Data Management system –
- Robust interface controlled by role based security to support collaborated authoring of reference data.
- Ability to manage and map relationships between different reference data sets which exist in an enterprise.
- Versioning and auditing capability
- Hierarchy management
- Provision reference data via web services to support SOA framework
- Ability to publish reference data
- Efficient load/extract functionality
- Good error tolerant search capability
In next post I will discuss the challenges organizations face with reference data, and how RDM system can help them resolve these challenges.
Please share your views and experience working with Reference Data via comments. What are the driving forces for your RDM implementation?
COMMENTS
Leave A Comment
RECENT POSTS
Composable Applications Explained: What They Are and Why They Matter
Composable applications are customized solutions created using modular services as the building blocks. Like how...
Is ChatGPT a Preview to the Future of Astounding AI Innovations?
By now, you’ve probably heard about ChatGPT. If you haven’t kept up all the latest...
How MDM Can Help Find Jobs, Provide Better Care, and Deliver Unique Shopping Experiences
Industrial data is doubling roughly every two years. In 2021, industries created, captured, copied, and...
Good 101 on RDM Prash.
A few additions:
I often say that good search functionality is the forgotten MDM capability. Even for the typically smaller data sets related to RDM a good error tolerant search can help.
A then there is big reference data. Examples are deep address directories telling about every valid address in a country, business directories like the D&B WorldBase with over 200 million business entities from all over the world. Often you might not host these big reference data on premise but use cloud services.
Agreed there is so much more to say and do about reference data management and this is often an in invested area (well, there are all the hidden costs of not dealin with it)
Thoughts on how this topic is covered in DAMA’s DMBOK? We are working towards the next edition.
Mehmet, great to hear MRDM is going into DMBOK. Please post updates or requests if you can use any support on the (Master) Reference Data Management LinkedIn group:
http://www.linkedin.com/groups/Master-Reference-Data-Management-1637247?trk=myg_ugrp_ovr
Prashant, thanks for your post of the relevant article on the MRDM group. Keep up the good work!
Great to see this topic covered, I’ve seen so many differences of opinion on RDM, I think we need more posts like this to get some solid ideas down.
To All,
What is your experience with RDM in context to MDM? By historic definition, isn’t Reference Data itself a type of Master Data in that it is the single source for a system or set of systems? Why add complexity and call it “MRDM”? What is the win to business and process customers? By the nature of “mastering” this type of data…it is usually static, whereas mastering data often implies working across context and barriers to harmonize. Over the years, I have had to implement many RDM approaches and systems, always balancing Enterprise Reference Data is relationship to local system Reference Data needs.
[…] Part 1 of the Reference Data Management (RDM) series, I gave an overview of Reference Data and discussed some of the key features of a RDM System. Part 2 looked into the challenges […]
Hi Prashant,
In our project there is requirement to integrate RDM with BI product and extract hierarchies. Can we connect to RDM and extract hierarchies.
Thank you
Parag K
[…] ・個人取引先 ・取引先責任者-to-複数取引先 ・商品の管理 ・Reference Data Management Key Concepts ・選択リストからの州と国の選択の許可 ・【Trailhead】Lightning […]
[…] Read the document – Data management key concepts. […]