Preparing MDM Hub for Enterprise Adoption
In a recent blog post, David Loshin discussed the alternative approaches for providing streamlined access to master data. One of the approaches discussed in this blog is to create MDM hub architecture, a common implementation style where master data is managed centrally while applications and consumers use different channels to interact with hub. Typically, these channel interactions are in the form of web service calls, remote method invocation (RMI) and message queues (asynchronous).
Once you consolidate master data and create a single authoritative source, the requirement for your MDM hub to meet enterprise wide master data requirements starts to arise. To serve these requests better, we have to architect MDM hub in a distributed architecture where application, database and presentation tiers are separated. This provides us flexibility along with ability to scale out and load balance the system as the demand for master data increases over the period of time.
As you start your hub design, here are key considerations that should be focused on.
Application Tier Performance and Scalability
The application tier of MDM does lot of key activities such as validations, standardization, external real time calls (Ex: address parsing, verification) etc. Along with this, application tier also acts as a container for complex algorithms that match records in real time. Matching engine can be complex due to business logic and volume of data thus gulping major portion of application tier resources.
Designing this layer is key part of the equation where correct amount of CPU, memory, disk I/O and network speeds need to be considered. Additional factors include – caching, session management, threads and notifications, which can take toll on the system during peak operating hours.
Database Operation and Tuning
Database system used to persist master data is vital to overall performance of the system.
Depending on the workload requirements buffer pool sizes and indexes should be created to support faster fetching of data. In case of large database, we also need to take partitioning strategies into account. Efficient logging and trigger mechanism should be in place to replicate data to downstream applications such as Data Warehouse.
A well-designed system is a result of good balance between application and the database layer. We have to find equilibrium where application tier sending thousands of requests is supported by quick response time from persistent layer. And in the process, ensuring there is low contention and high concurrency is vital to the design.
Flexible Interface Layer
A flexible and extensible interface layer (Aka, Request-Response framework) is an essential requirement of the hub. This permits easier interactions between MDM and external applications.
Usually a standard based approach to interfacing is the best option. This allows enterprise applications to adapt to the framework easily. Ensure the interface layer supports wide variety of formats such as JSON, XML, and SOAP over HTTPS/JMS etc.
Security
Master data is a critical asset of your organization and it needs to be secured from unauthorized access.
Provide right level of access to both applications and users of the system. Role based access and security restriction such as firewall should be in place to warrant master data is safe. Use https, WS-Security during SOAP call to ensure transmission of data is done securely over the network.
Availability
Ensuring the system is made available 24/7 is one of the vital aspects of MDM hub design. Load balancing and disaster recovery requirements should be thought of at the very beginning to ensure system is accessible all the time.
These considerations will help extensively during the design of MDM hub to meet performance and scalability goals. Note that performance of a system is fading feature and as the demand for master data grows in your organization, you will have to re-visit your design on a periodic basis to make sure your MDM hub is performing optimally.
Do share your thoughts about this topic via comments. I love you hear your experience creating Master Data Hub which is central to any enterprise architecture. And if you liked this post, please tweet about it.
Image courtesy of jscreationzs/FreeDigitalPhotos.net
COMMENTS
Leave A Comment
RECENT POSTS
Composable Applications Explained: What They Are and Why They Matter
Composable applications are customized solutions created using modular services as the building blocks. Like how...
Is ChatGPT a Preview to the Future of Astounding AI Innovations?
By now, you’ve probably heard about ChatGPT. If you haven’t kept up all the latest...
How MDM Can Help Find Jobs, Provide Better Care, and Deliver Unique Shopping Experiences
Industrial data is doubling roughly every two years. In 2021, industries created, captured, copied, and...
Nice post!
Is it me or you spell checker replaced JSon with Jason?
Thank you. Interesting post.
A couple of questions.
1. When you speak of “matching engines”, are you talking about “Customer MDM”?
I ask, because for “Product MDM” solutions, requests are made with specific search criteria. Unless I’m missing something (very possible), “Product” CRUD requests are very specific.
2. Are you writing about “design and architecture” requirements or adoption?
Methinks it’s more about design and architecture. Adoption would be more about data governance, corporate politics and the occasional public flogging.
Or maybe that’s just my company…
(names withheld to protect me)