How many times have you started a new reporting project with no documentation about the data structures involved? How many projects has your organization implemented that are not documented? Have you ever spent hours trying to decipher the logic used to create a report or load a data warehouse table? If you’ve worked at more than one organization or spent more than a couple years in information management, I’m sure you’ve seen this problem head on, maybe pounding your head on the wall.
These problems seem to have an obvious solution: just document the requirements and keep them up to date. But we always find it so difficult to accomplish this task. So often it is difficult to make the time in a project to properly document requirements, or to update them to the final result of the project, or to incorporate change requests made after project completion. But if this work is not done and the results are not stored in a central repository, we soon find out the repercussions of our failure. As soon as a significant project comes along, all the analysis must be completed again. If you’ve ever changed employers and started with an organization with poor data governance practices, you know exactly what this looks like.
With my current client, I’m working with a health plan to update business requirements for a new implementation effort. This client has made an admirable effort over the past decade to document, manage and update business requirements for all their data management projects. They do this using an application, CDMA by Knightsbridge/HP, built to collect and document source to target mappings, metadata, and business requirements. Each time a new project is created within this payer organization, business data analysts utilize the application to add new business requirements for each application, data element and data migration path. These requirements are leveraged by designers and developers to create new information management code to load the operational data store, data warehouse and reporting databases. All of this information is also available to business users via an online metadata repository providing analysts, report writers and other users with the ability to look up each data element, track data within reporting back to source systems and investigate data structures at any point in time.
There are a number of ways to approach containment and recording of business requirements, metadata and master data. My current client purchased an application which provided a great backbone for this effort. Tools like IBM Infosphere MDM or Informatica MDM platforms are options. You can approach the problem that way, bring in experts to help develop an approach specifically fitted for your organization, or put together an internal group to approach the problem. Check out some of Ajilitee’s past blog postings like DG and MDM: Two Sides of the Same Coin for ideas on how to get started. But ignoring the issue of managing ongoing change will only exacerbate the problem of lost requirements as time goes on. Ongoing focus on collecting and managing metadata, requirements and mappings can reduce costs for future projects.