Businesses are flushed with the understanding that data is the new gold and are frantically scurrying to be first to strike it rich. Yet how do they do this?
During a recent meeting with a financial institution, I was asked the question “who in an organisation should care about data lineage?”. This was an interesting question to raise. We’ve spent time discussing what data lineage is and why it’s important, but we’ve not yet delved into who owns it and who really cares about it.
In this new mini blog series, “Who cares…?”, I'll examine some use cases for different stakeholders and business units within an organisation outlining how data lineage benefits them and the positive impact it has on their organisation.
Although the “Who cares?” series will focus mainly on financial institutions, the stakeholders and use cases are for the most part industry agnostic and can be applied to any regulated sector.
Who should care about data lineage?
For people who are active in the data space, several key stakeholders and users immediately come to mind, however for people who are more removed I can understand that it could be challenging to identify who should care about data lineage and what the benefits to them are.
As regulation is currently one of the main drivers for the introduction of organisation-wide data lineage, let’s start with a stakeholder with a regulatory focus.
Globally, regulators are placing more and more emphasis on data lineage. The reason for this emphasis is that the regulators have become aware that an organisation that has a detailed understanding of their systems, processes and data flows can assert a far greater level of control and provide quality data consistently and promptly. Just as in grade 8 algebra where only 50% of the marks are allocated to the answer, providing the answer alone is no longer acceptable. Regulators are now demanding that organisations show their workings to prove compliance. Whether intentional or not this regulatory focus has given the financial sector the opportunity to fundamentally change the ad-hoc, organic growth of organisational ecosystems and replace it with mature, architectural engineering grounded in best practice. Increasing regulatory burden due to the multitude of new regulations has highlighted the need for organisations to become proactive rather than reactive to regulation in order to reduce the cost of compliance. Data lineage provides the organisational understanding to make that transition.
One of the often-overlooked areas where data lineage adds significant value is within a business as usual (BAU) IT project lifecycle.
Project Based Business Analyst - System Upgrade
When a software vendor releases a new version, which requires a significant system upgrade, an organisation with data lineage will excel. In an organisation with no data lineage, the project business analysts are expected to contact all of the up and down stream systems to determine what data flows between them. Only once the data flows been have determined will the analysts be able to estimate the organisational impact of the system upgrade. With large systems having single hop connected systems numbering in the hundreds to thousands, this process can take an extended period of time, require more resources and be costly. With the growing number of systems being installed, upgraded and retired the negative organisational impact is only going to grow. With up-to-date, distinct attribute level data lineage across all systems the analysis phase of all projects is vastly reduced, saving time, money and de-risking projects. Additional phases of development, testing, roll-out and support are all positively impacted, saving organisations millions annually. Operational efficiencies that lead to less time focusing on support and maintenance of current IT infrastructure means more time focusing on driving new revenue and new markets.
So, in conclusion, for the new “Data Rush” to yield the promised returns, just as in the original “Gold Rush” you first need to find a seam, understand everything about the seam and its flow before you are able to extract every valuable ounce.
Keep an eye out for our next blog in this series due next week.