BCBS 239 is summed up by the Basel Committee on Banking Supervision as: “ the right information needs to be presented to the right people at the right time”. In the global financial crisis of 2007/8 (remember that one?), regulators found some “globally systemically important” banks did not have a firm grip on their risk reporting and could not react quickly enough. The Principles for effective risk data aggregation and risk reporting were developed to try to ensure this did not happen again.

What do the principles require?

In many ways, BCBS 239 principles require application of ‘good practice’ data governance to ensure all the right data gets to decision-makers. The principles can apply to any organisation that depends on accurate reports for decision-making (are there any that don’t?). If applied, a bank should have better, faster decision-making.

With a 3 year deadline set in 2013 to implement BCBS 239, how are things looking now?

2 steps forward, 1 step back

According to the latest progress report (April 2020), “ In general, banks require more time to ensure that the Principles are effectively implemented.” So why is it taking so long? Why are banks with (supposedly) limitless resources not succeeding - and what can be learnt from their mistakes?

What is taking most of the time?

Perhaps surprisingly, over half the resources of a typical BCBS 239 project are taken up by determining and documenting data flows as data lineage. This is a critical part of the process – after all, if a bank can’t describe what data flows where, or document all of the sources of data used in a regulatory report, a regulator will not be satisfied. Seeing data in context is essential for a full high-level view.

The worst scenario for a bank is to spend vast quantities of money on a manual data lineage exercise, only for results to be discarded because they do not link together with other metadata and have gone stale. This is ‘dead’ data lineage.

The ideal scenario is for system owners to publish updated lineage information when it changes, and for that to be merged in and change-managed with all other metadata (like data catalog, data quality metrics, etc). This is ‘live’ data lineage.

Most banks are between these two extremities, able to extract lineage and metadata programmatically in some cases, and needing to merge this with data curated by system owners and subject matter experts. It is imperative to make this process as efficient as possible.

What takes up the rest of the time? Implementing data quality controls is the next largest resource demand. To fully benefit from data quality controls, issues in data need to be reflected in quality metrics of datasets – and consumers of affected data (not just immediate – all downstream consumers) need to be aware. With many quality assessment implementations for different architectures possible, the main aim for BCBS 239 should be to capture quality metrics for datasets, visualise data quality in context, and maintain it along with other metadata.

Data dictionaries and catalogs - Data dictionaries/glossaries/catalogs – providing business meaning for data and identifying critical data elements – is next for resources. Materiality (or impact of getting it wrong) of data can be used to prioritise governance. Data glossaries play a crucial role in completeness of reporting – classifying all relevant data for a report across multiple systems. Integrated with lineage flows and quality metrics, they help to ensure all data for a report is present with correct quality.

Harsh lessons in sustainable governance

The main mistake is to have a short-term view: ramp up a big push, hire some consultants to collect data and ‘get across the line’, without thinking of the bigger picture and a sustainable process. This approach ends up with wasted effort and the need to do it all again – properly this time.

To focus on the longer term, think about what you want to be able to do when you have all this regulation-required information. You want to see quality issues along the flows for data to your regulatory reports, and anywhere else. You want to find all the creditor data in your systems and make sure it is being reported. You want to help your team find the data they need to run new initiatives. You want to do ‘what-if’ experiments to see what would change if systems move to the cloud.

There are 2 key aspects to get right:
• A sustainable process to collect up-to-date metadata
• An integrated metadata platform

A sustainable process requires federating subject matter and system owners work across the organisation, and to automate where possible. Consider how business as usual will look.

Users need to easily apply their subject matter expertise and carry out manual tasks quickly and consistently, with intelligent suggestions for lineage and relationships from machine learning.

Connectors and open APIs, and integrations through expert partners, should be used to extract and load metadata and keep it up-to-date – including properties like dataset quality metrics.

An integrated platform for metadata treats metadata as a first-class asset, with all the change control and ownership that implies. The underlying architecture needs to be simple and adaptable, without artificial constraints. It needs to include lineage, quality metrics, glossaries, high level views, search, queries and reports – seamlessly.

Without integration, organisations face a synchronisation and maintenance nightmare and inability to see the end-to-end view.

Solidatus is designed by people who have been there and done it, and see the full picture. Why not use the regulatory imperative to deliver an asset of growing value to the whole organisation, where business leaders and regulators can see the full data picture?

“It is no secret. All power is one in source and end, I think. Years and distances, stars and candles, water and wind and wizardry, the craft in a man's hand and the wisdom in a tree's root: they all arise together. My name, and yours, and the true name of the sun, or a spring of water, or an unborn child, all are syllables of the great word that is very slowly spoken by the shining of the stars. There is no other power. No other name.”
Ursula K. Le Guin, A Wizard of Earthsea

While having a conversation with a highly regarded data management expert about all things data, we started to discuss Solidatus Version Four and its integration of a Data Catalog.

Featuring an integrated, elevated and enhanced industry-leading Data Catalog

London, UK - 8 June 2020: Solidatus, the leader in metadata management, is announcing the latest update of its award-winning platform, incorporating a freshly enriched and fully integrated Data Catalog, providing improved data management functionality.

Metadata isn’t a new phenomenon; it has been used for centuries. Some of the earliest examples of it come from the Great Library of Alexandria in 280 B.C., where tags were attached to the end of scrolls. Library users could infer the scrolls’ content based on the title, subject, and author detailed on the tags. Metadata in computing first made an appearance in 1967 from MIT, where it was used to assist researchers in finding the appropriate data. Today it is critical to almost all facets of our lives. Helping us find, understand and access data quickly and efficiently through Google search, for example. Over the last 40 years, management of metadata has evolved and continues to do so at a rapid pace.

After the global financial crash of 2007/2008, regulators demanded organisations have better data and metadata management in place. BCBS239 is a key piece of legislation that reset the bar of expectations for financial services organisations. It places a specific, significant emphasis on data governance and metadata management. However, over the boom period before the bust, technology use had skyrocketed in these organisations both in terms of the number of products and their diversity causing a complexity nightmare that contributed to the severity of the crash.

Regulated organisations rushed to find and implement technology solutions to their newly mandated obligations and to provide evidence of understanding and control. Organisations like IBM, Informatica and Collibra had software products to address the new mandatory data governance. Financial services companies have spent millions of pounds purchasing and invested years implementing these extensive products. However, as this was an emerging discipline, the scope, requirements and complexity were not well understood and continued to exponentially evolve. While these products offered significant functionality there wasn’t - isn’t - a product that covers all of the aspects of metadata management, data governance and what we once considered state of the art, quickly becomes legacy in our rapidly changing technology environment.

Solidatus takes a very unique approach with organisations that adopted software early to start their data journeys and have since found their current tooling does not meet their growing needs. Rather than focusing on ripping and replacing incumbent systems, it seeks to enhance, extend and join together the myriad of products that make up an organisation’s metadata ecosystem. Forming technology partnerships with companies such as Collibra to assist their clients, Solidatus bookends Collibra enabling an accelerated onboarding journey, providing an automated link between the business and technology, and delivering understanding at scale through an intuitive and performant visualisation. The Solidatus|Collibra connector allows Solidatus to both import data from Collibra as well as export data to Collibra programmatically through our API.

Solidatus has the advantage of learning from all of the solutions that have come before as well as being able to utilise all of the latest modern technologies and development patterns to create a solution that can truly offer organisations shift left and level 5 (Optimising) on the capability maturity model. The Solidatus methodology for digitally transforming organisations to be data-centric and lineage-enabled has proven to be a game-changer with clients achieving up to a 90% reduction in operational costs for metadata management.