OK…

Once upon a time, IT departments used to appoint someone to keep tabs on what data was held by the company, what the business understood about that data and the sort of processes that happened to that data. This person, usually a grizzled Systems Analyst (or Data Architect), would ensure that there was a common understanding of all the real entities involved, such as customers, products, employees, and the processes that acted upon them, such as releases, recalls, invoices, and so on. He would maintain a document, usually called a Data Dictionary or Data Model, which could be read and understood by anyone, and provided a basis for each individual IT project. Any misunderstandings were soon ironed out, and the activity was an essential part of strategic planning.

When the cyclic downturns in IT happened, it became tempting to get rid of this chore. The Data Architect, or whatever he was called, was packed off with an ornamental mantel clock under his arm, and there seemed to be no dire consequences. It all seemed painless. The integrity, and shared understanding, of Corporate data didn't seem to be an important asset after all.

However, with mergers, acquisitions and restructurings, many enterprises, including our own, have struggled to keep a consistent culture and nomenclature. The first sign of the disease is the increasing number of anomalies that seem to slip in to the company reports. Often, the root cause of such anomalies is laughably crude, such as the case when one division of a major bank counted a customer as a person who had one or more accounts, whereas another counted a customer as an account-holder.

The problems quickly became endemic and the only solution was to perform a major business-re-engineering. This suited the hotheads, but led to several high-profile disasters. The idea of 'Data Warehousing' was an attractive panacea, as it meant that nobody had to confront the essential problem of 'data-confusion' within the business. One could fiddle with the messy data and pummel it into a logical state, from which one could gather all that vital business information and reporting. Most bought into the dream.

Many a silver-haired ex-Data Architect will have smiled as he sat in his deck-chair, reading about the vast cost and wasted effort involved in implementing a dream that couldn't live up to its promise. If data is fundamentally unsound, no manipulation can help it. As they say in Essex, 'you can't turn turds into plum pudding'.

With this company's adoption of distributed architectures and complex messaging, the opportunities for confusion have increased. Whereas, one once had just to keep a consensus on what business entities comprised at the data layer, now one has to browbeat anyone who has the wit to nail an XML message together.

So we now come to the point where what this paper is suggesting is a type of Master Data Management initiative. We once more realise the importance of understanding the 'corporate metadata', the consensus on how the corporate entities are understood and accounted, and the nature of the processes that act on them. We understand a service-based approach to data and we now have the technology to integrate this model into our IT systems. And, ironically, it is only now that we have come to realize the importance of that ancient Systems analyst poring over his obscure diagrams of boxes and arrows. His prolonged absence is the reason why this new data system will now cost so much, and take so long to implement. And we still might not get it right…