Oops! The input is malformed!
Originally published 1 July 2009
“It’s data modelling Jim, but not as we know it”
Could these be the words of chief (data) engineer Scotty on the Starship Enterprise? Enterprise data modelling that is!
The problem is that “data modelling” has, in too many companies, received a lot of bad press. Have you heard any of the following statements?
Looking back into the history of data management, we see a number of key eras.
1950 – 1970: IT was starting to enter the world of commerce; and later in this period, we saw the introduction of the first database management systems such as IMS, IDMS and TOTAL. Who can remember the DBMS that could be implemented entirely on tapes? (It was IMS HISAM if you really want to know.) The cost of disc storage at the time was exceptionally high. The concept of “database” operations came into being and the early mentions of “corporate central databases” appeared.
1970 – 1990: Data was “discovered.” Early mentions of managing data “as an asset” were seen and the concepts of data requirements analysis and data modelling were introduced.
1990 – 2000: The “enterprise” became flavour of the decade. We saw enterprise data management coordination, enterprise data integration, enterprise data stewardship and enterprise data use. An important change began to happen in this period, in that there was a dawning realisation that “technology” wasn’t the answer to many of the data issues. Data governance started to be talked about seriously.
2000 and beyond: Data quality, data modelling as a service, security and compliance, service-oriented architecture (SOA), governance (still) and alignment with the business were/are the data management challenges of this period.
And all of this has to be undertaken in these rapidly changing times when we have a “new” view of Information: Web 2.0, blogs, mashups – anyone can create data! At the same time, we have a greater dependence on “packaged” or COTS applications such as the major ERPs. Also, there’s more and more use of SOA, XML, business intelligence and less traditional “bespoke” development.
Notice I snuck in “mashups” (or web application hybrid) there? There are many powerful facilities available now that enable you to create your own mashups. Make no mistake, these are now becoming the “cottage industry” IT applications of this decade. Remember the homegrown departmental Excel macros of the ‘90s and onward that became “critical” to parts of the business? Well mashups are doing the same thing now. But just who is looking at the data definitions, standards, applicability, etc.? Certainly not the data management group – because they don’t know these things are being built in departmental silos, and anyway the “data team” is pigeonholed as being only involved in DBMS development.
So that leads us on to examine the belief that many people have (too many unfortunately) that data modelling is only for DBMS development. Why is that?
In its early days, data modelling WAS primarily aimed at DBMS development. We’ll have a look at the two main techniques in a moment.
Just to illustrate, we can look at 4 typical roles:
The enterprise data customer: This might be at director or CxO level. The accuracy of data is critical, they are reports users, and the data “products” we as data professionals produce are key to serving the needs of this level of user.
The data architect: This person knows the business and its rules. He/she manages data knowledge and defines the conceptual direction and requirements for the capture of data.
The DBA: This person is production oriented, manages data storage and the performance of databases. He also plans and manages data movement strategies and plays a major part in data architecture by working with architects to help optimise and implement their designs in databases.
The developer DBA: This role works closely with the development teams and is focused on DBMS development. They frequently move and transform data often writing scripts and ETL to accomplish this.
Data models (more accurately the metadata) were (and are) seen as the glue or the lingua franca for integrating IT roles through the DBMS development lifecycle. All of the roles depend on metadata from at least one of the other roles.
What then are the steps for developing DBMSs using models? This could be the subject of a huge paper, but I’ll try and summarise it simply here:
There are two “main” approaches to creating DBMSs from models: One is the “top down” or “to-be” approach, and the other is termed the “bottom-up” or “as-is” approach.
This approach has the great advantage that the “new” or “to-be” business and data requirements are foremost. However, it doesn’t take account of any existing systems.
We also need to break away from the “you must read my detailed data model” mentality and make the information available in a format users can readily understand. This, for example,
means that data architects need to recognize the different motivations of their users and repurpose for the audience: Don’t show a business user a data model!
Information should be updated instantaneously, and we must make it easy for users to give feedback – after all, you’ll achieve common definitions more quickly that way.
We need to recognize the real world commercial climate that we’re working in and break away from arcane academic arguments about notations methodologies and the like. If we want to have data modelling play a real part in our business, then it’s up to us to demonstrate and communicate the real benefits that are realized. Remember, data modelling isn’t a belief system – just because you “get it,” don’t assume that the next person does.
In Part 2, we’ll be discussing:
Recent articles by Chris Bradley