Blog: Richard Hackathorn

« Entrepreneur Spirit is Alive and Well in Boulder | Main | BI and your Corporate Fantasy League »

Business by the Data – NOT!

As professionals in Business Intelligence and Data Warehousing, we live and die by the creed of Business by the Data. Deep inside, we know that, if we do our job properly, our companies will be managed better and will perform successfully because with better data comes better decisions. Right?

It is funny! After many decades of struggle, we are entering an era of BI/DW where that creed will be proven correct or exposed as a cruel lie. The evolving practice and technology of BI/DW are awesome! I never cease to be amazed at what innovative companies are doing and what innovative vendors are offering.

Today there was a WSJ article (front left column of Marketplace) that caused me to think deeper on this issue.

The article rambles but makes a useful point. “Managers can be so focused on perfecting today’s business that they lose sight of tomorrow’s” – that was a quote from a book by Sutton & Pfeffer entitled “Hard Facts, Dangerous Half-Truths & Total Nonsense”.

The point is that the data used in management discussions to make decisions is often filled with misleading and even false assumptions. In particular, we focus too much on the past and (with real-time BI) the present, without thinking deeply and clearly about how our business will change next month and especially this week. Those constant changes are so fundamental to the nature of business. Data from our BI systems often clouds our better judgments, which a hundred years ago would have been labeled common sense.

This echoes an old problem. Whatever we would print a report on the line printer, the data often seemed inconsistent or even mysterious, until we carefully analyzed the application programs that maintained the data. There buried in some COBOL clause was a hidden crucial assumption about how the data was to be used, created on a whim by a programmer meeting a deadline. Thus, the quest to capture meta-data merged.

The same is now true of analytics. Buried in the calculations are crucial assumptions that permeate our dashboards. Guess what? Those assumptions are deeply hidden, may or may not be valid, and affect business performance in a big way.

Therefore, Business by the Data – yes but be careful. It would be better to say Business by Insights, which can be stimulated by data from BI systems. Always ask why when viewing data. Clear thinking will never be replaced by sophisticated analytics.

Posted by Richard Hackathorn on Monday, 23 July 2007 | Permalink