Business Intelligence Network business intelligence resources

Blog: Barry Devlin

« July 2008 | Main | September 2008 »

Sunday, 24 August 2008

Instant Gratification vs. Quality Time

I was browsing through the blogs on B-eye-network.com this morning (Sunday - yeah, sad, I know) and came across two recent entries that spoiled my coffee. Given that I'm no fan of instant gratification (in IT anyway), I'm not going to give you links, so you have to work at finding them yourself. But the phrases that caught my eye were "Instant SOA", "Data marts in about an Hour" and "full EDW's with AS-IS star schemas in 2 weeks".

Now I'm as fond of a shortcut as the next guy, but I've learned the the word "Instant" is not all goodness. When I've bought some instant Spaghetti Bolognese in the local supermarket I've found that the cost is a lot higher than the individual ingredients and the taste, well, leaves a lot to be desired. Sure, I saved some time when I got home, but did I get value for money? And did I end up with what I really wanted? So, why should I expect more from an Instant DW?

"Caveat emptor" as the Romans used to say. Here are a few contra-indications for when instant gratification should not be expected in your next BI (or SOA) project:

  1. The business users are not quite sure what they want.
    Most BI projects start with a vague set of requirements from the potential users. It's going to take some time to hone these down to a usable definition of data and query needs. In the meantime, maybe it's best to let the users continue to play with their instant Excel spreadsheets and look over their shoulders to see what they're doing.
  2. Somebody forgot to document the meanings of the data in the source applications.
    This is the oldest metadata problem. If your data sources have not been properly described, an Instant DW is likely to be instantly dismissed as misleading and inaccurate. Do you want to go there?
  3. Garbage in, garbage out. Or worse...
    If your ingredients (data sources) are contaminated with erroneous data, you're going to end up with a very sick business on your hands if you just take the Instant DW approach. Understanding and fixing dirty data is time-consuming, but mandatory.

It's all about quality time... or quality vs. time. If I bring home my instant Spaghetti Bolognese, I may get it on the table within a few minutes. But, if the kids won't eat it or, worse, throw up that night, I'd argue I've made the wrong trade-off between time and quality. You need to consider the same balance in a BI or SOA project.

Now, I'm off to spend some quality time with my kids :-)

Posted by Barry Devlin at 12:20 PM |

Thursday, 14 August 2008

Access to quality external data

I was at the Business Object Summit this week in Boston, where the main emphasis was on linking strategy to execution and a seeming focus on the larger enterprises. All very SAP-inspired, I thought. And very insightful, especially if you're a large enterprise. There have been some comments in the blogs already on these topics. But it was a small conversation over lunch that caught my interest...

Information OnDemand. No, not the annual IBM Conference in Las Vegas, in October. But a rather low key effort from Business Objects with a website to allow companies to access market data and incorporate it into their BI efforts.

There's a definite growing interest these days in combining external data with the contents of the warehouse. But it does raise some concerns, not least about the reliability of the external data and how to create a valid semantic relationship between the two data sets. In the past, companies have addressed these concerns by obtaining key market and other external data from trusted sources like Dunn and Bradstreet, Reuters and others and then ensuring that such data entered the warehouse via a controlled feed designed by Information Architects who could match the two data sets correctly. After all, such external data is another information source for the warehouse and should be managed like any other.

This method works well for large enterprises with a centrally-controlled approach to the warehouse. And where the value-add derived from or risks incurred by using this data are significant, this method is probably still required. But what if you are a small or medium enterprise? Or what if you really only want to do a couple of once off analyses?

Shopping at the Information OnDemand website appears to be the answer! Here you can buy prebuilt, but customizable, reports combining your data with external market and financial data. You can buy one-time snapshots or subscribe for regular updates.

For larger companies, this could provide a safe and cost-effective way of dipping their toe in the big ocean of external data. For smaller companies, it could be all they need. Sounds like a useful idea to me!

The service has been available since September 2007, but I hadn't come across it before. Maybe there are some similar services I should know about, so please feel free to comment.

Posted by Barry Devlin at 5:14 PM |

Thursday, 7 August 2008

Enterprise search, Web 2.0 and BI

I came across an ad today for a Google Webcast on Universal Search for Business. It contained the phrase "As the volume of information inside enterprises explodes, most executives recognize the importance of a Google-like search solution for business content.", which set me wondering...

A Google-like search solution for business content? What exactly does that mean?

The phrase "Google-like search", of course, covers a multitude of marketing-speak, but let's assume that it includes the patented PageRank technology behind Google's Internet search success. Google itself describes PageRank as follows: "PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value." (http://en.wikipedia.org/w/index.php?title=PageRank&oldid=230400158 as of Aug. 7, 2008). A number of questions arise for me: Does an enterprise intranet usually have a vast link structure? Would business executives really consider the "democratic vote" of the organization as a valid indicator of a document's importance? Indeed, how democratic is the link structure in an intranet?

Google, Wikipedia and many Web 2.0 systems have an underlying belief in James Surowiecki's concept of "the wisdom of crowds". Data warehousing, Business Intelligence and, indeed, all traditional IT development tend to put more faith in experts and their accumulated knowledge. In the BI world, I'm beginning to see some level of acceptance that the so-called experts do not have a monopoly on business knowledge. We see that there is a growing need to allow and, indeed, facilitate the feedback of knowledge that emerges on the fringes of the BI community (the front-line staff and first-line managers) back into the core of the warehouse for wider promulgation and reuse.

But, to what extent does Google and the Web 2.0 community recognize that some knowledge is inherently more useful or valuable (although not necessarily "right") simply based on the authority of its source? And within the tighter and more closed confines of an enterprise, that not all the requirements for wise crowds are met? If not, we may see the many years of careful effort by data modelers and administrators, and information stewards overturned in the rush to Web 2.0. This would not be in anybody's interest.

On the other hand, if I've made the wrong assumption about what "Google-like search" means... Anybody care to comment? Or maybe I'll find time to sign up for the webinar!

Posted by Barry Devlin at 8:17 PM |