|
I was browsing through the blogs on B-eye-network.com this morning (Sunday - yeah, sad, I know) and came across two recent entries that spoiled my coffee. Given that I'm no fan of instant gratification (in IT anyway), I'm not going to give you links, so you have to work at finding them yourself. But the phrases that caught my eye were "Instant SOA", "Data marts in about an Hour" and "full EDW's with AS-IS star schemas in 2 weeks".
Now I'm as fond of a shortcut as the next guy, but I've learned the the word "Instant" is not all goodness. When I've bought some instant Spaghetti Bolognese in the local supermarket I've found that the cost is a lot higher than the individual ingredients and the taste, well, leaves a lot to be desired. Sure, I saved some time when I got home, but did I get value for money? And did I end up with what I really wanted? So, why should I expect more from an Instant DW?
"Caveat emptor" as the Romans used to say. Here are a few contra-indications for when instant gratification should not be expected in your next BI (or SOA) project:
- The business users are not quite sure what they want.
Most BI projects start with a vague set of requirements from the potential users. It's going to take some time to hone these down to a usable definition of data and query needs. In the meantime, maybe it's best to let the users continue to play with their instant Excel spreadsheets and look over their shoulders to see what they're doing.
- Somebody forgot to document the meanings of the data in the source applications.
This is the oldest metadata problem. If your data sources have not been properly described, an Instant DW is likely to be instantly dismissed as misleading and inaccurate. Do you want to go there?
- Garbage in, garbage out. Or worse...
If your ingredients (data sources) are contaminated with erroneous data, you're going to end up with a very sick business on your hands if you just take the Instant DW approach. Understanding and fixing dirty data is time-consuming, but mandatory.
It's all about quality time... or quality vs. time. If I bring home my instant Spaghetti Bolognese, I may get it on the table within a few minutes. But, if the kids won't eat it or, worse, throw up that night, I'd argue I've made the wrong trade-off between time and quality. You need to consider the same balance in a BI or SOA project.
Now, I'm off to spend some quality time with my kids :-)
I was at the Business Object Summit this week in Boston, where the main emphasis was on linking strategy to execution and a seeming focus on the larger enterprises. All very SAP-inspired, I thought. And very insightful, especially if you're a large enterprise. There have been some comments in the blogs already on these topics. But it was a small conversation over lunch that caught my interest...
Information OnDemand. No, not the annual IBM Conference in Las Vegas, in October. But a rather low key effort from Business Objects with a website to allow companies to access market data and incorporate it into their BI efforts.
There's a definite growing interest these days in combining external data with the contents of the warehouse. But it does raise some concerns, not least about the reliability of the external data and how to create a valid semantic relationship between the two data sets. In the past, companies have addressed these concerns by obtaining key market and other external data from trusted sources like Dunn and Bradstreet, Reuters and others and then ensuring that such data entered the warehouse via a controlled feed designed by Information Architects who could match the two data sets correctly. After all, such external data is another information source for the warehouse and should be managed like any other.
This method works well for large enterprises with a centrally-controlled approach to the warehouse. And where the value-add derived from or risks incurred by using this data are significant, this method is probably still required. But what if you are a small or medium enterprise? Or what if you really only want to do a couple of once off analyses?
Shopping at the Information OnDemand website appears to be the answer! Here you can buy prebuilt, but customizable, reports combining your data with external market and financial data. You can buy one-time snapshots or subscribe for regular updates.
For larger companies, this could provide a safe and cost-effective way of dipping their toe in the big ocean of external data. For smaller companies, it could be all they need. Sounds like a useful idea to me!
The service has been available since September 2007, but I hadn't come across it before. Maybe there are some similar services I should know about, so please feel free to comment.
I came across an ad today for a Google Webcast on Universal Search for Business. It contained the phrase "As the volume of information inside enterprises explodes, most executives recognize the importance of a Google-like search solution for business content.", which set me wondering...
A Google-like search solution for business content? What exactly does that mean?
The phrase "Google-like search", of course, covers a multitude of marketing-speak, but let's assume that it includes the patented PageRank technology behind Google's Internet search success. Google itself describes PageRank as follows: "PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value." (http://en.wikipedia.org/w/index.php?title=PageRank&oldid=230400158 as of Aug. 7, 2008). A number of questions arise for me: Does an enterprise intranet usually have a vast link structure? Would business executives really consider the "democratic vote" of the organization as a valid indicator of a document's importance? Indeed, how democratic is the link structure in an intranet?
Google, Wikipedia and many Web 2.0 systems have an underlying belief in James Surowiecki's concept of "the wisdom of crowds". Data warehousing, Business Intelligence and, indeed, all traditional IT development tend to put more faith in experts and their accumulated knowledge. In the BI world, I'm beginning to see some level of acceptance that the so-called experts do not have a monopoly on business knowledge. We see that there is a growing need to allow and, indeed, facilitate the feedback of knowledge that emerges on the fringes of the BI community (the front-line staff and first-line managers) back into the core of the warehouse for wider promulgation and reuse.
But, to what extent does Google and the Web 2.0 community recognize that some knowledge is inherently more useful or valuable (although not necessarily "right") simply based on the authority of its source? And within the tighter and more closed confines of an enterprise, that not all the requirements for wise crowds are met? If not, we may see the many years of careful effort by data modelers and administrators, and information stewards overturned in the rush to Web 2.0. This would not be in anybody's interest.
On the other hand, if I've made the wrong assumption about what "Google-like search" means... Anybody care to comment? Or maybe I'll find time to sign up for the webinar!
I've been meaning to resurrect this blog for some time now, but, hey! life gets in the way. But, the recent coverage of Microsoft's acquisition of DatAllegro proved to be the trigger to get me going, though. It's all about feeds and speeds, bigger volumes, faster access and cheaper warehouses. Debates about how this will help Microsoft move up in the market and how it will impact the other vendors.
That's all very well and good, but, excuse me, have I missed something? Since when did data marts become data warehouses? I know that the appliance vendors tend to label themselves as data warehouse appliances, but I thought we all knew that was marketing. Of course, any appliance will be part of a data warehouse system in the broader sense. But when you look at the features and strengths that appliances have, you can see that they are really data marts. Data "hypermarts" perhaps, but marts nonetheless.
By definition, a data mart is a subset of the data in the enterprise data warehouse that has been optimized for use by a particular set of users. Such optimization includes selecting the data needed for some set of business purposes and structuring it to allow the fastest, most appropriate query access for users. It's all about how you get the data out! Sounds to me like exactly what the appliance vendors emphasize.
On the other hand, the data warehouse focus is on getting the data in. How to cleanse and reconcile the diverse data. How to ensure the cross-source timing is right. How to create a model that reflects the needs of the wider enterprise. And finally to make the consolidated view of the business available to the users - usually through data marts.
So, does the Microsoft acquisition disrupt the entire data warehouse market, sending the large players into a spin? I doubt it. Building a real data warehouse will continue to be as challenging as ever, requiring the same strong integration and project management skills as before as well as the deep database integration and manipulation technology that only the big relational databases possess as of now.
I’m on the plane back from the Global Innovation Outlook Deep Dive on Media and Content in Helsinki, and here, as promised, is a quick view on the really fascinating discussions that took place.
First, though, a quick gripe at the airlines. First, they failed to transfer my bag from Dublin through Heathrow onto my flight – arriving in Helsinki at 17:45 and following three more flights from London over the next few hours – finally delivered it to my hotel at 2am. Then they added insult to injury by telling me that the system couldn’t accept a request to transfer my bag through Lhr on the way back (until they got exceptional management permission!) because Aer Lingus had left the Oneworld Alliance three weeks ago. Well done to BAA, British Airways, Finnair and Aer Lingus for making the trip such a pleasant experience!!!
Fortunately, the meeting was much more enjoyable.
You can read the official blog elsewhere, so what follows are my eye-openers.
“Democratisation of content” is changing the media business irrevocably. We may tend to think (as I did) of this phrase as meaning that Joe or Jill Bloggs can create brand new content such as text, images, audio or video far more cheaply than professionals can, and sometimes the quality can even be acceptable. But, in many cases, it’s not brand new content; it’s often a mash-up of content from existing material. This is a much more common form of creativity that has existed since time immemorial and can be seen in art forms such as collage. Calling this “pirating” just confuses the issue; the social acceptance of this behaviour and the ease with which it can now be done is a fact of the marketplace that content providers may just have to get used to. But what I think it means is that the sale of digital content “rights” (or “restrictions” as one of our participants phrased it) is fast becoming an untenable business model. The people who seem to be hurting most are the publishers, record companies and production / distribution organisations that have most profited from this model in the past; and, unsurprisingly, they are the most resistant to change.
One thought we had was that the emphasis in the Media industry needs to switch from the subset of users who are unwilling to pay for anything (estimated to be less than 20% of the market) to those who understand that there is no such thing as a free lunch, and are willing to pay a fair price for content and who often want to see that payment go in a larger share to the actual creative people who built the content. Unfortunately, at present there are not too many real and legal mechanisms to do this.
At the artist’s end of the spectrum, the internet is creating a new opportunity for collaborative working that may further diminish the role of the publishers and distributors. Artists can create collaborative communities on the Web where they can meet people with the complementary skills they need to turn an idea into a marketable product. In the music industry, that might include a producer, a sound engineer, a graphic designer, a marketer and a few more. Collaborative tooling and a place on the web to meet and work is probably all they need to get it all together. In this case, what’s the role of the record company in the future?
It certainly looks as if the traditional publishers and producers are in danger of being dis-intermediated.
The second topic that really caught my attention was a “digital identity” or “digital me” on the Internet. The technologists tend to see this identity as a cool way to enable a more free-flowing experience in the online world, where appropriate content in context (“the information you need and only the information you need”) could be channelled directly to you. Privacy advocates see significant dangers in such a consolidated digital identity, greatly increasing the risk of abuse of power, and so on.
While some of our debate centred on the question of “should we have a digital identity”, it struck me that we are acquiring a digital identity anyway by stealth – courtesy of data warehousing / business intelligence – so, the real question we need to ask is: “how can we, the consumers, take back control of our online identities?” What we need, therefore, is technology that makes it easier to create and manage our digital identities, as well a trustworthy institution or set of institutions who would hold these identities with maximum privacy and security.
The bottom line in all the topics – a new business model for content, communities of collaborating artists and digital identity – boils down to a single word: TRUST. Interestingly, trust has been the basis on which the open software industry has grown. It’s at the core of e-Bay’s business. And it looks like it’s going to be key to Media and Digital Content in the future.
I'll have the privelege of being part of IBM's Global Innovation Outlook (GIO) 3.0 project this year when I get to attend the deep dive on Nedia and Content in Helsinki in the middle of this month. I'll report back after that, but as I prepare for the session, I've been doing a little bit of reading about the topic, and wanted to share with you my amazement at the changes that are going on in that business. There's an excellent point-of-view from the IBM Institute of Business Value - "Navigating the Media Divide" that's well worth a read.
The increasing role of Joe or Jane Citizen in producing content is proving to be very disruptive to the traditional media businesses, and reflects a deep change in the relationship between "consumers" and "producers" in today's society. Who would have thought only a few years ago that people would choose to view personally-produced videos over professional ones. Yet that's what appears to be happening with the growing populatity of YouTube. The same shift can be seen in Wikipedia - an encyclopedia written by non-professionals, in Amazon with every Joe Bloggs writing book reviews; not to mention the explosion of blogging itself.
I'm only posing the question here, because I don't have the answer, but how is this power shift going to play out in other business areas, in govenrment and in education? What are the implications for the way software develops? And what's the knock-on effect on business intelligence when everybody wants to have their say on the business trends?
I'd say we're in for a sea-change. Hold tight to the rails. And follow the blog on the GIO website above to see how the thinking is developing.
It used to be the case that version 2.0 software delivered the function that was originally promised for the product and you had to wait until at least 3.1 before it performed. Now that Web 2.0 is all the rage, it seems that “2.0” can be applied to just about anything. So it wasn’t much of a surprise to see DW 2.0™ (and the TM is important, believe me!) from Bill Inmon last year and now Business Intelligence 2.0 as described by Neil Raden in his recent article.
In truth, Neil has got much closer to the new spirit of “2.0” as he recognises the impact of the changed paradigms of social computing, collaboration, mash-ups and other Web 2.0 approaches on traditional data warehousing. He’s also spot-on when says that “better ties between analysis and action” will emerge.
But my question is – will it be Business Intelligence 2.0 or will it simply be “Business Intelligence to zero”? If everything a business user needs is going to be mashed-up into a user-defined portal interface, and analysis and action (aka informational and operational) are intimately linked, why have Business Intelligence as a separate entity at all?
Two forces are acting simultaneously to change the face of BI beyond recognition. At the back end, SOA is gradually redefining the way applications interact, driving more integrated data and allowing more pervasive access to that same data. Data modelling, cleansing, extraction and transformation – traditional preserves of the data warehousing IT department – will become part of the mainstream IT business. Out on the user-facing side, business people increasingly expect an integrated view of all their tasks – without having to think if they are analytical or action-oriented.
As an example, take a look at the IBM Workplace Dashboard Framework, where traditional BI user function is delivered through a standard portal. Now recall that traditional action-oriented, operational applications can be surface through the same portal; and the portlets can exchange information! Where will the boundary exist between informational and operational in such a portal?
Neil is absolutely right when he says in conclusion: “Prepare for the inevitable”. It seems to me it’s the inevitable re-integration of BI into the mainstream of IT.
It's January 1st, 2007. The sun is shining in Dublin. It seems like a good day to start this blog!
First of all, let me welcome you all to my channel on BeyeNETWORK. It's a pleasure to be here and have the opportunity to talk to you all -- and most especially to hear your views, too.
This is going to be a short entry, since it's a holiday here in Ireland today, as in the rest of Europe. All the better to get over the hangovers, I guess!
I'd like to focus discussion here around the future of BI and how I see it evolving in the coming years. You can see that's where my interest lies from the contents of my first article and I hope it's something you are all interested in too. My belief is that we are going through quite a sea change in the BI world at present, driven by similar changes in the IT world in general. We have the opportunity now to define how we want to see BI evolve. If we don't participate, we will be swept along by the changes going on in the wider IT world. It would be a shame to be left behind ... so I hope you'll join me on this exciting journey.
|
UK EDITION
