Business Intelligence Network - The Vision for BI and Beyond

Part 1 of this series defined Web 2.0 and looked at its effect on business intelligence (BI) and data warehouse architectures. We observed that Web 2.0 empowers end users by facilitating collaboration and information sharing, thus enabling more innovation in analysis and decision making. We also noted that such freedom to innovate leads to the emergence of new ways of using and defining information which inevitably affects both the data and metadata architectures of the organisation. Our initial conclusion was that Web 2.0, like SOA, is necessary to support the concept of a highly evolved business (where operational, informational and collaborative environments work seamlessly together). We also saw that both these technology trends tend to put the traditional data warehouse and business intelligence architecture under some stress.

In Part 2, we examined Web 2.0 applications themselves and discovered that this market is still clearly in the hype phase, with hundreds of applications claiming to be Web 2.0. However, we were able to identify three application types - communicating, collaborative editing and collaborative working - that had the potential to disrupt traditional business intelligence approaches. While these types of Web 2.0 applications do empower users to collaborate better in decision making and to be more adaptive and innovative in their use of information, we did not find them on their own to be game-changers for business intelligence.

This article examines how the Web 2.0 paradigm actually does impact the traditional BI world when considered in combination with the simultaneous adoption of a service-oriented architecture (SOA) approach. For an in depth review of SOA and its own impact on business intelligence, please refer to Part 2 of my prior series of articles, “Business Intelligence is Dead Long Live the Highly Evolved Business”.

When thinking about both SOA and Web 2.0, the key word to keep in mind is “integration”. In the case of SOA, integration occurs behind the scenes; with Web 2.0, it’s in your face! These diverse but complementary types of integration are the functionality that change the playing field for business intelligence. Let’s first look at them separately.

Integration from the SOA Viewpoint

In an SOA environment, integration is between services and occurs in the network or on the servers of the business. The basic premise is that services that were never designed to work with one another can, through well-defined and understood interfaces, connect to an enterprise service bus or similar construct and communicate and call for one another’s support. This ability allows business processes to be constructed from piece parts and implemented more flexibly than is possible with traditional monolithic applications. The business processes are orchestrated through workflow technologies that mediate the inter-service communication that is required.

From a BI viewpoint, the first implication to note is that services need to be well-defined, both in terms of their actions and their interfaces. This implies that the data they emit or consume is also well-defined and described through metadata. BI developers have been asking for this for ever; and the arrival of well-defined and described services will certainly make their lives easier. So far, so good.

However, the second implication is more disruptive. While SOA is usually thought of as being applied in the operational environment, business users do not see any real difference between a service in the operational world (e.g., get customer name and address, update priority customer flag) and one in the informational environment (e.g., get top-spending customers by region). And a reasonable question then arises in the user’s mind: can we have a process composed of both operational and informational services (e.g., get top-spending customers by region and for each of these, get customer name and address, and update the priority customer flag)? Neither the operational system nor BI developers may individually know how create this workflow, but from the SOA point of view, it is still a very reasonable request. Furthermore, at the present stage of development, the workflow tooling still requires significant IT input to create and change process flows, so satisfying this request is not so easy. But, that will come, and when it does, the old barrier between the two parts of the organisation will have to fall.

The key point to note, though, is that SOA-driven integration, although demanded by the business, occurs deep in the IT world and under IT control. And by its very nature, it will force the currently separate worlds of operational and BI development to come together.

Integration from the Web 2.0 Viewpoint

Web 2.0 drives integration from the client or browser side of the network. In effect, users drive this type of integration directly. The Web 2.0 application types we discussed in my previous article don’t display integration in themselves. However, the user behaviour they support and encourage does create an integrative drive. For example, improved communication such as instant messaging or conferencing in a decision-making context inevitably requires sharing of information, the ability to compare different views of that information, bring in further resources and eventually to take a decision and update some part of the operational environment. Collaborative editing and working drive similar integrative behaviours.

However, it is one of the other Web 2.0 attributes that is the key integration enable for this mind-set. It’s not an application, but rather a way of creating and using the user interface - mash-ups. Mash-ups (derived originally from a term in the pop music industry for creating a new song by combining two or more existing pieces) is a technology that combines information from a number of sources into something more intrinsically useful than either piece alone. A common example is combining geographic location data with map data and displaying the result as an overlay such as houses for sale in my area.

Mash-ups can vary in complexity from the previous example, which requires some considerable level of programming skills right down to providing users with the ability to choose which applications they want to see in their browser at the same time (e.g., users can very simply build a home page experience using Google’s gadgets) and which can optionally share data and/or events between them. At this more simple level, mash-ups overlap with the traditional portal/portlet approach supporting dynamic web applications. However, as the data aggregation becomes tighter, the individual applications in the mash-up become true hybrid applications with their own unique business value.

While early mash-ups required significant programming skills from the developers, mash-up editors that simplify the process have now emerged so that increasingly complex types of mash-up are within the reach of end users. And users’ expectations of possible hybrid applications grow correspondingly.

The bottom line here is that Web 2.0 is leading end users to believe that they can and should have a new level of control on how they see and use their IT-provided service. And, increasingly, the technology is enabling them to bring that belief into reality.

Integration to the Power of Two

The emergence of both of these integration enablers in the same time frame is the real game-changer for business intelligence. Now, both IT and end users have the same belief structure: integration of currently disparate and diverse systems is good! It’s good for business and good for the users themselves. It’s good for IT and the way that the business views IT spending. Not just as a cost, but as a potential contributor to the company’s bottom line. With these beliefs, it becomes clear that integration will continue to grow, both within the operational, informational and collaborative disciplines and across them.

What are the implications for the business intelligence community? Trying to hold back the tide is probably not a viable strategy, as King Canute demonstrated to his fawning courtiers. This particular tide of integration, driven both by end users and the powerhouse of operational IT, is set to sweep away the currently independent IT princedoms of business intelligence and collaborative computing.

However, canny business intelligence princes may have already noticed that in some key, strategic areas, the IT king is wearing no clothes! Integration of information from disparate sources is something business intelligence has striven to achieve for more than 20 years. And it’s not as easy as it seems. So, integration of applications, based on the in-flight integration of the information they exchange, whether by SOA at the back end or Web 2.0 at the front, is likely to be a long and painful campaign. Clearly, a savvy BI prince may have something of value to bring to the battlefield. My next article, “Intelligent Integration Depends on Business Intelligence”, will deal with this topic.

For now, let’s wrap up the Web 2.0 topic. Our exploration of Web 2.0 has found it to be more of a shift in thinking than a deep, core change in the IT environment. The shift is largely in the minds of end users, empowering them to take more responsibility for their day-to-day interactions with the IT environment. Web 2.0 drives the need for a greater level of integration between currently disparate applications and the belief that such integration is possible and even to some extent within the hands of end users themselves. On their own, however, we have found only limited evidence that “Web 2.0 technologies” can effect the necessary level of change.

However, the combination of SOA and Web 2.0 is a potent force for change, uniting IT and end users in a common quest. Where Web 2.0 provides the user impetus, SOA contributes the IT engineering. Together, they do form the foundations of a highly evolved business.

If you found this article helpful and would like to receive the latest insights each month from Mike Ferguson and other experts featured on the Business Intelligence Network, please subscribe to the UK Business Intelligence Network Newsletter.