Business Intelligence Network - The Vision for BI and Beyond

Web 2.0 and the Highly Evolved Business, Part 2

Published: 3 October 2007

Barry Devlin provides a deeper look at Web 2.0 applications and how they impact traditional data warehouse and business intelligence architectures.

Part 1 of this series defined Web 2.0 and looked at its effect on business intelligence (BI) and data warehouse architectures. We observed that Web 2.0 empowers end users by facilitating collaboration and information sharing, thus enabling more innovation in analysis and decision making. We also noted that such freedom to innovate leads to the emergence of new ways of using and defining information, which inevitably affects both the data and metadata architectures of the organisation. Our initial conclusion was that Web 2.0, like SOA (service-oriented architecture), is necessary to support the concept of a highly evolved business (where operational, informational and collaborative environments work seamlessly together). We also saw that both these technology trends tend to put the traditional data warehouse and business intelligence architecture under some stress.

In this article, we’ll take a deeper look at Web 2.0 applications to try to determine how much they actually do stress the traditional architecture.

Defining a Web 2.0 application is not easy! A quick search of the Web reveals dozens, if not hundreds, of applications claiming to be or labelled as Web 2.0. For ease of use, we’ve categorized them as follows:

• Publishing: blogs and other forms of self-publication in differing formats
  Social networking: sharing personal information to create communities of interest on the Web
  
  
• Resource sharing: sharing files, photos, bookmarks
  
  
• Communicating: Atom/RSS feeds, e-mail systems, instant messaging, group chats and web conferencing
  
  
• Collaborative editing: wikis and similar collaboratively authored documents
  
  
• Mashing: combining information and/or applications from different sources to create new information
  
  
• Collaborative working: collaborative use of business assets such as spreadsheets or databases

There are two primary ways in which such applications can impact or influence the development of the highly evolved business. The first is the extent to which information usage is changed, particularly through collaboration. Changes in the use of information – how it is created, stored, verified, shared, edited or deleted – clearly have the potential to affect business intelligence systems, especially where they meet the user: in the data marts or in personalized subsets of data marts often stored and manipulated as spreadsheets.

The second area of influence of Web 2.0 applications is the extent to which they enable or drive changes in the underlying processes of the business. Because business intelligence has traditionally been, at best, loosely linked to business process, the impact on BI here might be considered minimal. This is true if we look only backwards. Looking to the future, however, is a different matter. The highly evolved business demands much tighter integration of information and process in order to ensure that information-based insights are reflected more rapidly into the running business process for accurate and speedy decisions. Similarly, changes in the operational world need to be made visible (almost) instantaneously in the informational environment. Understanding how Web 2.0 applications can change business processes is thus equally important in the larger context.

Three Important Categories of Web 2.0 Application

Figure 1 lays out the Web 2.0 application categories on a 2x2 grid according to their impact on collaborative information use (x-axis) and process change (y-axis).

Figure 1: Categories of Web 2.0 Application

In terms of their impact on the business, the three categories on the outer edges of the diagram – communicating, collaborative editing and collaborative working – are the most important and will be the focus of our attention.

Communicating, especially when done in real-time or near real-time, has the potential to improve or speed up existing processes by creating linkages between different parts of the overall process model of the business. For example, when some threshold has been exceeded in the business intelligence environment, information can be communicated via an Atom/RSS feed directly to a decision maker who needs to know about it or, in highly automated processes, directly to the operational system where action is taken. Instant messaging similarly enables interpersonal communication between different areas of the business, short-cutting slower or more error-prone traditional approaches. Web conferencing and group chats create collaborative communities in real-time that can quickly discuss and decide on actions based on a common view of business intelligence results.

Collaborative editing of wikis and similar group-authored documents was an early example of the type of collaborative information use that Web 2.0 enables. While not directly relevant to the highly structured data found in business intelligence environments, it can still play an important role in the highly evolved business. Collaborative editing is a powerful tool in the creation and ongoing maintenance of the descriptive information that supports process definitions, data meanings and similar metadata. When such metadata is subject to ongoing change as users are empowered to collaboratively create new processes or information, it is important that metadata maintenance is also collaborative and easily achieved.

Perhaps the most interesting category from a business intelligence viewpoint is that of collaborative work. These types of applications are still in the early stages of development, but are beginning to show promise. Collaborative applications allow multiple users to cooperate on the creation and editing of shared business information. eXpresso and Numbler are examples of tools that allow collaborative editing of spreadsheets and provide various ways of keeping different users informed of changes that have been made in the data. Many Eyes is an IBM Research project that encourages multiple users to play with different ways of visualizing the same data set and discussing their observations.

This last category of Web 2.0 application has the potential to be the most disruptive to the traditional business intelligence environment. By enabling end users to more easily manipulate data and the ways of looking at it, these applications encourage innovation. However, they will also drive a proliferation of incompatible views of the business that must then be reconciled as they go up the organisational hierarchy. The challenge for the IT department will be to create both a sandbox where such innovative thought and experimentation can thrive and a formal process and tooling through which useful innovations can be promoted to shared, corporate use. The challenge for the users themselves will be to understand when it’s appropriate to innovate and when the formal processes must be followed. For both parties, comprehensive and simple metadata management will be key and, as previously mentioned, collaborative editing can help.

At the end of Part 1, we posed this question: Is the data warehouse architecture being stretched beyond its breaking point by Web 2.0 combined with SOA? Looking at Web 2.0 applications alone, the answer clearly is that it is not. The pressures introduced by these applications are largely similar to those caused by the long-standing proliferation of spreadsheets as the primary tool for management information in many companies.

However, the effect of the combination of SOA and Web 2.0 is another question. But the answer to that must wait until Part 3!

If you found this article helpful and would like to receive the latest insights each month from Mike Ferguson and other experts featured on the Business Intelligence Network, please subscribe to the UK Business Intelligence Network Newsletter.