Techniques for Integrating Business Intelligence into the Enterprise, Part 3

Originally published 18 July 2006

Editor's note: This article was originally published at

In this third part of my series of articles on integrating business intelligence (BI) into business processes, I would like to investigate business intelligence Web services the third of the four techniques I defined. These four techniques are:

  1. Integration of analytical applications with operational applications using an enterprise portal for access and exploitation by internal and external users.
  2. Embedding analytics in operational applications during application development.
  3. Introducing Web services to dynamically integrate analytical processing with internal and partner operational applications for supporting collaborative commerce.
  4. Deploying event-driven on-demand processing for user alerts, real-time recommendations and automated actions. This approach includes business activity monitoring (BAM).

What Are Web Services?
Web services provide a means of dynamically integrating intra- and inter-enterprise application processing using Web technology. They can be used to provide access to business processes, applications, business intelligence cubes, reports, queries and data integration functions, databases, etc. Each Web service has an interface that describes operations accessible via standardized XML messages. Web services are described using the industry standard Web Services Description Language (WSDL), which is simply a set of XML tags. In order for people to find out what Web services exist, it is necessary to publish them to a registry, which acts as a yellow pages of all Web services. This registry is called a UDDI (universal description, discovery and integration) registry. Most companies have private internal UDDI registries. There are also external public UDDI registries on the Internet for services that you wish to expose to external portals and applications. In order to dynamically connect to a Web service, an application or portal simply queries the UDDI registry, finds the service and then dynamically connects to it by sending it an XML message known as a SOAP (simple object access protocol) message. This invokes the Web services to do something and return an XML response or answer. This is shown in Figure 1.

Figure 1

BI Web services are things such as queries, reports, cubes, ETL workflows and mining models that can all be published as Web services and invoked on-demand. BI Web services can be used at multiple levels. These include:

  • Application level BI Web services
  • Data level BI Web services
  • Process level BI Web services

Application Level BI Web Services
Application level BI Web services allow other applications and portlets to dynamically invoke business intelligence tools and analytic applications to request BI on demand via an industry standard Web service interface. For example, a request may be to run a query, produce a report, invoke a data mining model or access a cube. The resulting data from executing these services is then rendered in XML and passed back to the calling operational application for display or further processing. Business intelligence tools and packaged analytic applications can publish reports, cubes and mining models as Web services and also integrate them directly into a portal as Web service remote portal (WSRP) portlets so that users can gain access to such services directly from a portal user interface. In addition, application level Web services allow business intelligence tools and packaged analytic applications to consume XML from other Web services including BI Web services in the database management system (DBMS), unstructured content services and data integration Web services (e.g., enterprise information integration/EII and extract, transform and load/ETL Web services) that integrate data on-the-fly to provide the data back to a BI tool. Several BI tools (e.g., BusinessObjects, Oracle Discoverer) have the ability to access XML data from other services and can incorporate this data into a report for example to enrich the BI they produce. Web services also allow power users and developers to develop custom-built BI Web services portlets using analytic application development tools and Java or .NET IDE tools. Application level use of BI Web services is shown in Figure 2.

Figure 2

Several BI platforms, including Business Objects, Cognos, Hyperion and SAS, now have Web service interfaces. As an example, Figure 3 shows how Cognos' Web service interface to their BI platform can be exploited to integrate active cross-tabs and graphs into the BEA AquaLogic Portal Server.

Figure 3

This is made possible because enterprise portals allow portal administrators to integrate remote Web services into portals as portlets. Figure 4 shows a screen shot of how a portal administrator using IBM WebSphere Portal Server can do this.

Figure 4

Other applications can call BI Web services directly. For example, if a call centre operator is using a CRM package such as Oracle's Siebel as a customer service application while dealing with a customer, the operator can request BI about that customer on demand at the click of a button. Oracle Siebel could then request BI from a BI tool (e.g., predictive analytics such as propensity to buy or propensity to churn) by invoking mining models that have been published as Web services from within Oracle Siebel. This would retrieve the customer's history from a data warehouse and mine it to provide the requested analytics back to Oracle Siebel in XML. Equally, on-demand recommendations can be provided in a similar manner. Figure 5 shows how application BI Web services can be invoked.

Figure 5

Data Level BI Web Services
Two main forms of BI Web services exist at the data level. These are BI Web services in the database itself and data integration Web services such as EII federated query-based BI Web services and extract, transform and load workflow-based BI Web services that can deliver integrated BI and other related content on demand. Both database and data integration BI Web services can be consumed by applications, BI tools and portals.

BI Web services in the DBMS effectively appear as user defined functions (UDFs) that can be exploited by applications and tools that issue SQL or XML queries against the database. IBM's DB2 and Microsoft SQL Server 2005, for example, support Web services in the DBMS. Microsoft SQL Server Analysis Services and several other multidimensional DBMSs also support XML for Analysis (XMLA), which allows multidimensional queries to be executed as Web services. In IBM DB2, an administrator can browse a UDDI registry and import Web services into the DBMS as user defined functions. DB2 generates CREATE USER DEFINED FUNCTION statements for each Web service imported and therefore effectively adds additional functions to the pre-built set of statistical, regression and OLAP functions provided by the DBMS out of the box. BI tools and application can then invoke these functions via SQL which causes DB2 to invoke the appropriate Web services on the Net. This is effectively using BI Web services to exploit grid computing. Hence, the DBMS can outsource functionality to other BI Web services and then collect the data from these services before passing a result back to the application or tools querying the DBMS (see Figure 6).

Figure 6

Many extract, transform and load tools are also now introducing Web service interfaces which means that new ETL workflows can be built in ETL tools and published as Web services. Applications and portals can then invoke these to integrate data on demand (i.e., right now). Hence, data can be retrieved from multiple sources about a person, a product or an asset, for example, and integrated on the fly before passing the data back to an application. In this case, a data warehouse or data mart can be used as a source as opposed to a target. Here, the target is the application that requests the data via a Web service XML request (see Figure 7). This is effectively ETL tools evolving into the enterprise information integration market to support on-demand data integration. Several ETL tools already support this capability including BusinessObjects Data Integrator and Informatica. Of course, enterprise information integration products also exist such as BEA AquaLogic Data Services, Composite, Ipedo, MetaMatrix, IBM WebSphere Information Integrator and XAware. These products also integrate data on demand and use federated query to integrate data. Each EII query can be published as a Web service and invoked on demand to access data in data warehouses, data marts and other systems, and integrate it before passing it back to applications or portals as XML.

Figure 7

If you have a complex BI environment with multiple BI data stores, then on-demand data integration can also be used to integrate data from multiple BI systems to calculate higher-level metrics needed in executive scorecards and dashboards on demand.

Process Level BI Web Services
At the business process level, BI Web services can be invoked as activities that are part of an executing business process (e.g., to produce intelligence in the form of a report and distribute it to people). Here, business process management software would invoke the BI Web services at particular point(s) in a process to execute a BI Web service in the context of a particular process activity being performed at the time. In addition, business intelligence Web services can be invoked to provide support for business activity monitoring. In the case of BAM, it is likely to be an event such as an order change or a cancelled order that triggers the execution of a Web service. In this case, the event may trigger on-demand data integration, automated analysis and automated action. BAM is an exciting area for business intelligence and something I explore in Part 4 of this series.

Read other installments of this series: Part 1, Part 2and Part 4.

Recent articles by Mike Ferguson



Want to post a comment? Login or become a member today!

Be the first to comment!