Blog: Mike Ferguson Subscribe to this blog's RSS feed!

Mike Ferguson

Welcome to my blog on the UK Business Intelligence Network. I hope to help you stay in touch with hot topics and reality on the ground in the UK and European business intelligence markets and to provide content, opinion and expertise on business intelligence (BI) and its related technologies. I also would relish it if you too can share your own valuable experiences. Let's hear what's going on in BI in the UK.

About the author >

Mike Ferguson is Managing Director of Intelligent Business Strategies Limited, a leading information technology analyst and consulting company. As lead analyst and consultant, he specializes in enterprise business intelligence, enterprise business integration, and enterprise portals. He can be contacted at +44 1625 520700 or via e-mail at mferguson@intelligentbusiness.biz.

August 2006 Archives

Having travelled to San Francisco just over a week ago to speak at the Shared Insights (formerly DCI) Data Warehouse and Business Intelligence conference, the experience of going through Manchester and Heathrow Airport security was something I don’t think I’ll forget for a long time. I started on my travels on the Saturday morning just after the mid-week scare in the UK when everything went critical. It was the first time in my life that I have travelled on business without my laptop due to the policy of absolutely no hand baggage. I have to say I felt almost naked without my laptop, but I didn’t want to run the risk of losing it or having it crushed as it travelled through the checked baggage system given the fact that I have reached many a destination airport only to find that my luggage did not. So I took the safe route and took my presentation on EII In-Depth on a memory stick and packed that in my suitcase.

I have to say as an airline passenger I don’t think I have ever felt so safe given the fact that security searched everyone (and were fairly pleasant in the process) at Manchester and again at Heathrow. Belts, shoes, jackets, watches and just about everything else was x-rayed. Nothing other than your travel documents and prescribed medicine was allowed on the plane, and so I watched every video British Airways had on offer! Even when we got on board, there was a one-hour delay while details of everyone who boarded were sent to the U.S. for checking before we were allowed to take off. When I got to the U.S. and picked up my suitcase, I was relieved to know that I had retrieved my PowerPoint presentation on my memory stick that I packed in my suitcase and could fulfil my obligation of presenting at the conference. You might think I should have e-mailed the presentation in advance – well, I did that too, but as many of you may agree, it is best to have a backup. Talk about secure – having got through passport control, I was then selected at random to have another check by U.S. customs who looked through my luggage and asked me a few questions and then sent me on my way with a “Welcome back to the U.S., sir.” By the time I got to my hotel, all I could think of was the seriously secure BI presentation that travelled with me through security on my memory stick. It was like I was being protected so well in order to get me safely to the conference to do my presentation. I dutifully then turned at the conference having requested a laptop in advance to present from – which Shared Insights happily provided. Of course, I asked them if they had the presentation I emailed in advance, and the answer was NO! So the moral of this story is always take a backup, and thanks to airport security for looking after my memory stick!!

An article on EII In-Depth will be the feature article the UK Business Intelligence Network Newsletter. Be sure to subscribe (it is free!) so you can be one of the first to read all about EII. The newsletter will also feature a great article about the globalisation of business intelligence.

Let me know what you think and what you are doing with enterprise information integration (EII) and business intelligence here in the UK.


Posted August 25, 2006 5:48 PM
Permalink | No Comments |

The announcement by Teradata (a division of NCR) that it is entering the Master Data Management marketplace will set heads turning in a fast-moving market that already has a number of vendors competing for business. Vendors already in this increasingly crowded market include:

-- FullTilt Perfect Product Suite
-- Hyperion MDM
-- Kalido 8M
-- Initiate Identity Hub
-- I2 PIM
-- IBM WebSphere Product Center and Customer Center
-- ObjectRiver MDM
-- Oracle Customer and PIM data hubs
-- Purisma
-- SAP NetWeaver MDM
-- Siperian Hub, Master Reference Manager, Hierarchy Manager and Activity Manager
-- Stratature Enterprise Dimension Manager
-- Sunopsis AIP

In addition to these vendors, data integration vendors (e.g., Business Objects, DataMirror, IBM, Informatica, Microsoft, SAS) and data quality vendors (e.g., DataFlux, Trillium, Zoomix) are all jumping in to offer additional software.

Teradata's first MDM offering is Teradata Product Information Management (PIM) solution. No doubt Customer Information Management will follow. The Teradata PIM solution is being rolled out after Teradata announced an agreement with I2 to roll its PIM solution onto the Teradata platform. There are obvious reasons why this makes sense for Teradata. Firstly their message has always been about building Enterprise Data Warehouses on the Teradata platform and single version of the truth. Adding a master data management solution to this platform allows a core dimensional data source to be on the same platform as a data warehouse making it less costly to manage (no separate server needed, existing administration skills for Teradata shops, etc.) and easy to supply data to analytical systems. In addition, Teradata has added event-driven support over the last several years via integration with popular messaging systems. While this was originally implemented to introduce real-time event driven data capture into a data warehouse, there is nothing to stop Teradata exploiting it as a two-way mechanism so that data changes to master data can be set out of the Teradata MDM platform to other operational applications to keep them synchronised.

An obvious question is transaction processing and MDM on a platform optimised for analysis and business intelligence. Even there, it seems that this should not be a problem for Teradata since they have long had workload management and the ability to fence off hardware to dedicate to specific tasks. Furthermore, Teradata could instantly analyse changes to master data on their platform even before moving that data into a data warehouse to reveal early behavioural shifts. Not only that, but with master data and the data warehouse on the same platform it would make it relatively easy to introduce versions of dimensions into an analytic data warehouse environment and quickly compare business metrics across different hierarchies over time.

All in all, the arrival of Teradata in the MDM market makes a lot of sense for them. The question is whether this MDM solution is implemented as a system of record only (i.e., changes to product data come though other operational applications which remain systems of entry) or whether Teradata can convince prospects and existing customers to make it both a system of record and a system of entry. Doing the latter would take Teradata into new territory -- that of operational transaction processing. In a world whereby many companies are looking to simplify and consolidate data, this move may well trigger significant growth for a company with a long track record in data.


Posted August 11, 2006 7:54 PM
Permalink | No Comments |

Increasingly here in the UK, I am getting asked about whether master data management (MDM) has a role in business intelligence (BI). The obvious answer is that master data management systems can play a significant role in BI by supplying the dimension data needed in BI systems. Of course, MDM systems also provide operational systems with this core data and with changes to such data.

On the surface, MDM looks fairly straightforward, and looks like it just involves integrating data into core data entity (e.g., product, customer and asset) data hubs and then managing change to such data. But is it that simple? After all, some vertical industries seem to be well down the road here (e.g., manufacturing) and are supplying dimension data to BI systems already from MDM data hubs. However, many people I talk to, across most vertical industries, are still trying to get to grips with the impact of MDM on their business. Why so I wondered? As I have researched more deeply into this problem I am not surprised at the hesitation as there is a lot more to this than first meets the eye. In the work I have been doing with clients, it is clear that introducing master data management hubs into a company is not the problem as long as these hubs are simply systems of record (SORs) and not systems of entry (SOEs). If they are just systems of record, then this in not really master data management; it is master data integration. In other words, changes to master data continue to occur in existing operational applications and these changes are trickle-fed to the master data hubs. However, if we want true MDM, then this means that the MDM system has to become a SOE, and to make that happen means that a company has to undertake a huge amount of business analysis work and application change management to wrestle the problem of master data management to the ground. I am already clear that this is a multi-year project.

While there is a thirst for master data management applications such as customer information management, product information management etc., companies have to think about the consequences of these applications on their business and on capital and IT resources needed to implement it. In my mind, there are all sorts of problems here. Looking at customer data, for example, the issue here is really to do with parties (people or companies, for example). A person can have multiple identities (e.g., an employee number, a customer number, a national insurance number, etc.) Also, in some cases, specific vertical industries such as banking assign even more identifiers to a party. In this case, they often use product numbers to identify customers (e.g., a credit card number, a mortgage roll number, etc.) such that if you have a two credit cards, a loan, a mortgage, and are an employee, you are identified differently across a range of line of business applications. Therefore, a party, for example, is awash with identifiers.

So the first problem is that you need to assign a Global Identifier to a master data entity and a mechanism to map all the other identifiers to the global one. Then there are the master data entity (e.g., customer, product, asset...) attributes that describe that entity. When looking at these attributes, the problem I run into is that many different line of business operational applications are systems of entry (SOEs) for these attributes. These existing disparate operational systems are therefore involved in distributed maintenance of that master data. In addition, in many cases, these disparate systems also double up as systems of record (SORs) for just the attributes they maintain. In other words, we have fractured master data with parts of an entity (subsets of its attributes) maintained by different applications. The consequence for many companies today is that they have had to put in place FTP and messaging solutions to move subsets of master data between systems in order to synchronise it in different applications. While this may have been manageable initially on a few systems, many companies are now at the point were they have FTP and message mania. We talk about XML messaging - well we spelt it wrong. It should take out the M and pronounce it EXCEL messaging!!

When I look at vendor offerings that talk about data hubs as a way to centralise master data, it is clear that these hubs can be systems or record but are a far cry from systems of entry unless you switch off all existing ways to update such data in other applications. Therefore, all I can see is that master data hubs can become the system or record while existing applications remain the systems of entry maintaining changes to master data attributes. Most people I talk to can get their head around that - i.e., line of business changes to master data are propagated to the master data hub. However, this is just master data integration. It is not MDM. MDM requires that the MDM system is the system of record and the system of entry and also that it manages and tracks all changes to such data over time. This includes things like product and customer hierarchies.

Companies therefore have to make sure that what they are buying can do all of this and not just master data integration. Even if an MDM solution can be both an SOR and an SOE, getting to that point is not easy. To implement this means that companies have to:



  • Identify and model all the business processes they have in place in order to know what process activities are associated with changing master data - i.e., all SOE activities
  • Identify and make a list of all functions in all existing operational applications that maintain specific master data attributes - i.e., that perform maintenance on that data
  • Identify all screens in all existing applications that allow users to maintain master data
  • Identify all duplicate functions across existing systems that are maintaining the same master data attributes so that potential conflicts in master data systems of entry are identified
  • Start a change management implementation program that systematically causes change to line of business systems in order to

    • Weed out duplicate or overlapping functionality to stop master data conflicts in existing systems of entry
    • Change line of business SOE applications to switch off data entry screens or to remove master data attributes from line of business application forms
    • Introduce equivalent forms or screens in the MDM system to directly maintain the master data in MDM data hubs (this is a change from many systems of entry to one system of entry)
    • Understand the impact of that kind of change on existing line of business applications (e.g., line of business SOE systems may update more than master data in a specific transaction in which case the business logic in the line of business application may also have to change)
    • Introduce data synchronisation from the MDM system of record back to the line of business systems to keep these applications in sync
    • Change business processes to cause change to master data using the new process
    • Introduce common master data services that applications should call to access and maintain master data in such data hubs
    • Change existing ETL jobs to start to take specific master data from the MDM system rather than line of business systems to populate BI system databases
    • Etc. etc.


Looking at this, it is clear that in solving the MDM problem, the move towards a true enterprise MDM system as opposed to just a master data hub that integrates master data, is going to be a long hard slog. This is a multi-year program that is more like an "iceberg" project where there is a lot more to it than first might seem necessary.

My concern here is that part of the way down this road, many companies could hit a major roadblock in the form of packaged applications that act as SOEs for master data. An example might be CRM applications maintaining customer data. How can companies remove/disable screens or alter on-line forms in these packaged applications to switch to a true MDM system. It assumes all packages can be customised. Maybe in first tier packaged applications (SAP, Oracle packages, etc.) this is possible, but in other applications reality tells me we'll be lucky if it's the case. So reality states, therefore, that you may not get all the way to the true MDM promise land. The best thing is to keep advancing systematically as far as you can go.


Posted August 8, 2006 8:36 AM
Permalink | 2 Comments |

Welcome to my blog on the UK Business Intelligence Network. It seems strange that I should be writing a UK blog from Italy! Rapallo, to be precise. What can I say, if it's Friday and you can't figure out why your ETL job is not working or if your metrics won't calculate correctly on your OLAP server, just leave it, get out of the office and come to this stunning place for a weekend. You won't regret it. Just fly to Genoa and get a cab - it's about a 30 minute drive

Anyway, I hope to help you stay in touch with hot topics and reality on the ground in the UK and European business intelligence markets and to provide content, opinion and expertise on business intelligence (BI) and its related technologies. I also would relish it if you too can share your own valuable experiences on the discussion forums and surveys. Let's hear what's going on in BI in the UK. If you've got a gripe, let's hear it. If you think a product is cool, let's hear that too, and most of all, let's hear what your using BI for in your business and what you want to see covered. I'll try to explore established and newly emerging technologies in BI including data integration, BI platform tools such as reporting, OLAP, dashboard builders, predictive analytics and performance management software such as scorecarding and planning. I'll also look at data warehouse appliances, data visualisation, portals and operational BI and at BI applications.

I'll start the ball rolling by looking at the platform market. Consolidation is upon us in the business intelligence market with large BI vendors and software giants battling it out for market share. Business Objects, Cognos, Hyperion and SAS are the largest of the independent BI vendors competing with IBM, Microsoft, Oracle and SAP for the BI platform crown in most enterprises. Of course, there is plenty of action elsewhere from may other BI vendors. An example of that is the increasing number of data warehouse appliance offerings on the market from start-ups and established BI vendors including (in alphabetical order) Calpont, DATAllegro, Greenplum, IBM BCU, Netezza and SAP BI Accelerator - with more to come I am sure.

Much talk over the last few years has been about the move to single vendor BI platforms. In the UK, I hear a lot of talk about it. Companies building BI systems for the first time are definitely going for single supplier BI platform solutions. However, companies with long established best-of-breed BI systems still need to be convinced but will most likely slowly move to fewer suppliers if there is value in doing so. I think the impact of Microsoft in the BI platform market with SQL Server 2005 BI platform tooling (SQL Server Integration Services, SQL Server Analysis Services, SQL Server Reporting Services, SQL Server Notification Services and SQL Server Report Builder) appears to be having an impact on BI platform pricing. Certainly many companies talking to me have been raising the issue of pricing. Emerging open source BI initiatives (e.g., Jasper, Pentaho, BIRT and Palo, to name a few) are also bound to add to that pricing pressure forcing BI platform pricing down over the next 18 months. If this does happen, it is obvious that vendors will compensate by shifting the value to performance management tools and applications, vertical BI applications and the explosion of interest in data integration and enterprise data management.

Enterprise data management seems to be taking on a life of its own with data integration now breaking free of the BI platform and going enterprise-wide. Here we have ETL, data quality, EII, data modelling and metadata management all coming together into a single tool set. Much is being written about EAI, ETL and EII (or as I like to call it EIEIO !!), and I'll look into these areas in upcoming blogs and articles.


Posted August 1, 2006 10:13 PM
Permalink | 1 Comment |