I commented in a my previous blog entry that the controversy over ParAccel's TPC-H benchmark has become quite heated. This is especially true on Curt Monash's blog where at one point he made some personal comments about Kim Stanick, ParAccel's VP of Marketing. See this link for details.

This is the second blog this month that I have read where an analyst makes an attack, not only on the vendor, but also one of its employees. The other blog (and an associated article) was by Stephen Few entitled "Business is Personal - Let's Stop Pretending It Isn't." See this link for details.

The good thing about social computing is that it provides a fast way of sharing and collaborating about industry developments. However, these technologies have the same problems as e-mail and instant messaging, they enable people to react immediately to something that upsets or annoys them. With blogging, unlike email and instant messaging, everyone gets to see the results!

As analysts our job is to write balanced reviews of industry developments that provide useful information to the reader. My concern is that some analysts are behaving as though they are on cable television or writing for the tabloids. I believe we can critique a product without attacking a company, its products or its employees. Personal attacks by analysts are unprofessional, even if the company fights back against a review they take exception to. What do you think?

ParAccel, one of the new analytic DBMS vendors, recently announced some impressive TPC-H benchmark results. A good review of these results can be found on Merv Adrian's blog at this link.

Not everyone agreed with Merv's balanced review. Curt Monash commented that "The TPC-H benchmark is a blight upon the industry." See his blog entry at this link.

This blog entry resulted in some 41 (somewhat heated) responses. At one point Curt made some negative comments about ParAccel's VP of Marketing, Kim Stanick, which in turn led to accusations that his blog entry was influenced by personal feelings.

I have two comments to make about this controversy. The first concerns the TPC-H benchmark and the second is about an increasing lack of social networking etiquette by analysts.

TPC benchmarks have always been controversial. People often argue that that do not represent real life workloads. What this really means is that you mileage may vary. These benchmarks are expensive to run and vendors throw every piece of technology at the benchmark in order to get good results. Some vendors are rumored to have even added special features to their products to improve the results. The upside of the benchmarks is that they are audited and reasonably well documented.

The use of TPC benchmarks has slowed over recent years. This is not only because they are expensive to run, but also because they have less marketing impact than in the past. In general, they have been of more use to hardware vendors because they demonstrate hardware scalability and provide hardware price/performance numbers. Oracle was perhaps an exception here because they liked to run full-page advertisements saying they were the fastest database system in existence.

TPC benchmarks do have some value to both the vendor and the customer. The benefits to the vendor are are increased visibility and credibility. Merv Adrian described this as a "rite of passage." It helps the vendor get on the short list. For the customer these benchmarks show the solution to be credible and scalable. All products work well in PowerPoint, but the TPC benchmarks demonstrate that the solution is more than just vaporware.

I think most customers are knowledgeable enough to realize that the benchmark may not match their own workloads or scale as well in their own environments. This is where the proof of concept (POC) benchmark comes in. The POC enables the customer to evaluate the product using their own workloads.

TPC benchmarks are not perfect, but they do provide some helpful information in the decision making process.

I will address the issue of blog etiquette in a separate blog entry.  

My post a couple of days ago about data warehousing in the cloud led to requests for more information about this topic and related SaaS BI solutions.  

Claudia Imhoff and I recently published a research report on the BeyeNETWORK entitled "Pay as You Go: Software-as-a-Service Business Intelligence and Data Management." The report was sponsored by Blinklogic, Host Analytics, PivotLink and SAP BusinessObjects who offer SaaS BI solutions. It was also sponsored by Kognitio who (like Aster, GreenPlum and Vertica mentioned in my previous blog) have a data warehousing in the cloud offering. The report discusses SaaS BI and data warehousing and reviews the pros and cons of using this type of deployment model.

The report can be found on beyeresearch.com.

The use of cloud computing for data warehousing is getting a lot of attention from vendors. Following hot on the heels of Vertica's Analytic Database v3.0 for the Cloud announcement on June 1 was yesterday's Greenplum announcement of its Enterprise Data Cloud™ platform and today's announcement by Aster of .NET MapReduce support for its nCluster Cloud Edition.

I have interviewed all three vendors over the past week and while there are some common characteristics in the approaches being taken by the three vendors to cloud computing, there are also some differences.

Common characteristics include:

• Software only analytic DBMS solutions running on commodity hardware
• Massively parallel processing
• Focus on elastic scaling, high availability through software, and easy administration
• Acceptance of alternative database models such as MapReduce
• Very large databases supporting near-real-time user-facing applications, scientific applications, and new types of business solution

The emphasis of Greenplum is on a platform that enables organizations to create and manage data warehouses and data marts using a common pool of physical, virtual or public cloud infrastructure resources. The concept here is that multiple data warehouses and data marts are a fact life and the best approach is to put these multiple data stores onto a common and flexible analytical processing platform that provides easy administration and fast deployment using good enough data. Greenplum sees this approach being used initially on private clouds, but the use of public clouds growing over time.

Aster's emphasis is on extending analytical processing to the large audience of Java, C++ and C# programmers who don't know SQL. They see these developers creating custom analytical MapReduce functions for use by BI developers and analysts who can use these functions in SQL statements without any programming involved.

Although MapReduce has typically been used by Java programmers, there is also a large audience of Microsoft .NET developers who potentially could use MapReduce. A recent report by Forrester, for example, shows 64% of organizations use Java and 43% use C#. The objective of Aster is to extend the use of MapReduce from web-centric organizations into large enterprises by improving its programming, availability and administration capabilities over and above open source MapReduce solutions such as HADOOP.

Vertica see its data warehouse cloud computing environment being used for proof of concept projects, spill over capacity for enterprise projects and for software-as-service (SaaS) applications. Like Greenplum it supports virtualization. Its Analytic Database v3.0 for the Cloud adds support for more cloud platforms including Amazon Machine Images and early support for the Sun Compute Cloud. It also adds several cloud-friendly administration features based on open source solutions such as Webmin and Ganglia.

It is important for organizations to understand where cloud computing and new approaches such as MapReduce fit into the enterprise data warehousing environment. Over the course of the next few months my monthly newsletter on the BeyeNETWORK will look at these topics in more detail and review the pros and cons of these new approaches.

Aster Data Systems has announced the Aster nCluster Cloud Edition. The Aster nCluster is an analytical database system for supporting very large relational databases. Aster joins companies such as Vertica and Kognitio in providing data warehousing capabilities on a cloud computing platform. Aster supports both the Amazon and AppNextus cloud computing environments.

This announcement demonstrates the growing demand for analytical processing from companies that have a significant presence in the web marketplace. One of the users of the CloudEdition, for example, is Didit, a web ad marketing company. These types of companies need to analyze huge amounts of web-related data.

One interesting aspect of Aster is its support for MapReduce, which is a growing trend by database companies Greenplum has MapReduce support and IBM is working on supporting it (System S research project). MapReduce provides a framework for massive parallel processing and is used by a number of web-centric companies such as Google. A key distinguishing feature for Aster is that it supports custom SQL functions that exploit MapReduce. MapReduce capabilities could become a key differentiator in high volume cloud computing data warehousing environments.  

You can find out more about the use of MapReduce in database systems in an article I published on the BeyeNetwork entitled "Are Relational Database Systems Keeping Up with the Information Processing Needs of Companies?"

Given my database background, it's exciting to see that database products are moving away from being commodity items are now offering some important distinguishing features.

Microsoft today announced major changes to its PerformancePoint BI product. The current version of the product, PerformancePoint 2007, provides three BI related capabilities: business performance management (BPM), BI analytics, and business planning.

The BPM component evolved from the Microsoft Business Scorecard Manager, whereas the analytics component is based on a subset of the functionality acquired from ProClarity. The planning component was a new component developed by the Microsoft BI group aimed specifically at financial planning and budgeting.

Today's announcement breaks out the BPM and analytics components from PerformancePoint and merges them into the Enterprise Edition of Microsoft SharePoint Server. Existing SharePoint Enterprise Edition users (with Software Assurance) will now get these components as a part of their licensing agreement. These customers will be able to download the PerformancePoint components starting April 1.

In the summer, Microsoft will release Service Pack 3 of PerformancePoint 2007. This will be the final release of the product, which will be supported for ten years.

Microsoft's strategy is to move the responsibility for financial planning and budgeting to the Microsoft Dynamics FRX and Forecaster products. However, horizontal planning capabilities will continue be added to Microsoft SQL Server, Excel, and SharePoint over future releases.

This new direction makes sense for Microsoft. Although Microsoft was emphasizing the BPM and planning capabilities of PerformancePoint, it was achieving limited success in these areas. Instead, the majority of customers were buying the product for its analytics capabilities. This was especially true for ProClarity users.

Another reason why this makes sense is that Microsoft SharePoint is a very successful product, and this is leading to companies purchasing related Microsoft solutions. Over 80 percent of PerformancePoint customers, for example, are also SharePoint Server users. The penetration of SharePoint in the market is also a key factor in the success of SQL Server and its BI components.

Given that Microsoft Office is also increasingly being integrated with Microsoft SharePoint, it means that customers will now be motivated to purchase Microsoft Office, SQL Server, and SharePoint Server as a product set in order to deploy business intelligence and related collaborative tools to a mass business user audience.

I came across this article in the New York Times about the R programming language. It was interesting to note that it was the number 2 most read article in the technology section.

http://www.nytimes.com/2009/01/07/technology/business-computing/07program.html?em

The article suggests R is a threat to SAS. Any perspectives on this or the use of R for data analysis?

Well it’s the last day of 2008, and it’s tradition at this time of year to make predictions for the coming year. If the financial chaos of the last few months continues into 2009, which most financial pundits say it will, then the IT industry is heading for a tough time over the coming year. This makes predicting industry directions really difficult because IT organizations are less inclined to purchase new products and technologies when budgets are tight.

The business intelligence (BI) marketplace has often been immune to industry downturns. This is because companies often turn to BI in difficult times to help them identify areas where revenues can be increased and costs can be reduced. This is especially the case in front office sales, marketing, and support organizations. Given the potential size of the coming downturn, however, can even BI be immune? I doubt it.

I believe, however, that there are ways BI can ride out the coming storm and be of benefit to the business. I think the main task that organizations should focus on is using new BI technologies to reduce costs (rather than increasing revenues). This can be achieved by reducing the cost of delivering new BI business solutions and by increasing business user productivity.

The BI solutions that will have the most impact in 2009 will be those that provide IT and business users quick and low-cost approaches for discovering, accessing, integrating, analyzing, delivering and sharing information in a way that that helps business users become more productive and more self-sufficient.

This means that there will be increased interest in open source software, BI software-as-a-service, low-cost application appliances, search, the integration of BI with collaborative and social computing software, rich internet applications, web syndication, and data and presentation mashups. Many of these solutions will come from small innovative BI companies, rather than large software companies who are still struggling to integrate the morass of BI software they acquired in 2008.

The technologies mentioned support low cost and fast BI application deployment. Many of them will be used by line-of-business IT rather than the enterprise IT organization. This could result in a turf war where enterprise IT tries to control and govern the use of these new technologies by the business. This would be a huge mistake. Instead enterprise IT should look for best practices in the use of these technologies by business groups, replicate them in other parts of the organization, and look for ways of incorporating the cream of the crop into the existing IT environment.

The purists will cry that this will lead to anarchy and islands of data and software. If that is the case then so be it. In the coming 12 months we need to do what ever it takes to be productive and reduce short-term costs. This is not the time for fancy architectures, purist approaches, academic debates, or large projects.

Have a great 2009!

Relational database systems, such as IBM DB2 and Oracle Database, have undergone over a quarter century of development. During that time they have managed to successfully fight off competing database technologies for supporting mainstream database management. Do you remember the object/relational wars of the eighties?

MapReduce, a software framework introduced by Google for supporting parallel processing over large petabyte files has garnered significant attention of late. IBM is experimenting with this in conjunction with Google, and GreenPlum recently announced support.

The significant interest in MapReduce, and related technologies such as Hadoop and HDFS, has led to a backlash from the relational camp. David DeWitt and Michael Stonebraker have been especially outspoken (see www.databasecolumn.com/2008/01/mapreduce-a-major-step-back.html and www.databasecolumn.com/2008/01/mapreduce-continued.html).

Here is a small quote from their thoughts on the topic:

"As both educators and researchers, we are amazed at the hype that the MapReduce proponents have spread about how it represents a paradigm shift in the development of scalable, data-intensive applications. MapReduce may be a good idea for writing certain types of general-purpose computations, but to the database community, it is:

1. A giant step backward in the programming paradigm for large-scale data intensive applications

2. A sub-optimal implementation, in that it uses brute force instead of indexing

3. Not novel at all -- it represents a specific implementation of well known techniques developed nearly 25 years ago

4. Missing most of the features that are routinely included in current DBMS

5. Incompatible with all of the tools DBMS users have come to depend on"

Does this mean the database wars are starting up again?

My opinion is that MapReduce is not intended for general purpose commercial database processing and is therefore not a major threat to relational systems. However, it does have its uses (as Google has demonstrated) for certain types of high volume processing. It also demonstrates that as data volumes get bigger, and the complexity of data and data structures increases, other types of database technology may start to gain traction in certain niche marketplaces. The use by IBM of the SPADE language, instead of StreamSQL, in its InfoSphere Streams product (System S) also demonstrates the changes going on in the database market.

What do you think?

The Apple iPhone has had a dramatic impact on the mobile phone industry, and the new Google G1 is also aiming to take a slice of this market. So far, however, these devices are targeted primarily at personal users, but the question is, “What role, if any, will these gadgets have in the business environment for applications such as mobile BI?

At the recent SAP TechEd, I had the opportunity to address this topic with John Schwartz, CEO of Business Objects. I made the point that mobile computing was enjoying more success in Europe than the US, and asked him if he thought the Apple iPhone would change this. He agreed that mobile computing use was higher in Europe, but felt the majority of this was still for personal purposes. He noted that mobile BI was seeing growth in both the US and Europe. However, he said the device of choice for mobile BI was still the Blackberry, because its architecture was more suited to this type of processing. So far, he said, Business Objects has seen little demand for iPhone support.

Donald Cambell, CTO of Cognos, took the same position when I asked him this question at a recent IBM Cognos analyst meeting. He said IT is still primarily supplying Backberry’s for business use, but in some cases IT will support personally purchased iPhones. He said the iPhone still doesn’t have the promised capability to run processes in the background, which limits its use for BI. He noted that the new Google G1 has an excellent development platform, and if the device is successful it could have a major impact on mobile business applications. He also made the interesting point that Windows Mobile was the preferred mobile platform for packaged embedded solutions used in locations such as hospitals, for example.

I would be interested to hear from other people about how their organization is deploying mobile BI solutions.