Oops! The input is malformed! Why Do We Need Data Warehouse Appliances? Part 1 by Krish Krishnan - BeyeNETWORK UK


Why Do We Need Data Warehouse Appliances? Part 1

Originally published 4 October 2007

Recently, there has been a flurry of information about data warehouse appliances and their benefits. The information presented in articles and white papers has repeatedly shown the benefits of this technology, including:

  • Lower cost – initial and ongoing

  • Higher sustained performance

  • Scalability

  • Ease of use

Putting all of this aside, the hard question to ask is this: Why do we need data warehouse appliances? Before we answer this, let us look at the overall cost of a data warehouse. There are several critical components that can be broadly classified into the following:

  • Initial cost: server hardware, licenses, development, deployment

  • Ongoing cost: maintenance, upgrades, hardware and storage additions

  • Data cost: initial data volume, incremental transactional volume, mergers and acquisitions

  • Business cost: data requirements – granularity, near real time, historical data retention

(Author's note: Business intelligence tools and their costs are not included in this discussion.)

There are costs that are necessary and cannot be controlled, such as:

  • Initial cost of hardware

  • Initial cost of development and deployment

  • Initial cost of software licenses

  • Ongoing cost for upgrades and maintenance

  • Ongoing development cost

And there are costs that can be controlled, such as:

  • Ongoing storage expansion

  • Ongoing server expansion

One might be tempted to ask: If we conclude that just by adding a data warehouse appliance, we are magically saving dollars and reducing cost, is this justification enough to adapt to a new technology and bring it into the data warehouse ecosystem?

Figure 1 is a sample of costs in the data warehouse; the comparison is spend per dollar over a period of time.

Figure 1: Initial Cost of Data Warehouse

From this graph, you can see that we are constantly in the process of spending money in the first year of the data warehouse build, which is normal for any new IT initiative. But as we progress through the timeline, we see the consistent increase in spend for storage and services. We can argue that as the data warehouse maturity happens, it is natural to spend more on storage and services. This should be an anticipated expenditure.


Figure 2: Data Warehouse Maturity – User Growth



Figure 3: Data Warehouse Maturity – Data Growth




Figure 4: Data Warehouse Maturity – Response Time

As shown in Figures 2, 3 and 4, we can see that with increase of adoption, performance and availability become issues. To mitigate these issues on the current implementation, there is the requirement for more investment into the hardware, storage and software areas. This provides interim relief and is not a long-term solution. This is exactly where the spending cycle starts.

But, wait a minute! Where is the cost coming from? Disk is cheap and processor is cheap. It is only the need for additional license fees for software that is causing the need for additional spending, correct? Actually, not quite correct, and that will be covered in the next article in this series.

Part 2 of this article will continue our look at why there is a need for data warehouse appliances and will examine data warehouse costs from an infrastructure perspective.

SOURCE: Why Do We Need Data Warehouse Appliances? Part 1

  • Krish KrishnanKrish Krishnan
    Krish Krishnan is a worldwide-recognized expert in the strategy, architecture, and implementation of high-performance data warehousing solutions and big data. He is a visionary data warehouse thought leader and is ranked as one of the top data warehouse consultants in the world. As an independent analyst, Krish regularly speaks at leading industry conferences and user groups. He has written prolifically in trade publications and eBooks, contributing over 150 articles, viewpoints, and case studies on big data, business intelligence, data warehousing, data warehouse appliances, and high-performance architectures. He co-authored Building the Unstructured Data Warehouse with Bill Inmon in 2011, and Morgan Kaufmann will publish his first independent writing project, Data Warehousing in the Age of Big Data, in August 2013.

    With over 21 years of professional experience, Krish has solved complex solution architecture problems for global Fortune 1000 clients, and has designed and tuned some of the world’s largest data warehouses and business intelligence platforms. He is currently promoting the next generation of data warehousing, focusing on big data, semantic technologies, crowdsourcing, analytics, and platform engineering.

    Krish is the president of Sixth Sense Advisors Inc., a Chicago-based company providing independent analyst, management consulting, strategy and innovation advisory and technology consulting services in big data, data warehousing, and business intelligence. He serves as a technology advisor to several companies, and is actively sought after by investors to assess startup companies in data management and associated emerging technology areas. He publishes with the BeyeNETWORK.com where he leads the Data Warehouse Appliances and Architecture Expert Channel.

    Editor's Note: More articles and resources are available in Krish's BeyeNETWORK Expert Channel. Be sure to visit today!

Recent articles by Krish Krishnan



Want to post a comment? Login or become a member today!

Be the first to comment!