Information Control Corporation Hits Gold with Agile Data Warehousing Method

Originally published 25 March 2010

Information Control Corporation (ICC), Ohio's largest privately-owned IT services company, recently doubled its development speed and nearly eliminated programming defects for its business intelligence services division by converting its data warehousing operations to a Scrum-based methodology, as outlined in Ceregenic's book Agile Data Warehousing.

Until late 2008, ICC ran a very traditional DW/BI shop with a staff of 60, following an industry-standard waterfall method, where a project is extensively planned with fully-specified requirements followed by detailed designs before coding begins. That method was “highly manual, paper-based, slow and error prone,” says Jim Gallo, ICC's Manager of Partner Channels. “Industry articles and trade organizations are rife with failure statistics for such waterfall projects ranging from 50 to nearly 70 percent.”

ICC anticipated the recent economic slowdown and initiated a program to remake their methodology to provide DW/BI services that could compete with off shore firms. While other computer services companies shrank in size substantially or went out of business altogether as clients cut funding for IT projects, ICC's BI revenue grew an impressive 48% in 2009. Gallo noted, “We understood that in order to be successful in a global marketplace we had to figure out a way to compete with off shore firms who were offering very attractive hourly rates. We wanted to figure out a way to drive down client costs and dramatically shorten their time to value.”

So ICC assembled its leadership team to rethink the DW/BI delivery process. “We literally set out to figure out how to bill fewer hours on every project,” says Gallo. They based a lot of their new thinking on Ceregenics' recent book, “Agile Data Warehousing.” “We read it and re-read it, discussing the contents and how to adapt the ideas to create our own methodology.”
They arrived at a process built upon small, collocated work “pods” which emphasizes developers working eye-to-eye in time-boxed development sprints. The method's other important innovations include streamlined requirements management, rapid prototyping, test-led development, several “PM lite” techniques, and the incorporation of some novel tools from innovative vendors like Balanced Insight.

ICC's first proof-of-concept trials for the new method and initial projects for clients went spectacularly well, achieving efficiency gains of 30 to 60 percent over the waterfall method. Perhaps the best part of the new approach is Agile Data Warehousing's developer-led, continuous improvement process which is performed at the end of each cycle or every week. “We have seen velocity improvements of up to 60% between time-box iterations,” says Gallo.

As if cutting in half the time-to-value for their customers was not enough, Agile's innovations regarding quality assurance have also paid off handsomely. “We have defect rates approaching zero percent,” says Don Jackson, ICC's Director of Business Intelligence. “The collective results have been so outstanding that Agile BI methods are now part and parcel to every project we do. We have every confidence that our Agile method can be applied to very large enterprise solutions and have started pursuing with it projects that may go as long as 1 to 2 years.”

“The ICC managers and staff have been very innovative with the basic method presented in our book,” says Ralph Hughes, Ceregenics' Chief Systems Architect and lead author of "Agile Data Warehousing." “For example, I liked their notion that there are separate value chains for the customer and the developers,” Hughes said after a visit to ICC's offices this past January. “It has prompted my team at Ceregenics to now discuss the difference between user stories and developers' stories. We've folded that into a larger Agile requirements management framework that we'll be detailing in the next volume of our Agile Data Warehousing book.”

SOURCE: Information Control Corporation Hits Gold with Agile Data Warehousing Method