You Can’t Handle the Single Source of Truth! (Yet)
An Enterprise Data Warehouse (EDW) is a central repository for an organization’s data. The EDW integrates an organization’s source systems under one data model from which the company can build, among other things, a Business Intelligence (BI) program.
The value of an EDW is substantial—done correctly, it provides a single source of truth for information a user would want to know about the business. When supported by strict Master Data Management (MDM), Data Governance, and Quality Assurance (QA) methodologies, data flowing from the EDW is consistent, secure, and readily available. Imagine being able to tie together sales forecasts in California with supply chain information for a plant in Texas, revenue data with employee overhead, or simply bringing together employees’ mailing information to distribute the company newsletter—the EDW seeks to put this analysis at a user’s fingertips.
As with many high value undertakings, pursuing an EDW is a high risk venture. The cost of fitting all major data sources into one model, developing Extract, Transform and Load (ETL) processes, and the hardware capable to collect and manage the data can be staggering. In addition, there are the less tangible costs: changing work streams that have been in place for years, gaining the buy-in of business units, and locking up resources for elongated periods of time during the delivery process can wear on an organization. Industry expert Les Barbusinski believes that “most of the fatal risks for a data warehouse project are organizational rather than technical.”¹ From Kenway’s experience, this statement rings true. To address these issues, we believe that a docket-based, business need driven, iterative project structure can help alleviate these issues and set organizations up to succeed.
The fundamental ideal behind an iterative approach to building an EDW is leveraging smaller projects with the enterprise model in mind. Think about it like you are building a desk. Obviously, you have the base of it: the tabletop, the legs, and let us throw in a drawer. You build these to directly contribute to the final product. Now along the way, you see a sale for a shelf you believe you could add to the desk. After making sure the shelves would fit and the desk could hold the weight, you go ahead and buy them with the intention of adding it to the base. After all this, you find a dart board that you absolutely love. It doesn’t have anything to do with the desk you just built (clearly you are heavily invested in your furniture) but you buy it anyway, with the understanding that it is completely separate from the base product. This same ideal can be applied to building an EDW by segmenting small docket projects into three categories:
This methodology addresses both the technical and organizational risks listed above. Technical risks are quelled because each project is now a separate effort. Smaller teams are able to engage in them which allows organizations to manage the ebb and flow of resource availability against project demands. The greater value, however, is in the organizational space. These projects are business driven, increasing the buy-in from the technology teams and business sponsors. Furthermore, their shorter duration allows end users to see value more frequently as opposed to having to wait the (potentially) years required in a full EDW endeavor. Finally, because the EDW initiative is broken into smaller lab projects as opposed to full enterprise engagements, it allows for a much safer funding model. Organizations no longer have to commit a large sum of money from the beginning to be used over time. They can wait to see if the business finds the results of each lab project to be meaningful, useful assets before committing to incorporating them to the full, enterprise environment. If any project is deemed a failure or was simply a one-time answer, it can be discontinued. All of this is accomplished faster and at a lower cost than had it been part of a large-scale, long-term initiative.
While in order to build a true EDW, an organization will have to eventually take on a prolonged undertaking—the hope is that after these small successes, they will have the tools and support to accelerate that process. Oates, Joe, et. al. “What are the Major Risk Factors in a Data Warehouse Project Implementation?” Information Management. 9 December 2003. Web. 13 December 2013.