Data warehousing should be an ideal workload for the System z. It already houses the production data that mostly populates the data warehouse. It can run Cognos on Linux on z for BI and with a zEnterprise (z196 and zBX) it can run the Smart Analytics Optimizer, either as a zBX blade or as an appliance. And do it all with scalability, reliability, and performance.
But IBM is moving beyond conventional data warehousing, which entails an enterprise data store surrounded by myriad special purpose data marts. Data warehousing as it is mainly practiced today in the distributed environment is too complex, difficult to deploy, requires too much tuning, and too inefficient when it comes to bringing in analytics, which delays delivering the answers business managers need. And without fast analytics, well, what’s the point? In addition, such data warehousing requires too many people to maintain and administer, which makes it too costly.
On top of these problems, the world of data has changed dramatically since organizations began building enterprise data warehouses. Now a data warehouse should accommodate new types of data and rapidly changing forms of data.
IBM’s recommendation: evolve the traditional enterprise data warehouse into what it calls the enterprise data hub. This will entail consolidating the infrastructure and reducing the data mart sprawl. It also will simplify analytics, mainly by deploying analytics appliances like IBM’s Netezza. Finally, organizations will need to data governance and lifecycle management, probably through automated policy-based controls. The result should be better information faster and delivered in a more flexible and cost-effective way.
Ultimately, IBM wants to see organizations evolve the enterprise data warehouse into an enterprise data hub with a variety of BI and analytics engines connected to it along with engines tuned for analyzing streamed data and vast amounts of unstructured data of the type Hadoop been shown to be particularly good at. DancingDinosaur wrote about Hadoop on the z196 back in November.
The payback from all of this, according to IBM, will be increased enterprise agility and faster deployment of analytics, which should result in increased business performance. The consolidated enterprise data warehouse also should lower TCO and speed time to value for both the data warehouse and analytics. All desirable things, no doubt, but for many organizations this will have require a gradual process and a significant investment in new tools and technologies, from appliance to analytics.
Case in point is Florida Hospital, Orlando, which deployed a z10 with DB2 10, which provides enhanced temporal data capabilities, with a primary goal of converting its 15 years of clinical patient data into an analytical data warehouse for use in leading edge medical and genetics research. DancingDinosaur referenced the hospital’s plans recently.
The hospital calls for getting the data up and running on DB2 10 this year and attaching the Smart Analytics Optimizer as an appliance connected to the z10 in Q1 2012. Then it can begin cranking up the research analytics. Top management has bought into this plan for now, but a lot can change in the next year, the earliest the first fruits of the hospital’s z-based analytical medical data exploration are likely to hit.
IBM does not envision the enterprise data hub exclusively as a System z effort. To the contrary its Power platform is as likely to be the preferred platform as any. Still, a zEnterprise loaded with Smart Analytics Optimizer blades might make a pretty good choice too. Florida Hospital probably would have gone with the z196 if it had known the machine was coming when it was upgrading from the z9 to z10.
The point here: existing data warehouses probably are obsolete. In a recent IBM study, half the business managers complained that they don’t have the information they need to do their jobs and 60% of CEOs admitted they need to do a better job of capturing and understanding information rapidly in order to make swift business decisions. That should be a signal to evolve your existing data warehouse into an enterprise data hub now and the z you have sitting there is just the vehicle for doing that.