Rube Goldberg variations for data center management

How data center monitoring software can resemble Rube Goldberg contraptions

New Data Center Strategies Newsletter By John Burke, Network World, 09/05/06

Ridiculously complicated mechanical systems for accomplishing straightforward tasks are often referred to as Rube Goldberg machines. They’re funny in cartoons, less so in real life.

Unfortunately, as Nemertes is discovering in our soon-to-be-published benchmark on Service Delivery and Management, many enterprises' application monitoring systems are beginning to resemble Rube Goldberg contraptions. For IT executives tasked with managing these applications, that’s no laughing matter.

The transformation of well-engineered systems into Rube Goldberg machines has happened gradually over the years, as applications have come less and less to correlate directly with a single process running in one place. Applications have moved into multiple tiers, with clustering in some places and load balancing across server farms in others (sometimes across separate data centers). Multiple virtual servers can be piled onto each physical server. Storage has moved out of servers onto networks of its own and has become a shared resource.

At the same time, applications more often rely on other applications (which are in turn multi-tiered, clustered, farmed, etc.) for critical services. The links of interdependence among applications don’t look like simple flow charts anymore - more like diagrams of complex ecosystems. Take away any of the pieces and the consequences can be surprisingly far-reaching; drop one server, and what seems like an entirely unrelated service may disappear.

That brings us to the crucial point: what, exactly, it is that the end user sees as an “IT service.” IT has in the past primarily made promises about - and built service-level agreements (SLA) around - the availability of individual servers or applications. But that has progressively less to do with what users actually need: SLAs around services as they see them, any one of which (like an HR self-service portal) may involve dozens of servers and applications and supporting systems like SANs.

In the face of this kind of organic complexity, trying to use traditional monitoring tools to tell you whether a service is available (let alone how well it is performing) is difficult or impractical. They were built with the older paradigm in mind. At the same time, service-based SLAs are on the rise as sensibilities like those embodied in the IT Infrastructure Library (ITIL) continue to diffuse through U.S. enterprises.

IT managers who don’t want to run Rube Goldberg management systems have three basic choices:

* Put aside minding what users care about and focus on what’s already been instrumented, as many organizations have done. This simplifies the problem, but at the expense of user requirements and IT staff time. Unfortunately, ignoring user concerns tends to backfire in the long run.

* Add user-perspective monitoring tools for measuring service performance and availability, and treat the entire infrastructure as a black box for SLA purposes. This is the simplest way to balance what IT’s current tools can do and what users really require.

* Look to newer tools that will discover, map, and monitor most of the complex interdependencies of the new data center on their own. Products available from companies like BMC and EMC can do this (or claim they can). This is the most strategic approach for the long run, although it requires the greatest change and currently relies on first- or second-release products for this most sophisticated level of function.

The bottom line: Imaginary Rube Goldberg contraptions are funny; real ones in the data center are not. If your management system looks like one, it’s time to revisit your strategy.