Table of Contents
1 EXECUTIVE SUMMARY 4
2 THE NEW DATA CENTER 5
2.1 OVERVIEW 5
2.2 KEY THEMES 5
3 TRENDS IN NETWORK ARCHITECTURE 8
3.1 THE IMPACT OF DATA CENTER CONSOLIDATION ON WAN ARCHITECTURE 8
3.2 DATA CENTER INTERCONNECTS 10
3.3 DATA CENTER BANDWIDTH: HOW MUCH IS ENOUGH? 12
3.4 MPLS AND THE DATA CENTER 14
3.5 WAN REDUNDANCY STRATEGIES 20
3.6 THE ROLE OF THE INTERNET 22
3.7 NETWORKING INSIDE THE DATA CENTER: STORAGE NETWORKING 25
3.8 NETWORKING INSIDE THE DATA CENTER: SERVER INTERCONNECTS 28
4 CONCLUSIONS AND RECOMMENDATIONS 32
5 APPENDIX A - BENCHMARK METHODOLOGY 35
5.1 PROCESS 35
5.2 BY INDUSTRY 36
5.3 BY SIZE 36
5.4 PARTICIPANTS 39
5.5 TIMEFRAME 40
5.6 DISCLAIMER 40

Table of Figures
FIGURE 1: WAN ARCHITECTURE FROM 1995-2002 9
FIGURE 2: WAN ARCHITECTURE FROM 2002-2010+ 10
FIGURE 3: DATA CENTER WAN TECHNOLOGIES IN USE 11
FIGURE 4: DATA CENTER-TO-BRANCH OFFICE CONNECTIVITY 12
FIGURE 5: DATA CENTER BANDWIDTH BY VERTICAL 13
FIGURE 6: MPLS WAN ADOPTION 15
FIGURE 7: “AGGRESSIVE-BLEEDING EDGE” DEPLOYMENT OF MPLS 16
FIGURE 8: “CONSERVATIVE/MODERATE” DEPLOYMENT OF MPLS 17
FIGURE 9: DEPLOYMENT OF MPLS AMONG COMPANIES WITH GLOBAL OPERATIONS 18
FIGURE 10: DEPLOYMENT OF MPLS AMONG COMPANIES LACKING GLOBAL OPERATIONS 18
FIGURE 11: DEPLOYMENT OF MPLS AMONG COMPANIES OVER $1 BILLION 19
FIGURE 12: DEPLOYMENT OF MPLS AMONG COMPANIES UNDER $1 BILLION 19
FIGURE 13: IS YOUR DATA CENTER MULTI-HOMED? 21
FIGURE 14: INTERNET CONNNECTIVITY AND THE DATA CENTER 23
FIGURE 15: RUNNING BGP 24
FIGURE 16: OUTSOURCED ROUTE OPTIMIZATION 25
FIGURE 17: STORAGE NETWORKING 26
FIGURE 18: SERVER INTERCONNECTIONS 28
FIGURE 19: TYPE OF COMPANY 36
FIGURE 20: ANNUAL REVENUE 37
FIGURE 21: IT BUDGET 38
FIGURE 22: NUMBER OF EMPLOYEES 39
FIGURE 23: PARTICIPANT TITLES 40

1 Executive Summary

Data centers have been around since the dawn of computing. But just because they’ve been around forever doesn’t mean they’ve endured unchanged. In fact, we’re smack in the middle of a radical shift in data-center architecture, design and operations.
Nemertes’ groundbreaking benchmark, “The New Data Center 2006,” details these changes and their impacts, highlights critical issues for IT executives, and provides best practices and success strategies for organizations seeking to deploy and leverage this new data center.
In Volume 1, we examine the overall state of the data center, uncover the four major themes of consolidation, explosive growth, always-on availability and operational efficiency/automation.
In Volumes 2 through 7, we look at the six disciplines of the data center and the major trends in each:
* Computing – Major trends such as virtualization and dense computing.
* Storage – Extreme growth in storage demand and storage virtualization.
* Networking – Optical interconnects, high-speed server interconnects, and more reliance on the WAN to deliver applications.
* Facilities – Power and cooling for dense computing and storage.
* Management – Business service management, Web services, ITIL and CMDB.
* Security – Re-trenched perimeter, mobility and identity.
Finally, in Volume 8, we examine best practices, strategies for success and Nemertes recommendations.

2 The New Data Center

2.1 Overview

The major themes and shifts of the new data center contained within are based on detailed discussions with 82 IT executives from 65 companies, spanning various industries and company sizes (For further details, please see Appendix A - Benchmark Methodology starting on page 35). The average revenue of the participants is $5 billion, and the average IT budget is $145.5 million. Participant organizations had, on average, 24,000 employees in 385 locations. To serve these employees, organizations operated an average of eight data centers.

Some historical perspective: In the early 1990s and again in the early ‘00s, data-center technology underwent some significant transformations. In the ‘90s, the Internet spawned the need for new data centers that handled e-commerce and Web-based applications. Sprawling commodity servers, either “white box” systems or 1U rack-mountable servers, dominated the data center of the early Internet era. As servers proliferated, so did capacity problems for cooling and power systems. A second important transition in the late ‘90s, and continuing today, has been the development of more and more dense computing architectures.

As a result, many companies found that their existing power and cooling systems could not support the next generation of server technology—so unsurprisingly, our research participants cite facilities upgrades as a top-funded initiative for 2006.

Consolidation (both of discrete sites and of resources within a data center site) has served to exacerbate the impact on facilities design of these architectural shifts, while also leading to major changes in WAN architecture and backup strategies. (As we’ll see, the majority of applications and data are now delivered across the WAN—which makes WAN bandwidth, availability and management more critical than ever before.) And, finally, next-generation technologies like virtualization and clustering dramatically affect information processing and storage within data centers.
The result? Data centers today are truly “new,” from every aspect: Facilities, storage, management, computing, networking—and in how they affect the delivery of services and applications.

2.2 Key Themes

Our research uncovered four major themes in data centers:

* Consolidation – Whether due to regulatory compliance or the increasing cost and complexity of IT, companies from all industries and all sizes are consolidating and centralizing IT resources, including data centers. Specifically, IT managers are consolidating dozens of data centers into a few central locations—then enhancing the capacity of those few across all areas: networking, redundancy, computing, storage and management. The low-end of the consolidation trend is either two data centers, for redundancy, or up to eight data centers organized in redundant pairs in each of the geographical regions (eg. Two in North America, two in South America, two in Europe, two in Asia Pacific for a total of eight). The consolidation trend has some significant corollary impacts. For one thing, data and applications are increasingly delivered to remote users over the WAN (as our research shows, as few as 11% of workers work at headquarters and the number of branch offices is growing by 7.9% annually), which has major impacts on WAN architecture and management. For another thing, centralized data is more easily backed-up, made redundant and controlled for compliance, which is sparking major initiatives in all these areas. Finally, centralized applications, delivered via the Web, client-server protocols or thin clients, are becoming the norm, as they are easier to manage, update and patch against security exposures.

* Growth – Demand for data-center resources such as servers and storage has generated tremendous growth. The average number of servers across all data centers is growing 11% per year on average, while total storage capacity (in terabytes) is growing at a median rate of 22%. Rapid growth is causing strain in data centers, with insufficient capacity in environmental controls (cooling, humidity controls) power supplies, and floor space. Companies struggle to balance between large footprint (sprawl of low-density servers) or high density (super-hot and power-hungry, dense racks).

* Availability – Business demand for higher availability is increasing across all industries. Half of all participants list availability as one of the top challenges. Correspondingly, almost half of the research participants are setting up secondary data centers for continuity and disaster recovery, either by repurposing an existing data center, or by building new facilities in which they consolidate existing data centers. After Hurricane Katrina, disaster-recovery consultants and managers have re-evaluated traditional assumptions about the potential geographic footprint of a disaster. As a result, secondary and tertiary data centers are often built more than 200 miles from the primary data center. Before hurricane Katrina, many organizations assumed that 10-30 miles separation would be sufficient. After the Katrina disaster it became obvious that some natural disasters can have very wide footprints, even if only through secondary effects (road damage, electrical outage etc.)

* Operational Efficiency – Organizations are focusing on data-center management and operations, attempting to cut costs, improve efficiency, automate repetitive tasks, and better align IT spending with business needs and demand for services. More than three-quarters of respondents take a holistic view of the data center as part of a broader shift in IT culture to “IT as a service.” This translates to data centers as business-service delivery centers. The emphasis on business services extends to broad adoption (more than 50%) of governance and service delivery standards such as IT Infrastructure Library (ITIL®) and Control Objectives for Information and related Technology (COBIT®) . Not all companies are implementing these standards formally and in their totality, but even partial implementations can bring benefits in change management practices and, as a result, in higher availability and lower operational costs.

3 Trends in Network Architecture

3.1 The Impact of Data Center Consolidation on WAN Architecture

Data center consolidation and concomitant branch office proliferation has had a major impact on WAN architectures. Five or more years ago, the stereotypical WAN architecture was highly hierarchical. The top tier consisted of large data centers and administrative offices, interconnected by high-bandwidth services such as ATM or private fiber networks. Below that was a middle tier of midsized offices interconnected by a “mid-speed” service such as frame relay. Remote offices and users were linked in via low-bandwidth, high-latency interconnections such as dial-up or VSAT services (please see Figure 1, “WAN Architectures From 1995-2002”, page 9).
As data centers have consolidated, companies have moved to a simpler structure: Data centers are interconnected via a high-speed network such as dense wave-division multiplexing (DWDM) over dark fiber or multi-protocol label switching (MPLS). Data centers are then linked to every other site in the network via high-speed WAN technologies, typically MPLS. (Please see Figure 2, “WAN Architectures 2002-2010+”, page 10).

Why? MPLS offers increased scalability, any-to-any (mesh) connectivity, and the promise of combining voice and data traffic over a common communications framework. Plus, it’s significantly cheaper: “[We have] an MPLS core [with a] mesh topology. We expect to save about $40m in 3 years from that transition to mesh,” says the IT director for a $10 billion organization, who adds, “MPLS is so much cheaper that every site has increased their bandwidth as they migrated from frame to MPLS.”

[...]

Complete benchmarks are available to clients only. If you're not a client and would like to receive a copy of the complete benchmark, please contact us.