FAQ for 2008

November 17, 2008

What kind of research is this?

This is a study, not (just) a report. We captured data, constructed a model, tested the model against additional data, and wrote a report about the findings. This is very ifferent from merely collecting data points, analyzing them, and writing a eport, because the model allows us to create “what-if” scenarios and projections and an be modified on an ongoing basis as we’re provided with increasingly accurate data. he model reflects the best available data provided by independent third parties and ublicly-available documents (such as financial statements from service providers and quipment vendors), but also includes the ability to be modified as new data emerges hat may affect the outcome.

Why is this study unique?

Nemertes model is unique in that it’s the only one that looks at both user demand and capacity, and assesses them independently. More importantly, it is the only study that looks at user demand prior to the point at which that demand is translated into internet traffic.

There are a few reputable studies that assess actual Internet traffic by looking at data that travels across the “core” of the Internet or across peering points (most typically, Odlyszko’s MINTS research). These are meaningful and useful (in fact, we validated our model against this data to verify its accuracy) but they suffer from two fundamental limitations: First, as discussed in Nemertes Internet Singularity Delayed: Why Limits in Internet Capacity will Stifle Innovation on the Web 2007 report, they measure only the traffic that has been able to make it onto the Internet in the first place. If, as our research indicates, that traffic is constrained by access limitations and throttled back before even reaching the core, then information about what’s happening on the core doesn’t reflect what users are actually trying to do. In fact, a slowdown in core traffic could reflect anything from a true decrease in user demand to a situation in which the access is constraining the core—either could be the case. Second, as discussed in Nemertes Internet Interrupted: Why Architectural Limitations will Fracture the ‘Net 2008 report, traffic is continually moving away from the public peering points to private and overlay networks and this traffic is not “seen” when only looking at public peering points.

What does this study say will occur?

We predict that if current trends continue, demand will continue to grow to a point at which it outpaces capacity at the access layer. Globally, this will occur in approximately 2011/2012, and dependent on how the current economic state affects provider build out and user demand. In North America, the situation is slightly worse than last year’s projections, in fact, since last year, indications are that sophisticated users already are experiencing capacity limitations. But it’s important to understand that we’re not saying this will occur. The model is an “if-then” model. We plug in assumptions and generate outcomes, but the outcomes depend on the assumptions. We validated the assumptions against the best available data to ensure that our model delivers reasonably correct outputs against known data (ie the past) but when it comes to the future, it does not say that anything “will” occur. If current assumptions and data are correct, the outcome will be such-and-such. Obviously, a lot can change between now and 2011. The recommended investment in access bandwidth could occur, thus heading off performance declines, for example. Or we could receive better data that we could use to plug into the model and refine its results.

We also assert that the Internet is rapidly running out of addresses: 85% of addresses have been allocated. The rapid growth of internet-enabled devices and machine-machine communications leads us to project address depletion will occur sometime before 2012. Further, requirements for multi-homing and ever-increasing mobility are placing added stress on the current Internet logical infrastructure. As with the physical limitation discussion, we are not saying this “will” occur. We are making our projections based on our analysis of historical trends and projections for future address requirements. And, we project that IPv6 – long believed to be the answer to IPv4 address depletion – is too little too late.

How does Nemertes fund its research?

Our model is that we have a base of clients who subscribe to our research and advisory services. Our clients include users, makers of, and investors in, technology. Specifically, they include the IT departments of Fortune 200 enterprise organizations, vendors and service providers, not-for profits, financial services firms/investors, and a couple of publications. We base our insights on best practices which we uncover while conducting our benchmarks (for more on the benchmarking process, read here: What We Do [1]). Clients pay to have access to our data, our insights, and us--our data comes from the research projects that we choose to undertake, based on our own best judgment on what makes sense. Again, the cost is shared across a portfolio of players who have diverging agendas, a range of interest levels in any individual topic, and most importantly do NOT have line-item veto (or rights of approval) on our topic selection, process, methodology, or findings. All of our clients have exceedingly different agendas, which generally conflict with each others'. Our role is not to serve any one agenda, but rather to provide objective data that can be used by all.

Is this a commissioned study?

No. Commissioned studies are works for hire in which the commissioning agent owns the data and controls the distribution of the findings. Nemertes retains all rights to the study and, as with all its research findings, distributes the research to all its clients.

What were the goals of the research? Did you have any preconceived expectations?

Nemertes set out with two goals in 2008: to update our analysis on Internet infrastructure, conducted in the Fall of 2007 and to assess the potential of logical Internet architectural limitations. We did not have any preconceived expectations.

Did you model the study to obtain a desired outcome?

No. Nemertes did not set out to prove an agenda with this project and the analysis is explicitly agnostic with respect to politics and economics. There was no agenda or presumed outcome behind the research.

Why Didn’t You Discuss Investments This Year? (2008)

Last year (2007), we projected that $137 Billion may need to be spent globally of which $42 to $55 Billion would be for North America to close the gap between demand and supply. These numbers were only for Capital Expense (CapEx) and were developed by calculating a dollars per petabyte rate for access lines based on historical and projected costs. Since the demand-supply picture is largely unchanged, the investment required should also be unchanged. There have been no breakthrough technologies in the past year to dramatically drop the cost/PB for broadband access. For these reasons, we did not revisit investment requirements this year.

Now that you have completed the study, do you have any recommendations for changes in policy?

Setting policy is out of scope for us and our organization. However, as researchers, we came away with the overwhelming conviction that there is a deep industry need for better (more comprehensive and accurate) data in this area. The Internet is almost opaque to serious researchers, even those with the technical skills, integrity and desire, for the simple reason that carriers and content providers refuse to reveal their inner workings. This is often for very good reasons, but it’s detrimental to the industry. So we urge content and service providers to cooperate with researchers in sharing data in a way that can benefit all.

Additionally, public policy has been concerned with the issues associated with broadband access. As this study discloses, there are more issues associated with the logical structure of the Internet that have been largely overlooked by public policy makers. Since things like address exhaust have the potential to trump any public policy plans for access, it seems desirable for public policy makers to include the
logical Internet in their deliberations.

Where did the data come from?

We relied on a handful of key sources for our data: 

* research data collected by academic organizations such as CAIDA [2] and MINTS [3] 

* publicly-available documents, including vendor/service provider financials 

* confidential interviews with enterprise organizations, equipment vendors, service
providers, and investment companies. During the course of this project (2007 and 2008) we spoke with 70+ individuals and organizations for these interviews, and we relied on our base of several hundred IT executives who participate in our enterprise benchmarks.

Who conducted the research?

The study and data are the work of Nemertes analysts. Nemertes has the sole rights to the study and can license distribution of the report to interested clients should they desire.

How did you create the model?

To assess infrastructure capacity, we reviewed details of carrier expenditures and vendor revenues, and compared these against market research studies. To compute demand, we took a unique approach: Instead of directly modeling user behavior based on measuring the application portfolios that users had currently deployed, and projecting deployment of those applications in future, we looked directly at how user consumption of available bandwidth has changed over time. To assess IPv4 address exhaustion we plotted historical address assignment and then projected growth forward, based on our assessment of increasing demand for addresses over time.

How did you validate against the data?

We anchored the model at a few points in time for which we have solid (or relatively solid) data. We then “predicted” what the other data points should have been, and where the data and model disagreed, we modified the model.

A linear growth rate for access capacity seems low. Why is that? Did you consider the growth in wireless access?

We did consider the growth in wireless access as currently projected. The challenge with access capacity is that it's gated not only by technology, but also by the cost and effort of installing cabling, cell towers, etc. That's what results in a linear, rather than exponential, growth.

Won't technology fix the problem? Moore's Law appears to apply to switching and routing capacity--why not to access capacity?

See above. Moore's Law does indeed appear to apply to switching and routing capacity, but access capacity is gated by the physical deployment of media (wireless, wired, fiber, etc.) And, according to the best available data, access capacity has been growing linearly, not exponentially, over the past 7 years. Although it's not impossible that we'll see an exponential growth rate in access capacity at some point in the future, the data strongly suggests that we are not seeing exponential growth now. So, no, technology alone won't fix the problem.

You say that the IPv4 address pool will be exhausted in 2012. What does this mean?

We based our projections of IPv4 address exhaustion on historical assignment of addresses. We are saying that assignment of all IPv4 addresses to service providers will be complete by 2012. This does not mean that all addresses will be in use, but it does mean that all addresses will be held by private entities. We expect that some sort of address re-use scheme will be put in place to recover and re-assign addresses.

Wouldn't re-use solve the problem of IPv4 address exhaustion?

It is a stop-gap at best, and has serious side-effects. Once addresses start getting re-used, address blocks will fragment, leading to less efficient routing. This will slow down routing and could have a negative impact on Internet service delivery.

Will address translation help?

Network Address Translation (NAT) could help conserve addressers, but, unlike today’s NAT which occurs at the edges of the network, we will need carrier-grade NAT in the core of the Internet, something that isn’t available today. Even if it does become available in the next few years, carrier-grade NAT is likely to introduce translation delays and will significantly impact certain services like enterprise VPNs, perhaps to the point of making them nonfunctional.

Would switching to IPv6 solve the problem?

Yes and no. If it were possible to do a global flash-cut to IPv6 and wake-up one morning in an all-v6 world, then yes it can solve the problem. However, a flash-cut is not possible and since IPv6 and IPv4 addresses are incompatible, all plausible transition strategies greatly exacerbate routing problems by dramatically increasing the size of routing tables.

What can be done about address exhaustion?

At this point, not much. It is likely that some combination of re-use, carrier-grade NAT, and router improvements will keep the Internet going, but performance is likely to degrade beginning in 2012. There are things that businesses and service providers can do to avoid many of the problems, but consumers are likely to suffer slower performance for routine Internet use. Ultimately, Nemertes believes that public policy makers will need to get involved to find ways to promote the development of systemic solutions.