© 2005 Steve Corbato and Steve Cotter
EDUCAUSE Review, vol. 40, no. 3 (May/June 2005): 68–69.
Spurred in part by opportunities emerging from the recent downturn in the telecommunications sector, the U.S. research and education (R&E) community is aggressively pursuing a revolutionary new means for delivering advanced networking capabilities. Dark fiber refers to those fiber-optic cables that have already been deployed either in underground conduits or on aerial utility transmission facilities but that have not yet been "lit"—that is, equipped with the optical electronics needed to transmit information over significant distances. The recent availability of these assets, the evolution of economical long-distance optical transmission technology, and the emergence of leading-edge Grid applications capable of generating multiple Gigabit-per-second (Gbps) flows are driving this community’s surge of interest in a domain that was once the exclusive province of the telecommunications carriers.
Higher education’s interest in dark fiber is not new, however. Over a decade ago, several leading research universities in major urban areas—including the University of Maryland, the University of Southern California, and the University of Washington—began to acquire segments of metropolitan dark fiber to connect affiliated medical centers and other remote locations and also to tie their campus networks directly into downtown buildings housing the major telecommunications carriers. A natural outgrowth of these efforts was the extension of fiber-based optical connectivity from the major cities to more remote campuses (e.g., land-grant institutions). In 1999, Larry Smarr (then director of the NCSA supercomputing center) initiated the first such effort, I-WIRE, to connect the Urbana-Champaign campus of the University of Illinois to the major academic, research, and telecommunications facilities in the Chicago area 140 miles to the north.
Concurrently in the 1990s, the anticipated explosion of bandwidth-hungry commercial applications, the ready availability of capital funding, and the fundamental economics of fiber deployment spurred major telecommunications companies to lay vast amounts of dark fiber between the largest cities in the country. When a carrier was going to the significant expense of provisioning a single-fiber pair (note that optical transmission typically requires a two-fiber strand to support bidirectional data flow), it was only marginally more expensive to deploy fifty or even one hundred such pairs at the same time. Unfortunately for many of these companies, the demand for consumer-driven bandwidth did not materialize as quickly as hoped. The wave of subsequent bankruptcies and dramatic changes in the remaining carriers’ business plans led to the emergence of a more open marketplace for intercity dark fiber. Thus, for the first time, high-end customers, such as those within the higher education and financial sectors, could obtain this vital raw material for constructing advanced optical networks.
Moreover, the ongoing development of the optical transmission technology known as dense wave division multiplexing (DWDM) allowed a single-fiber pair to carry many more channels of information in parallel (in bandwidth increments of 10 Gbps!). This development increased the capacity of the new fiber-optic infrastructure geometrically, with a single-fiber pair now capable of carrying well over a Terabit per second of aggregate data.
Finally, and perhaps most important, individual applications are emerging that can efficiently consume multiple Gbps of bandwidth. Campus networks, high-performance computing clusters, and even individual workstations are increasingly moving to the Gigabit Ethernet local area network standard, and the follow-on Ethernet technology (operating at 10 Gbps) is already available. In this new network environment, various academic disciplines—including high-energy physics, radiology, atmospheric sciences, and the performing arts—are exploring applications such as Grid computing, massive file transfers, and High Definition videoconferencing.
Yet the most revolutionary aspect of dark fiber networking is not so much the breadth of new technical possibilities as it is the opportunity for higher education to own the underlying assets and thus to deploy new technologies and services as needed. The shift in the underlying business model is analogous to moving from renting an apartment to owning a condominium. Whereas ownership brings a new level of responsibility (e.g., ongoing maintenance), the flexibility and independence of ownership provide a significant long-term advantage for higher education.
The momentum toward developing facilities-based optical networks is most pronounced at the state and regional levels. Over twenty such efforts either are already in production or are under active development. The significance of this movement is underscored by a new name for these efforts: regional optical networks, or RONs. With early leadership from such networks as CalREN in California, I-Light in Indiana, and the Third Frontier Network in Ohio, these efforts are increasingly viewed as essential to supporting the many collaborations spanning the research university community and thus as vital for maintaining research competitiveness. In many cases, the urgency for RON development is tied to the fact that many of the carriers did not lay dark fiber in the same quantities to the smaller cities and towns even during the Internet boom in the late 1990s. In particular, one recently funded RON—the Louisiana Optical Network Initiative (LONI)—is making significant strides toward the development of an integrated system of distributed high-performance computing and optical networking capabilities across that state.1
As RONs began to emerge across the country, many had trouble obtaining consistent pricing and acquisition terms for dark fiber. As a result, after a discussion with Ron Johnson (of the University of Washington) and Tom West (then at CENIC), Internet2 established FiberCo (http://www.fiberco.org/) as a nonprofit holding company for dark fiber assets. Operating on behalf of the Internet2 membership and the emerging set of RONs, FiberCo acquired an initial allocation of over 2,500 route-miles of dark fiber from Level 3 Communications, a major wholesaler of telecommunications services, in early 2003. FiberCo retains the ability to procure additional fiber segments from Level 3 under the same terms through the spring of 2006. The model behind FiberCo is relatively simple: its objective is not to light the fiber itself or even to hold the dark fiber assets indefinitely. Instead, FiberCo acts as a facilitating third party in assigning the needed dark fiber assets directly to those groups actually developing the RONs.
As of early 2005, FiberCo had made six fiber assignments to emerging RONs—as well as placing fiber with National LambdaRail (NLR) and Internet2. Viewed together, these transactions account for over 5,600 route-miles of dark fiber. FiberCo’s approach has evolved to act as both a planning consultant and a market maker in regional and metropolitan fiber procurements by the research university community. Accordingly, FiberCo has developed strong relationships with other carriers now active in this market, such as WilTel Communications.
Two examples drawn from recent FiberCo assignments reveal the variety of needs being addressed by the development of RONs:
- The University of Wyoming, located in a city with a population of just 27,000, has long struggled to obtain economical, high-performance telecommunications services for delivering advanced networking to campus researchers. Due to the fortuitous routing of a major transcontinental fiber pathway, CIO Robert Aylward and his colleagues were able to procure a fiber pair into Denver and subsequently deployed a 180-mile-long optical network to connect to the Front Range Gigapop. As a result, the University of Wyoming now benefits from the same level of access to advanced networks, such as Abilene and NLR, as do its collaborating institutions located in the Denver/Boulder metropolitan area.
- Florida LambdaRail, or FLR (http://www.flrnet.org/), is a consortium of, initially, ten higher education institutions collaborating to build a unified, statewide optical network to serve R&E in Florida. The state’s geography presents a daunting distance challenge, further exacerbated by the fact that until recently, major national R&E networks did not establish significant presences within the state. As the footprint for the statewide RON that will stretch from Pensacola in the western Panhandle to Miami in the south, FLR recently acquired over 1,500 route-miles of dark fiber through FiberCo and is moving aggressively to light this network.
At the same time, several other significant efforts are under way to facilitate RON development around the country. The Southeastern Universities Research Association (SURA) has received a donation of 8,000 route-miles of next-generation dark fiber from AT&T. SURA is now working with its members and several RONs to efficiently allocate these assets. In addition, a very active working group within the Quilt Project, the collaboration of the twenty largest U.S. advanced regional networks, is sharing best practices for RON development and management and also is investigating the options for lighting fiber assets.
All told, the U.S. R&E community in recent years has acquired over 27,000 route-miles of intercity dark fiber, with the vast majority of these assets now devoted to regional efforts. This community’s shift to asset ownership has triggered new types of relationships with telecommunications carriers—relationships based more on wholesale principles such as supporting services and direct fiber interconnectivity. This unprecedented access to dark fiber has been critical to developing a national optical networking facility—NLR—as well as to enabling investigations of next-generation Internet architectures, including the Internet2 Hybrid Optical and Packet Infrastructure (HOPI) project. In aggregate, these newly acquired assets will form the basis for the Internet2 community’s advanced networking efforts over the next decade. However, with the inevitable economic rebound and further consolidation within the telecommunications industry, we do not expect that our community will be able to acquire this critical raw material indefinitely. Thus, for the U.S. research universities and their collaborators, time truly is of the essence in developing facilities-based, regional optical networking capabilities.
1. Notably, this type of optical network is now being developed to serve R&E around the globe: whereas networks in Ontario and Quebec, as well as the Netherlands, have led the way, significant national-scale efforts are also under way in developing economies such as Poland and the Czech Republic.