How the Data is Collected

The collected data is aggregated from seven Internet service providers serving roughly 50 percent of US residential broadband subscribers. Data is collected every 5 minutes across link aggregation groups in each region, depicting capacity and utilization per region, as well as fluctuation over time. All provided data and visualizations are of ingress capacity and utilization. CITP receives aggregated and anonymous data from each participating ISP, and will refresh the data set on a regular basis to provide a picture of how the Internet is evolving.

All ISPs have installed a common data collection platform from Deepfield Networks enabling comparability and aggregation. This approach also enables other ISPs and other network operators of all kinds to join the Interconnection Measurement Project. The Interconnect Measurement Project welcomes these parties to join and further extend this research effort. If interested in becoming a contributor, more information can be found on the Contribute Data page.

This data differs from complementary datasets that study network congestion. Whereas other techniques attempt to infer congestion properties based on measurements from Internet endpoints, this data set represents utilization information that is collected directly at the interconnection points. For a complete description of methodology, see Revealing Utilization at Internet Interconnection Points.

As expected from time to time, CITP experiences interruptions or degradations to the data collection and/or data feed associated with this project. This is not unusual or unexpected given the constantly evolving nature of the networks and infrastructure. To provide transparency into these issues, first, we provide in the data file the proportion of collective interconnection capacity included in each month’s analysis, which varies, depending on the integrity of the data received by CITP.  For instance, the November 2016 analysis includes just over 97 percent of the collective capacity for the paid peering, settlement-free peering, and ISP-paid transit interconnection links of the participating ISPs, which is typical of the proportion of collective interconnection capacity included in the monthly analysis.  Second, any significant data issues are described in a blog post associated with the specific update.  Most recently, in December 2016, CITP experienced a data outage that impaired CITP’s ability to conduct a fulsome analysis for that month and we’ve detailed that issue in a blog post, available here.

Participating ISPs

Bright House Networks
Comcast
Cox
Mediacom
Midco
Suddenlink
Time Warner Cable

Definitions

Interconnection / Interconnect

The physical link that connects an Internet Service Provider (ISP) to others in the Internet ecosystem.

Interconnection Capacity

The amount of data that can traverse an interconnection point per unit of time. Generally measured in bits per second or bps.

Interconnection Utilization

The amount of data that does traverse an interconnection point at one time. Also generally measured in bits per second or in percent of interconnection capacity. Consistent with industry practice, peak utilization is defined as the 95-percentile of utilization measurements.

ISP

An internet service provider offering end-user Internet access. Sometimes also referred to as an ‘access network provider’ or ‘last mile provider’.

Content or Edge Provider

A provider of applications or services that utilize the Internet.

Aggregated Link Aggregation Group (ALAG)

An aggregation of link aggregation groups that connect each unique ISP – interconnection partner pair in a given city or region. A link aggregation group is the aggregation of physical ports between two routers in a single location and there may be multiple link aggregation groups in a given region for each unique ISP – interconnection partner pair. Each ALAG is composed of a number of individual interconnection circuits.