Today, CITP provided the third quarterly update to the IMP website, in furtherance of the project’s goal of providing ongoing insights into the evolution of interconnection capacity and utilization. The site now includes monthly data from October 2015 through and including December 2016. Throughout this period, aggregate peak utilization remains below 50 percent and total interconnect capacity continues to grow, outpacing growth in peak usage.
CITP, working closely with CableLabs, Deepfield, and the participating ISPs, continues to monitor the IMP data feed to ensure quality and integrity of the utilization and capacity measurements. As background, Deepfield provides a cloud-based common measurement platform that has been installed by each of the participating ISPs. Deepfield collects 5-minute measures of utilization and capacity from each border router for each of the participating ISPs. Deepfield in turn processes and aggregates this data to provide CITP with the data feed that underpins this project. More detail on the measurement methodology can be found here.
In December, Deepfield deployed updates to participating ISPs’ measurement platforms. Unfortunately, these updates caused a complete interruption and/or a partial degradation of the CITP data feed from each participating ISP for a significant portion of the month. The outage took the form of both a complete loss of data at times and also at times a loss of the regional data that enables our more detailed analysis. The restoration of the data feed required two steps. First, Deepfield restored aggregate measures for a participating ISP, then subsequently, Deepfield restored the more detailed information, including regional data, for that ISP. In January, data feeds had been restored representing 85% of the capacity under study. In February, this had risen to 98%. We continue to work with Deepfield on restoration of the remaining data feeds and on the potential recovery of missing 1Q 2017 data.
Given the interruption to CITP’s normal data feed, we were unable to generate the full suite of analysis and measures for the month of December. For the “Aggregate Views,” we were unable to generate the “Peak Interconnect Utilization as a Proportion of Capacity” or the “Distribution of Peak Interconnect Utilization” for the month of December. Given the nature of the outage, we still received sufficient data on utilization and capacity to generate aggregate measures, but without the normal more detailed data, we were unable to generate meaningful distributions of utilization. For this same reason, we also were unable to generate the December heat map and histograms under “Views by Interconnect.”
Moreover, we omitted the analysis for the month of December for all of the “Regional Views.” As noted above, the outage also caused a loss of the regional breakdowns of utilization and capacity and prevented the normal analysis by region.
The above omissions are also reflected in the data file. In addition, for December, we omitted the measure of “% of Capacity Excluded from Measurements due to Data Integrity”. In recent months, the percent of aggregate interconnection capacity excluded from the analysis has been running between 2% and 3%. But, given the nature of the data outage and the loss of granularity in the December data, we were unable to calculate a percentage of excluded capacity in any meaningful way and found it necessary to omit this value.
CITP continues to work closely with CableLabs, the participating ISPs, and Deepfield to ensure the integrity of the data and to help avoid future outages.