LANDER:internet outage adaptive a38all-20191001 From Predict README version: 10919, last modified: 2020-01-31. This file describes the trace dataset "internet_outage_adaptive_a38all-20191001" provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 2.1 Combined outage information • 2.2 Downup outage data • 2.3 Code Version Metadata • 3 Citation • 4 Results Using This Dataset • 5 User Annotations LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ internet_outage_adaptive_a38all-20191001 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ reachability of responsive of IPv4 /24s │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ Combined reachability information for over 3.5M /24 subnets; this dataset is │ │ │ created by post-processing data from datasets │ │ │ internet_outage_adaptive_a38c-20191001 internet_outage_adaptive_a38e-20191001 │ │ │ internet_outage_adaptive_a38g-20191001 internet_outage_adaptive_a38j-20191001 │ │ │ internet_outage_adaptive_a38n-20191001 internet_outage_adaptive_a38w-20191001 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2019-10-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2020-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2020-02-18 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:address-space-status-data, subcategory:network-outage, ip-address, sweep, │ │ │ ping, icmp │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ text │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ internet outages datasets │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 1046478848 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 4832005442 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/109/1517809 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See https://ant.isi.edu/datasets/#getting-datasets for information on obtaining │ │ │ this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents This dataset contains processed data from datasets: • LANDER:internet_outage_adaptive_a38c-20191001 • LANDER:internet_outage_adaptive_a38e-20191001 • LANDER:internet_outage_adaptive_a38g-20191001 • LANDER:internet_outage_adaptive_a38j-20191001 • LANDER:internet_outage_adaptive_a38n-20191001 • LANDER:internet_outage_adaptive_a38w-20191001 Each block outage is labeled by what vantage point saw it, thus making it easy to differentiate connection problems between two particular end-points (vantage point and the target) and connection problems between the target and the rest of the world. The more vantage points saw the target's outage, the more severe the outage is. Generally, we provide two types of datafiles: a38all.{version}.outages.fsdb.bz2 a38all.{version}.outagedownup.fsdb.bz2 where outages is outage information for each block, obtained by merging individual sites' data; while outagedownup is further processed to arrive to decision if block was up/down/unmeasurable/gone-dark at a particular time. Since our processing code has been evolving, we started labeling each file with the version of processing code used. This versioning is summarized here: https://ant.isi.edu/datasets/outage/format.html#versions Combined outage information a38all.outages.fsdb.bz2 (and a38all.outages.raw.fsdb.bz2, if present) format: #fsdb -F t block start duration uncertainty status Where fields are: block block address of the /24 in hex (with trailing zeros) start when the status was takes effect (seconds since the Unix epoch) duration how long the status is in effect uncertainty our confidence in the precision of the start time. In non-raw data uncertainty is sometimes lowered when we merge observations from multiple observers. precision_improvement is either unused ('-') or precision improvement of the onset of a state change resulting from merging data from multiple vantage points vantage point that saw the outage (each letter 'c','j','w', 'g' is one of the sites from our status observers; corresponding capital letter 'W', 'C', 'J', 'G' means the vantage point saw no outage; the order is fixed to [wW][cC][jJ][gG]) The .raw. file is the direct observations from the observers (this file is present only in early datasets). The file without .raw. has been post-processed with precision improvement to reduce ambiguity around status start times, as described in section 4.5 of [Quan13c]. (This ambiguity arises because we probe every 11 minutes, the non-.raw file represents our best estimate.) Downup outage data a38all.outagedownup.fsdb.xz (where available) contains post-processed data from a38all.outages.fsdb.bz2. It's format: #fsdb -F t block start duration uncertainty downup Where fields are: block block address of the /24 in hex (with trailing zeros) start when the status was takes effect, in seconds since the Unix epoch. duration how long the status is in effect, in seconds. our confidence in the precision of the start time. The true start time is sometime between start and uncertainty start-uncertainty. The true duration is between duration-NextEventUncertainty and duration+ThisEventUncertainty. In non-raw data uncertainty is sometimes lowered when we merge observations from multiple observers. downup up (1), down (0), unmeasurable (-1, typically due to insufficient active observers), or gone dark (-2, typically out for more than 10 days) Post processing includes the following steps:  1. insufficient VP detection  2. merging roles  3. unmeasurability detection  4. hole filling of periods with insufficient observers Code Version Metadata The file a38all.code_meta.yml, if present, contains versioning information for the code used in the generation of this dataset. It records the date and git commit hash of last modification of each file. This can be useful in comparing datasets processed at different times to see what, if anything, changed in the processing pipeline. Citation If you use this trace to conduct additional research, please cite it as: USC/LANDER Project. Internet Outage Dataset, PREDICT ID: USC-LANDER/internet_outage_adaptive_a38all-20191001. Provided at https://ant.isi.edu/. Results Using This Dataset Lin Quan, John Heidemann, and Yuri Pradkin. Trinocular: Understanding Internet Reliability Through Adaptive Probing. In Proceedings of the ACM SIGCOMM Conference , p. 255-266. Hong Kong, China, ACM. August, 2013. Lin Quan. LEARNING ABOUT THE INTERNET THROUGH EFFICIENT SAMPLING AND AGGREGATION. PhD Thesis, May 2014. User Annotations Currently no annotations. Categories: • LANDER • LANDER:Datasets • Datasets • LANDER:Datasets:AddressSpace • LANDER:Datasets:AddressSpace:Adaptive Probing • LANDER:Datasets:AddressSpace:Adaptive Probing ALL