LANDER:survey validation tcpvsicmp-20071009 From Predict README version: 4056, last modified: 2014-06-6. This file describes the trace dataset "survey_validation_tcpvsicmp-20071009" provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 3 Citation • 4 Results Using This Dataset • 5 User Annotations LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ survey_validation_tcpvsicmp-20071009 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ TCP, ICMP effectiveness probing hosts │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ During this experiment 1 million of random allocated IP addresses were probed │ │ │ using a single ICMP ECHO_REQUEST and a single TCP SYN on port 80. The order of │ │ │ probing was ICMP, then TCP for the first half of probed addresses and then TCP │ │ │ followed by ICMP for the second half. A slightly modifed version of NMAP was used. │ │ │ Output of NMAP as well as tcpdump of all sent/received packets are included in │ │ │ this dataset. │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2007-10-09 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 11:52:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2007-10-14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 04:14:55 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2012-01-27 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 17:06:19 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:address-space-status-data, subcategory:internet-census-and-survey-data, │ │ │ topology, internet, active-measurement, tcp, icmp, ping │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ text │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 566231040 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 565794022 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/109/1353561 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See https://ant.isi.edu/datasets/#getting-datasets for information on obtaining │ │ │ this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents survey_validation_tcpvsicmp-20071009.README.txt     copy of this README data/     iana-092807-ipv4-address-space.txt iana allocations used for probing     random.1M.ips.txt text files with 1M IPs probed     random.1M.ips.txt.1half first half of the above file (probed ICMP, then TCP)     random.1M.ips.txt.2half second half of the above file (probed TCP, then ICMP)     random.1M.ips.txt.nmap1 nmap output (text) from the first half     random.1M.ips.txt.nmap2 nmap output (text) from the second half     random.1M.ips.txt.tcpdump pcap trace of all interesting packets     create_random_ips.pl perl script used to randomly select addresses     drive_nmap.sh shell script used to start this experiment     parse_tcpdump.pl perl script used to process results     1M.jdb text file summarizing results (output of parse_tcpdump.pl)     ping_tcp.pl perl script for pinging addresses (used in earlier versions of this experiment, but not for this one)     .sha1sum SHA-1 checksum Subdirectory data contains raw unprocessed results, i.e. *.nmap{1,2}, *.tcpdump, as well as processed data 1M.jdb. It also contains all scripts used to drive this experiment. A slightly modified version of NMAP was used which was tweaked so that a TCP and ICMP probes could be sent back-to-back and their order could be controlled. These modifications are not included as they were lost. The file 1M.jdb summarizes the result: for each IP address that ever replied it contains RTTs for ICMP-ack (echo-response), TCP-ack (which was defined more broadly as any indication that the TCP probe got through), ICMP-nack (icmp *-unreachable) and TCP-nack (icmp proto tcp unreachable coming from a different IP). Absense of data is indicated as -1. The last column is firsthalf which is set to 1/0 depending on the order in which TCP/ICMP probing was performed. The file .sha1sum contains SHA1 checksums of individual compressed files. The integrity of the distribution thus can be checked by independently calculating SHA1 sums of files and comparing them with those listed in the file. If you have the sha1sum utility installed on your system, you can do that by executing: sha1sum --check .sha1sum This has to be done before files are uncompressed. Citation If you use this trace to conduct additional research, please cite it as: Internet Addresses Census dataset, PREDICT ID: USC-LANDER/survey_validation_tcpvsicmp-20071009. Traces taken 2007-10-09 to 2007-10-14. Provided by the USC/LANDER project (http://www.isi.edu/ant/lander). Results Using This Dataset Traces similar to this one containing collections of "live" IP addresses have been used the following previously published work: • John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, Genevieve Bartlett, and Joseph Bannister. Census and Survey of the Visible Internet. In Proceedings of the ACM Internet Measurement Conference, p.169-182. Vouliagmeni, Greece, ACM. October, 2008 http://www.isi.edu/~johnh/PAPERS/Heidemann08c.html. • Yuri Pryadkin, Robert Lindell, Joseph Bannister, and Ramesh Govindan An Empirical Evaluation of IP Address Space Occupancy Technical Report ISI-TR-2004-598, USC/Information Sciences Institute, November 2004 ftp://ftp.isi.edu/isi-pubs/tr-598.pdf. • Lin Quan, John Heidemann, Yuri Pradkin. Detecting Internet Outages with Precise Active Probing (extended). Technical Report ISI-TR-2012-678b, USC/Information Sciences Institute, May, 2012 ftp://ftp.isi.edu/isi-pubs/tr-678b.pdf. User Annotations Currently no annotations. Categories: • Datasets • LANDER • LANDER:Datasets • LANDER:Datasets:AddressSpace:Adaptive Probing • LANDER:Datasets:AddressSpace