LANDER:iperf emulated attacks-20090621 From Predict README version: 4038, last modified: 2014-06-6. This file describes the trace dataset "iperf_emulated_attacks-20090621" provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 3 Dataset Generation • 4 Citation • 5 Results Using This Dataset • 6 User Annotations • 6.1 Converting File Format • 6.2 Other Questions and Comments LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ iperf_emulated_attacks-20090621 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ Emulated attacks in enterprise traffic │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ These traces were created by monitoring an emulated DoS attack from a single known │ │ │ source to single known target, mixed with ongoing traffic into a large enterprise. │ │ │ The attack was generated with Iperf sending UDP packets at a known rate. Data was │ │ │ collected in the middle of the network, close to the target, │ │ │ │ │ │ at the network connection for USC. │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2009-06-21 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2009-08-04 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2012-01-27 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 17:06:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ cryptopan/full │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:synthetically-generated-data, subcategory:experimental-data, one-time, │ │ │ packet-header │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ text │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 44518342656 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 539301623680 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/109/1353637 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See http://www.isi.edu/ant/traces/index.html#getting_datasets for information on │ │ │ obtaining this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents iperf_emulated_attacks-20090621.README.txt      copy of this README pcap_txt/     traceSeg_MMDDYY_HHMMSS-f.txt.bz2 tcpdump text files (compressed with bzip2); approximately 30-second long traces gathered on date MMDDYY at time HHMMSS.     .sha1sum SHA-1 checksum All IP addresses have been fully anonymized with prefix-preserve property. The file ".sha1sum" contains SHA1 checksums of individual compressed files. The integrity of the distribution thus can be checked by independently calculating SHA1 sums of files and comparing them with those listed in the file. If you have the sha1sum utility installed on your system, you can do that by executing: sha1sum --check .sha1sum This has to be done before files are uncompressed. Dataset Generation These traces were created by monitoring an emulated DoS attack from a single known source to single known target, mixed with ongoing traffic into a large enterprise. The attack was generated with Iperf sending UDP packets at a known rate. Data was collected in the middle of the network, close to the target, at the network connection for USC. A traceroute from the source to the target, taken on April 22, 2010 is: 1 fullanon:SOURCE.colostate.edu (fullanon:150.69.155.229) 0.777 ms 0.877 ms 0.995 ms 2 cs-gw-1.cs.colostate.edu (unanon:129.82.44.1) 1.422 ms 1.482 ms 1.549 ms 3 unanon:129.82.2.9 (unanon:129.82.2.9) 2.570 ms 2.645 ms 2.742 ms 4 unanon:nlrb-frgp.frgp.net (unanon:192.43.217.113) 3.288 ms 3.335 ms 3.595 ms 5 unanon:frgp-nlr.frgp.net (unanon:192.43.217.138) 3.784 ms 3.793 ms 3.784 ms 6 unanon:losa-denv-90.layer3.nlr.net (unanon:216.24.186.28) 32.539 ms 32.710 ms 31.970 ms 7 unanon:hpr-lax-hpr--nlr-pn.cenic.net (unanon:137.164.26.149) 31.674 ms 31.721 ms 31.766 ms 8 unanon:137.164.27.229 (unanon:137.164.27.229) 44.803 ms 44.977 ms 45.068 ms *** capture was here 9 unanon:v249-gw-9.usc.edu (unanon:68.181.194.72) 45.007 ms 45.099 ms 45.105 ms 10 fullanon:TARGET.usc.edu (fullanon:63.76.175.149) 44.978 ms 45.066 ms 45.067 ms (The above list mixes fully anonymized IP addresses/host names, tagged "fullanon:", that match what you will see in the dataset, with unanonymized addreses, tagged "unanon:". Unanonymized addresses here are provided only for routers in the Internet infrastructure; those routers do not correspond to any end users. The trace files include background traffic from other traffic entering and exiting USC. The text trace files was obtained by running the following command: traceconvert erf:/auto/samfs-00/LANDER/thatte/incoming_host/landeri2/20090507-092728-160187-h5813 pcap:- | /usr/sbin/tcpdump -r- -nn -e -tt dst net 151.46.0.0/16 or dst net 63.76.0.0/16 or dst net 194.167.0.0/16 or dst net 204.249.90.0/24 > traceSeg_050709_092728.txt The ERF files were filtered by destination IP addresses to ensure that only unidirectional (USC-incoming) traffic was considered in these traces. The Iperf UDP flow was generated by running the following commands. iperf -s -u -l 345 at the server side, anonymized IP address for June 21/25: 63.76.175.149, July 9: 63.76.175.16, Aug 4th: 63.76.175.122; and iperf -c 63.76.175.149 -u -b [R]m -t 80 -l 345 at the client side, anonymized IP address 150.69.155.229. The dataset contains the following Iperf UDP flows, with the attackRate [R] specified in Mbps where AttackTime reflects the start of the attack in hhmmss format. The DatagramsLost field comes from the Iperf output on the sending and reception of the UDP flow. The AttackRate (%) field is computed as 100*AttackRate(Mbps)/(AttackRate(Mbps)+BackgroundRate(Mbps)) The AttackPacket (%) field is calculated as 100*NumAttackPackets/(NumAttackPackets+NumBackPackets) June 21, 2009 AttackRate AttackTime AttackRate(%) AttackPacket(%) DatagramsLost 35 160402 14.49 25.18 0.15% 35 160702 13.43 24.34 9.7e-05% 35 161003 17.05 26.77 0% 35 161302 17.07 26.43 0% 35 161603 17.29 27.10 0% 35 161902 17.64 26.85 0.034% 30 162302 17.00 24.79 0% 30 162602 14.65 23.67 0% 30 162902 13.96 22.44 0.041% 30 163202 13.20 21.86 0.00011% 30 163502 15.43 23.54 0% 30 163802 13.67 21.89 0.0023% 30 164102 12.90 22.02 0.00057% 30 164402 15.87 25.10 0.056% 30 164702 15.31 23.99 0% 30 165002 13.08 22.34 0% 25 165402 12.98 20.58 0.054% 25 165702 11.03 19.17 0% 25 170002 11.02 19.07 0.012% 25 170302 10.60 19.12 0% 25 170602 13.66 21.01 0% 25 170902 11.12 19.18 0.034% 25 171202 12.39 20.56 0% 25 171502 12.34 19.57 0% 25 171802 11.67 19.71 0% 25 172102 13.74 20.89 0% 20 172602 12.48 18.05 0% 20 172902 11.81 17.63 0.028% 20 173202 13.01 18.03 0.00017% 20 173502 12.11 17.44 0% 20 173802 12.55 17.87 0% 20 174102 11.69 17.41 0% 20 174402 11.87 17.56 0.0083% 20 174702 10.51 16.05 0% 15 175102 8.40 13.13 0% 15 175402 8.67 13.35 0.0011% 15 180102 8.10 12.92 0% 15 180402 7.60 12.91 0.0018% 15 180702 7.64 12.66 0% 15 181002 8.39 13.58 0% 15 181302 8.44 13.97 0% 15 181602 8.47 13.42 0% 15 181902 8.52 13.76 0.0018% 10 182402 5.19 9.08 0% 10 182702 5.59 9.52 0% 10 183002 5.97 9.46 0% 10 183302 6.00 9.66 0% 10 183602 5.61 9.48 0% 10 183902 5.59 9.23 0% 10 184202 5.80 9.66 0% 10 184502 6.01 9.69 0% 5 184902 3.46 5.18 0% 5 185102 3.46 5.30 0% 5 185402 3.85 5.54 0% 5 185717 3.83 5.62 0% 5 190002 3.79 5.52 0% June 25, 2009 AttackRate AttackTime AttackRate(%) AttackPacket(%) DatagramsLost 30 130906 6.40 12.38 0% 30 131206 6.21 12.51 0.072% 30 131506 6.27 12.43 0% 30 131806 5.63 11.83 0% 30 132106 5.98 12.18 0% 30 132406 6.60 12.67 0% 25 132806 5.33 11.14 0% 25 133106 5.48 11.25 0.00069% 25 133406 4.99 10.68 0% 25 133706 5.41 11.14 0.083% 25 134006 5.73 11.55 0% 25 134306 5.28 11.12 0% 20 134706 3.92 8.20 0.028% 20 135006 4.45 9.00 0.00017% 20 135306 4.69 9.22 0.0076% 20 135606 4.51 9.00 0% 20 135906 4.58 9.05 0.0024% 20 140206 4.78 9.29 0.027% 15 140606 3.33 6.70 0.00046% 15 140906 3.41 6.78 0.00046% 15 141206 3.52 7.13 0.0062% 15 141506 2.84 6.25 0% 15 141806 3.28 6.93 0% 10 142206 2.22 4.69 0% 10 142506 1.87 3.45 0% 10 142806 1.97 4.15 0% 10 143106 2.03 3.58 0.0014% 10 143406 1.93 3.53 0% 10 143706 2.01 3.52 0% July 9, 2009 AttackRate AttackTime AttackRate(%) AttackPacket(%) DatagramsLost 60 113024 10.34 19.52 0.0056% 60 113424 10.88 20.00 0.029% 60 113824 10.29 19.42 0.05% 60 114225 11.67 21.28 5.7e-05% 60 114624 11.65 20.16 0.002% 60 115024 9.89 19.63 0.0013% 60 115425 9.06 18.61 0.082% 60 115825 10.14 19.67 5.7e-05% 55 120425 7.94 16.87 0.05% 55 120824 8.84 17.87 0.0034% 55 121224 8.42 17.45 0.0089% 55 121624 8.86 17.83 0.033% 55 122024 9.33 18.24 0.0022% 55 122425 7.66 16.55 0.0038% 55 122824 8.64 17.48 0% 55 123225 8.90 17.53 6.2e-05% 50 123825 7.31 14.87 0.0063% 50 124225 8.03 16.72 6.9e-05% 50 124625 7.74 16.27 0.028% 50 125025 7.72 16.45 0% 50 125424 7.10 15.66 0.0076% 50 125825 6.90 15.10 0.0014% 50 130224 7.38 15.86 0.013% 50 130524 7.22 15.80 0.016% 45 131025 7.53 15.91 0% 45 131424 7.24 15.36 0.0033% 45 131825 7.50 15.51 0% 45 132225 6.99 14.98 0.005% 45 132624 7.83 15.68 0.064% 45 133024 8.10 16.13 0.00099% 45 133425 8.23 16.02 0.14% 45 133824 8.29 16.44 0% 40 134424 7.05 14.19 0.0031% 40 134825 7.57 14.49 0% 40 135224 7.00 13.93 0.039% 40 135625 7.54 14.02 0.074% 40 140025 6.85 13.98 0.034% 40 140425 6.66 13.58 0.066% 40 140825 7.30 14.20 0.025% 40 141224 6.77 13.47 0.0028% 35 141924 6.22 12.80 0% 35 142425 6.33 13.14 0% 35 142825 7.10 14.01 0% 35 143224 6.61 13.41 0.026% 35 143625 6.78 13.57 0.039% 35 144024 6.73 13.52 0% 35 144425 6.82 13.67 0.0016% 35 144825 7.12 13.80 0.018% 30 145424 5.60 11.40 0% 30 145925 5.43 11.04 1.1% 30 150425 4.93 10.43 1.3% 30 150825 4.82 10.35 1.3% 30 151225 4.81 10.22 0.99% August 4, 2009 AttackRate AttackTime AttackRate(%) AttackPacket(%) DatagramsLost 60 114000 11.91 21.35 0.05% 60 114300 12.07 21.51 0.0045% 60 114600 12.54 22.24 0.0085% 60 114900 12.24 21.88 0.16% 60 115200 12.62 21.56 0.055% 60 115500 11.78 20.99 0.019% 60 115800 11.40 20.93 0.12% 60 120100 11.41 21.01 0.24% 60 120400 11.08 20.54 0.16% 60 120700 11.36 21.20 0.054% 60 121000 11.24 20.99 0% 60 121300 10.44 20.11 0.0091% 60 121600 11.53 21.23 0.047% 60 121900 10.86 20.61 0.066% 60 122200 10.49 19.84 5.8e-05% 40 122600 6.87 13.97 0% 40 122900 7.09 14.26 0.095% 40 123200 7.37 14.14 0.0072% 40 123500 8.04 15.02 0.067% 40 123800 7.75 14.89 0% 40 124100 6.99 13.93 0.025% 40 124400 7.11 13.97 0.16% 40 124700 7.76 14.55 0.029% 40 125000 7.87 14.86 0.027% 40 125300 8.31 15.17 0.027% 25 125700 5.20 9.96 0% 25 130000 5.15 9.99 0.025% 25 130300 5.43 10.13 0% 25 130600 5.74 10.62 0% 25 130900 5.28 9.93 0.051% 25 131200 5.50 10.07 0.0025% 25 131500 4.91 9.52 0% 25 131800 5.05 9.57 0% 20 154100 4.31 8.53 0% 20 154400 4.55 8.69 0.047% 20 154700 3.99 8.08 0% 20 155000 4.19 8.09 0% 20 155300 4.30 8.34 0.0012% 20 155600 4.09 8.15 0% 20 155900 4.16 8.26 0.03% 20 160200 3.84 7.88 0.00017% 20 160500 4.29 8.40 0.12% 20 160800 3.96 8.01 0% 20 161100 4.23 8.16 0% 20 161400 3.96 7.92 0.036% Citation If you use this trace to conduct additional research, please cite it as: Emulated attacks in enterprise traffic, PREDICT ID: USC-LANDER/iperf_emulated_attacks-20090621. Traces taken Traces taken 2009-06-21 to 2009-08-04. Provided by the USC/LANDER project http://www.isi.edu/ant/lander. Results Using This Dataset Traces similar to this one have been used the following previously published work: • Gautam Thatte, Urbashi Mitra, and John Heidemann. "Parametric Methods for Anomaly Detection in Aggregate Traffic," submitted for review to the IEEE/ACM Transactions on Networking, March 2010. • Gautam Thatte, Urbashi Mitra, and John Heidemann. "Parametric Methods for Anomaly Detection in Aggregate Traffic," Technical Report ISI-TR-2009-663, USC/Information Sciences Institute, August 2009 ftp://ftp.isi.edu/isi-pubs/tr-663.pdf. User Annotations Converting File Format A user asked: This dataset appears to be the text output of tcpdump. Is it possible to regenerate the binary format pcap files? (Presumably to play them through other tools?) There are several problems that make regeneration of binary-format pcap files difficult.  1. These traces were generated with Stream Merger application [1], to merge in controlled artifical attacks with real-world traces. Unfortunately Stream Merger currently only reads and writes text files.  2. The sources (the real-world traces) are anonymized packet headers, so they don't include packet payloads. In principle, one could extend Stream Merger to support binary pcap, and re-run it with new source data. Alternatively, one could write a custom program to reverse the text format into binary tcpdump, filling in missing fields. Johnh 16:47, 11 April 2011 (PDT) Other Questions and Comments (please put other questions/comments here) Categories: • LANDER • LANDER:Datasets • Datasets