![[ANT logo]](../images/logos/ant_logo_1_128px.png){kind=logo}
Retro-Future: The Retrospective Future in the Internet
Project Description
New network security events, from new zero-day attacks to insider threats, are only apparent after they have occurred. By definition, these events work in unexpected ways and only through hindsight may one learns what should have been collected and analyzed. To address these events, one either needs to predict the unexpected or travel back in time to replay the event. The premise of this project is to record enough network state to replay network security events and effectively travel back in time–in effect, to provide an Internet Digital Video Recorder (DVR).
The challenge in this effort is to provide Internet time-travel that is efficient, maximizing the effective history that is saved, to be cost-effective commodity hardware and software, and most importantly, to accommodate permission and privacy constraints that are necessary to deploy this system.
We expect that the resulting Internet DVR will help aid development and testing of network defenses.
Retro-future is a joint research effort of USC Information Sciences Institute Colorado State University’s Network Security lab, and Los Alamos National Laboratory. It is part of the ANT: the Analysis of Network Traffic research group.
This work supported (2012-2016) by Department of Homeland Security Science and Technology Directorate, Cyber Security Division , via SPAWAR Systems Center Pacific under Contract No. N66001-13-C-3001. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of SSC-Pacific.
It is also supported (starting in 2016) by the Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via contract number HHSP233201600010C. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Department of Homeland Security.
People
- Manaf Gharaibeh, collaborating faculty (Colorado State University CS Dept.) (currently faculty at Princess Sumaya University For Technology, Jordan; also Colorado State PhD graduate, 2020)
- John Heidemann, PI on this project, project leader and professor (USC/ISI)
- Christos Papadopoulos, co-PI on this project, professor (University of Memphis) christos.papadopoulos (at) memphis.edu
- Yuri Pradkin, researcher (USC/ISI)
Alumni
- Calvin Ardi, USC CS PhD graduate (2020) (USC CS Dept. and ISI)
- Daniel Byrne, graduate research assistant (LANL)
- Xun Fan, USC CS PhD graduate (2015) (USC CS Dept. and ISI)
- Paul Ferrell, researcher (LANL)
- Gina Fisk, project lead (LANL)
- Mike Fisk, co-PI on this project, project leader (LANL)
- Andy Gu, undergraduate researcher (USC CS Dept. and ISI)
- Zi Hu, USC CS MS graduate (2014) (USC CS Dept. and ISI)
- Dan Massey, professor (University of Colorado Boulder CS Dept.)
- Neale Pickett, researcher (LANL)
- Abdul Qadeer, PhD student (USC CS Dept. and ISI)
- Anant Shah, Colorado State U. CS PhD graduate (2018) (Colorado State University CS Dept.)
- Shannon (Shane) Steinfadt, researcher and project lead (LANL)
- Ben Uphoff, researcher (LANL)
- James Wernicke, researcher (LANL)
- Han Zhang, Colorado State U. CS PhD graduate (2017) (Colorado State University CS Dept.)
In addition, we thank Dan Massey for his involvement early in the effort. He is currently on leave from Colorado State and this project.
Publications
- Lan Wei and John Heidemann 2020. Whac-A-Mole: Six Years of DNS Spoofing. Technical Report arXiv:2011.12978v1. USC/ISI. [PDF] Details
- Manaf Gharaibeh, Christos Papadopoulos, John Heidemann and Craig Partridge 2020. Delay-based Identification of Internet Block Movement. Technical Report CS-20-101. Colorado State University Computer Science Department . [PDF] Details
- Abdul Qadeer and John Heidemann 2019. Plumb: Efficient Processing of Multi-User Pipelines (Poster). USC/Information Sciences Institute. [PDF] Details
- Liang Zhu and John Heidemann 2018. LDplayer: DNS Experimentation at Scale. Proceedings of the ACM Internet Measurement Conference (Boston, Massachusetts, USA, Oct. 2018), to appear. [DOI] [PDF] Details
- Abdul Qadeer and John Heidemann 2018. Plumb: Efficient Processing of Multi-User Pipelines (Poster Abstract). Proceedings of the Symposium on Cloud Computing (Carlsbad, CA, USA, Oct. 2018), 519. [DOI] [PDF] Details
- Abdul Qadeer and John Heidemann 2018. Plumb: Efficient Processing of Multi-Users Pipelines (Extended). Technical Report ISI-TR-727. USC/Information Sciences Institute. [PDF] Details
- Hang Guo and John Heidemann 2018. IP-Based IoT Device Detection. Proceedings of the ACM SIGCOMM Workshop on IoT Security and Privacy (Budapest, Hungary, Aug. 2018), 36–42. [DOI] [PDF] [Dataset] Details
- Calvin Ardi and John Heidemann 2018. Leveraging Controlled Information Sharing for Botnet Activity Detection. Proceedings of the ACM SIGCOMM Workshop on Traffic Measurements for Cybersecurity (Budapest, Hungary, Aug. 2018), 14–20. [DOI] [PDF] Details
- John Heidemann 2018. Internet Outages: Reliablity and Security. Invited talk at University of Oregon Cybersecurity Day. [PDF] Details
- Hang Guo and John Heidemann 2018. Detecting ICMP Rate Limiting in the Internet. Proceedings of the Passive and Active Measurement Workshop (Berlin, Germany, Mar. 2018), to appear. [PDF] Details
- John Heidemann 2018. Outage Clustering: From Leaves to Trees. Talk at CAIDA Active Internet Measurement Workshop (AIMS). [PDF] Details
- Basileal Imana, Aleksandra Korolova and John Heidemann 2018. Enumerating Privacy Leaks in DNS Data Collected Above the Recursive. Proceedings of the ISOC NDSS Workshop on DNS Privacy (San Diego, California, USA, Feb. 2018). [PDF] [Dataset] Details
- Lan Wei and John Heidemann 2018. Does Anycast Hang up on You (UDP and TCP)? IEEE Transactions on Network and Service Management. 15, 2 (Feb. 2018), 707–717. [PDF] Details
- John Heidemann 2018. Internet Reliability, from Addresses to Outages. Talk at MIT CSAIL. [PDF] Details
- John Heidemann, Yuri Pradkin and Aqib Nisar 2018. Back Out: End-to-end Inference of Common Points-of-Failure in the Internet (extended). Technical Report ISI-TR-724. USC/Information Sciences Institute. [PDF] Details
- Liang Zhu and John Heidemann 2017. LDplayer: DNS Experimentation at Scale. Technical Report 722. USC/Information Sciences Institute. [PDF] Details
- Wouter B. de Vries, Ricardo de O. Schmidt, Wes Hardaker, John Heidemann, Pieter-Tjerk de Boer and Aiko Pras 2017. Verfploeter: Broad and Load-Aware Anycast Mapping. Proceedings of the ACM Internet Measurement Conference (London, UK, 2017), 477–488. [DOI] [PDF] [Dataset] Details
- Moritz Müller, Giovane C. M. Moura, Ricardo de O. Schmidt and John Heidemann 2017. Recursives in the Wild: Engineering Authoritative DNS Servers. Proceedings of the ACM Internet Measurement Conference (London, UK, 2017), 489–495. [DOI] [PDF] [Dataset] Details
- Kensuke Fukuda, John Heidemann and Abdul Qadeer 2017. Detecting Malicious Activity with DNS Backscatter Over Time. ACM/IEEE Transactions on Networking. 25, 5 (Aug. 2017), 3203–3218. [DOI] [PDF] [Dataset] Details
- Liang Zhu and John Heidemann 2017. LDplayer: DNS Experimentation at Scale (abstract with poster). Technical Report ISI-TR-2017-721. USC/Information Sciences Institute. [PDF] Details
- Liang Zhu and John Heidemann 2017. LDplayer: DNS Experimentation at Scale (poster abstract). Proceedings of the SIGCOMM Posters and Demos (Aug. 2017), 60–62. [DOI] [PDF] Details
- Lan Wei and John Heidemann 2017. Does Anycast Hang up on You? IEEE. [DOI] [PDF] Details
- Jelena Mirkovic, Genevieve Bartlett, John Heidemann, Hao Shi and Xiyue Deng 2017. Do You See Me Now? Sparsity in Passive Observations of Address Liveness. IEEE International Conference on Traffic Monitoring and Analysis (Dublin, Ireland, Jul. 2017), 1–9. [DOI] [PDF] Details
- John Heidemann 2017. Digging in to Ground Truth in Network Measurements. Talk at the Network Traffic Measurement and Analysis PhD School. [PDF] Details
- Moritz Müller, Giovane C. M. Moura, Ricardo de O. Schmidt and John Heidemann 2017. Recursives in the Wild: Engineering Authoritative DNS Servers. Technical Report ISI-TR-720. USC/Information Sciences Institute. [PDF] Details
- Wouter B. de Vries, Ricardo de O. Schmidt, Wes Hardaker, John Heidemann, Pieter-Tjerk de Boer and Aiko Pras 2017. Verfploeter: Broad and Load-Aware Anycast Mapping. Technical Report ISI-TR-719. USC/Information Sciences Institute. [PDF] [Dataset] Details
- Hang Guo and John Heidemann 2017. Detecting ICMP Rate Limiting in the Internet (extended). Technical Report ISI-TR-717. USC/Information Sciences Institute. [PDF] Details
- Ricardo de O. Schmidt, John Heidemann and Jan Harm Kuipers 2017. Anycast Latency: How Many Sites Are Enough? Proceedings of the Passive and Active Measurement Workshop (Sydney, Australia, Mar. 2017), to appear. [PDF] Details
- John Heidemann 2017. Collecting and Visualizing Outages Over the Long Haul. Talk at CAIDA Active Internet Measurement Workshop (AIMS). [PDF] Details
- Lan Wei and John Heidemann 2017. Does Anycast Hang up on You? (extended). Technical Report ISI-TR-716. USC/Information Sciences Institute. [PDF] Details
- Anant Shah, Romain Fontugne and Christos Papadopoulos 2016. Towards Characterizing International Routing Detours. Proceedings of the 12th Asian Internet Engineering Conference (AINTEC) (Bangkok, Thailand, Nov. 2016), to appear. Details
- Giovane C. M. Moura, Ricardo de O. Schmidt, John Heidemann, Wouter B. de Vries, Moritz Müller, Lan Wei and Christian Hesselman 2016. Anycast vs. DDoS: Evaluating the November 2015 Root DNS Event. Proceedings of the ACM Internet Measurement Conference (Nov. 2016). [DOI] [PDF] Details
- John Heidemann, Ricardo de O. Schmidt and Jan Harm Kuipers 2016. Anycast Latency: How Many Sites are Enough? Presentation at DNS-OARC Meeting. [PDF] Details
- John Heidemann, Giovane C. M. Moura, Ricardo de O. Schmidt, and Wouter B. de Vries, Moritz Muller, Lan Wei and Christian Hesselman 2016. Anycast vs. DDoS: Evaluating Nov. 30. Presentation at DNS-OARC Meeting. [PDF] Details
- Jelena Mirkovic, Genevieve Bartlett, John Heidemann, Hao Shi and Xiyue Deng 2016. Do You See Me Now? Sparsity in Passive Observations of Address Liveness (extended). Technical Report ISI-TR-2016-710. USC/Information Sciences Institute. [PDF] Details
- Giovane C. M. Moura, Ricardo de O. Schmidt, John Heidemann, Wouter B. de Vries, Moritz Müller, Lan Wei and Christian Hesselman 2016. Anycast vs. DDoS: Evaluating the November 2015 Root DNS Event (extended). Technical Report ISI-TR-2016-709b. USC/Information Sciences Institute. [PDF] Details
- Ricardo de O. Schmidt, John Heidemann and Jan Harm Kuipers 2016. Anycast Latency: How Many Sites Are Enough? Technical Report ISI-TR-2016-708. USC/Information Sciences Institute. [PDF] Details
- Z. Hu, L. Zhu, J. Heidemann, A. Mankin, D. Wessels and P. Hoffman 2016. Specification for DNS over Transport Layer Security (TLS) . Technical Report 7858. Internet Request For Comments. [DOI] [PDF] Details
- Abdul Qadeer, John Heidemann and Kensuke Fukuda 2016. Improving Long-term Accuracy of DNS Backscatter for Monitoring of Internet-Wide Malicious Activity (poster). Technical Report ISI-TR-2016-707. USC/Information Sciences Institute. [PDF] [Dataset] Details
- Manaf Gharaibeh, Han Zhang, Christos Papadopoulos and John Heidemann 2016. Assessing Co-Locality of IP Blocks. Proceedings of the 19th IEEE Global Internet Symposium (San Francisco, CA, USA, Apr. 2016). [PDF] Details
- Han Zhang, Manaf Gharaibeh, Spiros Thanasoulas and Christos Papadopoulos 2016. BotDigger: Detecting DGA Bots in a Single Network. Proceedings of the IEEE International Conference on Traffic Monitoring and Analysis (Louvain La Neuve, Belgium, Apr. 2016), 16–21. [DOI] Details
- Liang Zhu, Zi Hu, John Heidemann, Duane Wessels, Allison Mankin and Nikita Somaiya 2016. T-DNS: Connection-Oriented DNS to Improve Privacy and Security (poster abstract). Technical Report ISI-TR-2016-706. USC/Information Sciences Institute. [PDF] Details
- Calvin Ardi and John Heidemann 2016. AuntieTuna: Personalized Content-Based Phishing Detection. Proceedings of the NDSS Workshop on Usable Security (San Diego, California, USA, Feb. 2016). [PDF] Details
- Han Zhang, Manaf Gharaibeh, Spiros Thanasoulas and Christos Papadopoulos 2016. BotDigger: Detecting DGA Bots in a Single Network. Technical Report CS-16-101. Colorado State University . Details
- Manaf Gharaibeh, Han Zhang, Christos Papadopoulos and John Heidemann 2015. Assessing Co-Locality of IP Blocks. Technical Report CS-15-103. Colorado State University Department of Computer Science . [PDF] Details
- Kensuke Fukuda and John Heidemann 2015. Detecting Malicious Activity with DNS Backscatter. Proceedings of the ACM Internet Measurement Conference (Tokyo, Japan, Oct. 2015), 197–210. [DOI] [PDF] [Dataset] Details
- Kensuke Fukuda and John Heidemann 2015. Detecting Malicious Activity with DNS Backscatter (extended). Technical Report ISI-TR-2015-704. USC/Information Sciences Institute. [PDF] [Dataset] Details
- Abdulla Alwabel, John Healy, John Heidemann, Brian Luu, Yuri Pradkin and Rasoul Safavian. 2015. Evaluating Externally Visible Outages. Technical Report ISI-TR-701. USC/Information Sciences Institute. [PDF] Details
- Calvin Ardi and John Heidemann 2015. Poster: Lightweight Content-based Phishing Detection. Technical Report ISI-TR-2015-698. USC/Information Sciences Institute. [PDF] Details
- Liang Zhu, Zi Hu, John Heidemann, Duane Wessels, Allison Mankin and Nikita Somaiya 2015. Connection-Oriented DNS to Improve Privacy and Security. Proceedings of the 36thIEEE Symposium on Security and Privacy (San Jose, Californa, USA, May 2015), 171–186. [DOI] [PDF] [Dataset] Details
- Xun Fan, Ethan Katz-Bassett and John Heidemann 2015. Assessing Affinity Between Users and CDN Sites. Proceedings of the 7th IEEE International Workshop on Traffic Monitoring and Analysis (Barcelona, Spain, Apr. 2015). [DOI] [PDF] [Dataset] Details
- Liang Zhu, Zi Hu, John Heidemann, Duane Wessels, Allison Mankin and Nikita Somaiya 2015. Connection-Oriented DNS to Improve Privacy and Security (extended). Technical Report ISI-TR-2015-695. USC/Information Sciences Institute. [PDF] Details
- Liang Zhu, Zi Hu and John Heidemann 2015. Evaluation of Future DNSSEC Response Sizes at a Root and a TLD Server. [PDF] Details
- Lin Quan, John Heidemann and Yuri Pradkin 2014. When the Internet Sleeps: Correlating Diurnal Networks With External Factors. Proceedings of the ACM Internet Measurement Conference (Vancouver, BC, Canada, Nov. 2014), 87–100. [DOI] [PDF] Details
- John Heidemann 2014. Internet Populations (Good and Bad): Measurement, Estimation, and Correlation. Presentation at ICERM Workshop on Cybersecurity. [PDF] Details
- Liang Zhu, Zi Hu, John Heidemann, Duane Wessels, Allison Mankin and Nikita Somaiya 2014. T-DNS: Connection-Oriented DNS to Improve Privacy and Security (extended). Technical Report ISI-TR-2014-693. USC/Information Sciences Institute. [PDF] Details
- Lin Quan, John Heidemann and Yuri Pradkin 2014. When the Internet Sleeps: Correlating Diurnal Networks With External Factors (extended). Technical Report ISI-TR-2014-691b. USC/Information Sciences Institute. [PDF] Details
- Lin Quan, John Heidemann and Yuri Pradkin 2014. When the Internet Sleeps: Correlating Diurnal Networks With External Factors (extended). Technical Report ISI-TR-2014-691. USC/Information Sciences Institute. [PDF] Details
- Michael E. Fisk and Curtis L. Hash 2014. FileMap: Map-Reduce Program Execution on Loosely-Coupled Distributed Systems. Proceedings of the 4th International Workshop on Cloud Data and Platforms (Amsterdam, the Netherlands, Apr. 2014), to appear. [DOI] Details
- Zi Hu, Liang Zhu, Calvin Ardi, Ethan Katz-Bassett, Harsha V. Madhyastha, John Heidemann and Minlan Yu 2014. The Need for End-to-End Evaluation of Cloud Availability. Proceedings of the Passive and Active Measurement Workshop (Marina del Rey, California, USA, Mar. 2014), 119–130. [DOI] [PDF] Details
- Alefiya Hussain, Yuri Pradkin and John Heidemann 2013. Replay of Malicious Traffic in Network Testbeds. Proceedings of the 13th IEEE Conference on Technologies for Homeland Security (HST) (Waltham, Massachusetts, USA, Nov. 2013), (to appear). [PDF] Details
- Matt Calder, Xun Fan, Zi Hu, Ethan Katz-Bassett, John Heidemann and Ramesh Govindan 2013. Mapping the Expansion of Google’s Serving Infrastructure. Proceedings of the ACM Internet Measurement Conference (Barcelona, Spain, Oct. 2013), 313–326. [PDF] Details
- Matt Calder, Xun Fan, Zi Hu, Ethan Katz-Bassett, John Heidemann and Ramesh Govindan 2013. Mapping the Expansion of Google’s Serving Infrastructure. Technical Report TR 13-935. University of Southern California Computer Science Department. [PDF] Details
- Lin Quan, John Heidemann and Yuri Pradkin 2013. Poster Abstract: Towards Active Measurements of Edge Network Outages. Proceedings of the Passive and Active Measurement Workshop (Hong Kong, China, Mar. 2013), 276–279. [DOI] [PDF] Details
- John Heidemann 2013. Third-Party Measurement of Network Outages in Hurricane Sandy. Proceedings of the FCC Workshop on Network Resiliency (Brooklyn, New York, USA, Feb. 2013). [PDF] Details
For related publications, please see the ANT publications web page.
Software
- antlink Manage a tree of git or other VC repositories with funky symlinks
- babarchive Manage babarchives, checksummed directory trees that can be validated
- cryptopANT C/C++ Library/tool for IP address anonymization
- dag scrubber Tool for scrubbing packet traces
- dnsanon extract DNS traffic from pcap to text with optionally anonymization
- dnsanon_rssac Dnsanon_rssac is an implementation of RSSAC-002v2 processing for DNS statistics
- LANDER Trace Software LANDER Trace Capture software handles for packet capture, scrubbing, and triggering user-provided scripts
- LDplayer/dns-replay-client Replay DNS queries against a DNS server with correct timing and optionally log timing or latency.
- LDplayer/dns-replay-controller LDplayer component
- LDplayer/dns-query-mutator LDplayer component
- LDplayer/dns-replay-proxy A proxy that helps to emulate DNS hierarchy in DNS trace replay.
- LDplayer/dns-route-setup LDplayer component
- LDplayer/dns-zone-constructor LDplayer component
- timefind and indexer Software to handle indexing and selection of multiple network data types based on a given time range.
See also the ANT software web page.
Datasets
The primary goal of Retro-Future is to develop new network measurement capabilities. When we generate new datasets we can release as a side-effect of this process we will announce them here.