Network Archaeology 1

What to expect

Network Archaeology is the study and practice of unearthing and understanding hidden data in network traffic and obfuscated files. This class will introduce the concepts, skills, and thought processes that provide the foundation for this work.

Computer science, information theory, and mathematics provide a foundation. However, this class is designed as a practical onramp to network archaeology for those who may not count such topics in their leading passions. While the Network Archaeology 2 and 3 classes will have students developing custom protocol decoders, this class focuses on building a foundation for students in concepts such as:

  • Data representation (bytes, ascii, encodings),
  • Common network protocols,
  • Packet structure,
  • Data analysis and manipulation using the Linux command line, and
  • Common analytic tools;

And culminates with introducing more advanced concepts like:

  • Encryption and compression,
  • Cryptanalysis, and
  • Back/reverse engineering

Light on slide presentations and heavy on hands-on challenges, Network Archaeology is organized as a self-paced lab class, with intermittent instructor lectures. We introduce students to a concept and teach them to apply it by having them develop and test their own solutions to problems of incrementally increasing difficulty. Between lectures, instructors traverse the room helping people with labs. Our friendly teaching assistants are available throughout and will happily sit with you and help you think through your approach.

Encouraging “living off the land”, this class makes heavy use of the Linux command line, as the most likely “least common denominator” toolset found in the wild. We encourage students, however, to focus more on techniques than on tools, being ready to adapt to different tools they encounter in the real world. Beyond the command line, some demonstrations will include the use of Wireshark and of web-based tools such as Cyber Chef. While some challenges may defy such tools, participants can make notable progress using them.

While a coding background is far from necessary, those who have one are welcome to apply it, and some challenges may even demonstrate light use of scripting (with starter code provided). Each lab exercise either introduces new or builds on previously presented concepts. There is no expectation that everyone will finish all lab exercises during the three days of class.

Technical Content intermediate

Audience

  • Incident responders
  • Threat analysts
  • Digital forensics analysts
  • Cybercrime investigators
  • Cybersecurity Analysts
  • Network operators
  • System administrators
  • Software engineers
  • Cybersecurity instructors
  • Cyberspace operators
  • Exploitation analysts

Applicability

Network Archaeology teaches students how to discover and approach unknown data that existing tools may struggle to handle. This introductory course provides participants with a robust foundation on which to begin building their network archaeology skills.

People expecting to walk away with a recipe book will be disappointed. Our goal is for you to gain insight about how network protocols work; how adversaries may misuse and abuse the network to hide their activities; and basic encoding, encryption, and compression techniques. Further, we want you to walk away understanding what common tools and techniques can be applied (and when) to “break” these malware protocols.

Network Archaeology is broadly interesting to anyone who wants a better understanding of the process of network forensic techniques. Even if you don’t intend to engage in this activity in your job, going through the instructor-led exercises will provide insight into challenges facing your organization.

Objectives

  • Understand common number bases (binary, decimal, and hexadecimal), how to convert between them and their relevance to Network Archaeology
  • Recognize and differentiate different encodings (e.g., hexadecimal, Base64)
  • Recognize and differentiate common TCP/IP protocols
  • Understand byte structure of TCP/IP and DNS
  • Recognize common tools used in Network Archaeology - know when they may apply, and their basic functionality
  • Know how to extract transferred files out of packet captures (expected protocols)
  • Understand practical single- and multi-byte XOR encoding/decoding
  • Understand what entropy represents and its relation to plain-, compressed-, and encrypted-data
  • Understand how guessed plaintext attacks are used
  • Apply knowledge of network protocols to recognize and isolate undocumented and misused network protocols

Typical Agenda

Day 0
30m
  • Course introduction
300m
  • More detailed agenda to be posted closer to the event
Day 1
360m
  • More detailed agenda to be posted closer to the event
Day 2
360m
  • More detailed agenda to be posted closer to the event

Setup

You will need a computer with a modern web browser, and a Linux command line (preferably bash). If you need a suggestion on which Linux distribution, our team typically recommends Ubuntu, but this isn’t a requirement.

Many participants choose to use a dedicated VM for the class. Whether you do the same or bring a laptop with Linux installed as the native OS is up to you. In the past some participants have run into issues installing software at the event due to corporate policies, slow hotel wifi, or other challenges. We will be unable to help overcome most of these issues, so we strongly encourage you to complete your setup prior to your departure.

In any case, please have the following packages pre-installed PRIOR to the start of class:

  • wireshark
  • tcpflow
  • tcpdump
  • python3
  • A C build toolchain:
    • apt install build-essential on Ubuntu / Debian / Mint
    • yum groupinstall 'Development Tools' on Red Hat / CentOS
  • The Fluffy Suite (Install from source using the linked instructions)

Other Operating Systems

If you really know what you’re doing, you can complete this class with MacOS or Windows. Be prepared to figure out your OS quirks on your own, however. Windows users should be prepared to write a lot of code, as our command-line recipes won’t work at all in Windows.

A further note about modern Apple laptops: Recent Apple laptops (those with Apple processors) use an ARM architecture. Many participants have arrived planning to install virtualization software and use common VMs only to find that this does not work, as many common VMs are not ARM compatible. All of the above software should be available either as a Mac package or able to be built and installed on ARM. Homebrew can be helpful for this, and Kali does offer an installer image specifically for Apple Silicon (ARM64). It should be noted that both MacOS and kali default to zsh - however, bash is our recommended shell, so we encourage you to make that switch. If you are wondering why, consider trying the commands we demonstrate in both shells and observing the difference. As previously noted, our staff will not have the time to help you work through this setup, so please complete it prior to arrival.