Network Archaeology

What to expect

The Network Archaeology class trains analysts and defenders to unearth and understand hidden data from network traffic and obfuscated files. The course covers foundational concepts such as:

  • data representation (bytes, ascii, encodings),
  • common network protocols, and
  • packet structure; And progresses to more advanced concepts like:
  • encryption and compression,
  • cryptanalysis, and
  • back/reverse engineering.

Those with a passion for theoretical CS, information theory, or mathematics will find it weaved throughout the course, but those without such passions need not flee in the other direction. These concepts are taught by hands-on incident response adventures where learning often occurs without even realizing it.

The class is presented as a series of technical challenges, each teaching a concept by allowing students to invent different approaches and try them out, seeing which work well and which do not. Challenges build on each other, until, by the final challenge, students are writing custom protocol decoder software from scratch.

Network Archaeology is a self-paced lab class, with intermittent instructor lectures. Some participants may find the first dozen or so labs easy: they are encouraged to proceed through as quickly as they like. The instructors lead occasional “how-to” lectures, starting with the first lab, eventually bringing the class to the same point. Between lectures, instructors traverse the room helping people with labs. Our friendly teaching assistants are available throughout and will happily sit with you and help you think through your approach.

This class is primarily taught using the Linux command line. The instructor will progressively apply command-line tools, iterating on a piece of data until it becomes something recognizable. The instructor will demonstrate multiple tools, encouraging students to prepare for “living off the land” and adapting their techniques to different tools they may encounter in the real world. Beyond the command line, some demonstrations will include the use of Wireshark and of web-based tools such as Cyber Chef - While some challenges will defy such tools, participants can make decent progress using them. Those with a coding background won’t be left out either, as some challenges will even demonstrate python scripting (with starter code often provided).

Each lab exercise either introduces new concepts or builds on previously presented concepts. Very few people make it through every exercise, and there is no expectation that anyone will “finish” in three days. Many Network Archaeology attendees come back to Cyber Fire to take this class a second or even third time.

Technical Content expert

Audience

  • Computer scientists
  • Mathematicians
  • Information theorists
  • Incident investigators
  • Software engineers
  • Applied mathematicians
  • System administrators
  • Site reliability engineers

Applicability

Network Archaeology teaches students how to approach unknown data that no existing tool can handle. People expecting to walk away with a recipe book will be disappointed. Our goal is for you to gain insight about how network protocols work, how encryption works, and what common techniques can be used to “break” malware protocols.

Network Archaeology is broadly interesting to anyone who wants a better understanding of the process of network packet forensic techniques. Even if you don’t intend to engage in this activity in your job, going through the instructor-led exercises will provide insight into challenges facing your organization.

Objectives

  • Understand byte structure of TCP/IP and DNS
  • Understand the theory of Base64 encoding
  • Recognize Base64 and hexadecimal encodings
  • Recognize and differentiate popular TCP/IP protocols
  • Know how to carve transferred files out of ICMP, DNS, and HTTP packet captures
  • Understand practical single- and multi-byte XOR encoding/decoding
  • Know techniques to approach guessed plaintext attacks against basic obfuscation
  • Understand cryptographic keyspace
  • Understand the significance of entropy in cryptanalysis
  • Be able to use protocol design knowledge and deobfuscation attacks to decode undocumented and obfuscated network protocols
  • Recognize and be able to leverage multiple tool chains to achieve the same result
  • Understand the fundamental structure and encoding techniques used in the SUNBURST DGA

Typical Agenda

Day 0
90m
  • Base arithmetic
  • Encoding schemes
120m
  • Introduction to network protocols
  • Byte structure of TCP/IP
  • DNS packet structure
120m
  • Examining packet captures
  • Extracting transferred data from packet captures
30m
  • Attack techniques against weak encryption
  • Helpful tools for Network Archaeology
Day 1
90m
  • Attack techniques against weak encryption
  • Helpful tools for Network Archaeology
90m
  • Entropy as it relates to cryptography
  • Mechanics of compression and its impact to analysis
120m
  • Using sequencing meta-information to reconstruct transferred information
  • Application-layer protocol tunneling
60m
  • Attacking novel weak cryptography with no prior knowledge
Day 2
90m
  • Attacking novel compression with no prior knowledge
90m
  • Analysis and decoding of novel binary protocols with no prior knowledge
90m
  • SUNBURST Domain-Generation Algorithm
  • Base64 vs. Base32
  • Endianness issues (Esab32)
  • SUNBURST monoalphabetic substitution cipher
90m
  • First-pass decode of SUNBURST DGA
  • Accurate prediction of Esab32 vs substitution
  • Decoding GUID
  • Parsing of domain lists

Setup

You will need a computer with a modern web browser, and a Linux command line (preferably bash). If you need a suggestion on which Linux distribution, our team typically recommends Ubuntu, but this isn’t a requirement.

Many participants choose to use a dedicated VM for the class. Whether you do the same or bring a laptop with Linux installed as the native OS is up to you. In the past some participants have run into issues installing software at the event due to corporate policies, slow hotel wifi, or other challenges. We will be unable to help overcome most of these issues, so we strongly encourage you to complete your setup prior to your departure.

In any case, please have the following packages pre-installed PRIOR to the start of class:

  • wireshark
  • tcpflow
  • tcpdump
  • python3
  • A C build toolchain:
    • apt install build-essential on Ubuntu / Debian / Mint
    • yum groupinstall 'Development Tools' on Red Hat / CentOS
  • The Fluffy Suite (Install from source using the linked instructions)

Other Operating Systems

If you really know what you’re doing, you can complete this class with MacOS or Windows. Be prepared to figure out your OS quirks on your own, however. Windows users should be prepared to write a lot of code, as our command-line recipes won’t work at all in Windows.

A further note about modern Apple laptops: Recent Apple laptops (those with Apple processors) use an ARM architecture. Many participants have arrived planning to install virtualization software and use common VMs only to find that this does not work, as many common VMs are not ARM compatible. All of the above software should be available either as a Mac package or able to be built and installed on ARM. Homebrew can be helpful for this, and Kali does offer an installer image specifically for Apple Silicon (ARM64). It should be noted that both MacOS and kali default to zsh - however, bash is our recommended shell, so we encourage you to make that switch. If you are wondering why, consider trying the commands we demonstrate in both shells and observing the difference. As previously noted, our staff will not have the time to help you work through this setup, so please complete it prior to arrival.