Intro Network Archaeology

What to expect

We begin with several fundamental CS concepts: binary network protocols, elementary cryptanalysis, and back engineering (reverse engineering/cleanroom design). We also have an intense but occasional focus on mathematics and information theory.

People who are not deep into mathematics and computer science will still benefit from this class, but those who have a passion for theoretical CS, information theory, or mathematics, will gain the most.

The class is presented as a series of technical challenges, each teaching a concept by allowing students to invent different approaches and try them out, seeing which work well and which do not. Challenges build on each other, until, by the final challenge, students are writing custom binary protocol decoder software from scratch.

Network Archaeology is a self-paced lab class, with intermittent instructor lectures. Some participants may find the first dozen or so labs easy: they are encouraged to proceed through as quickly as they like. The instructors lead occasional “how-to” lectures, starting with the first lab, eventually bringing the class to the same point. Between lectures, instructors traverse the room helping people with labs.

This class is taught using the Linux command line. The instructor will use command-line tools, to create increasingly powerful tools, but participants can make decent progress using Wireshark (local install) and Cyber Chef (web-based tool).

Each lab exercise either introduces new concepts or builds on previously presented concepts. Very few people make it through every exercise, and there is no expectation that anyone will “finish” in two days. Many Network Archaeology attendees come back to Cyber Fire to take this class a second or even third time.

Technical Content intermediate

Audience

  • Computer scientists
  • Mathematicians
  • Information theorists
  • Incident investigators
  • Software engineers
  • Applied mathematicians
  • System administrators
  • Site reliability engineers

Applicability

Network Archaeology teaches students how to approach unknown data that no existing tool can handle. People expecting to walk away with a recipe book will be disappointed. Our goal is for you to gain insight about how network protocols work, how encryption works, and what common techniques can be used to “break” malware protocols.

Network Archaeology is broadly interesting to anyone who wants a better understanding of the process of network packet forensic techniques. Even if you don’t intend to engage in this activity in your job, going through the instructor-led exercises will provide insight into challenges facing your organization.

Objectives

  • Understand byte structure of TCP/IP and DNS
  • Understand the theory of Base64 encoding
  • Recognize Base64 and hexadecimal encodings
  • Recognize and differentiate popular TCP/IP protocols
  • Know how to carve transferred files out of ICMP, DNS, and HTTP packet captures
  • Understand practical single- and multi-byte XOR encoding/decoding
  • Know techniques to approach guessed plaintext attacks against basic obfuscation
  • Understand cryptographic keyspace
  • Understand the significance of entropy in cryptanalysis
  • Be able to use protocol design knowledge and deobfuscation attacks to decode undocumented and obfuscated network protocols
  • Recognize and be able to leverage multiple tool chains to achieve the same result
  • Understand the fundamental structure and encoding techniques used in the SUNBURST DGA

Typical Agenda

Day 0
90m
  • Base arithmetic
  • Introduction to Network Protocols
90m
  • Byte structure of TCP/IP
  • Encoding schemes
90m
  • Examining packet captures
  • Extracting transferred data from packet captures
90m
  • Attack techniques against weak encryption
  • Helpful tools for Network Archaeology
Day 1
90m
  • Entropy as it relates to cryptography
  • Application-layer protocol tunneling
90m
  • Using sequencing meta-information to reconstruct transferred information
90m
  • Analysis and decoding of novel binary protocols with no prior knowledge
90m
  • Attacking novel compression with no prior knowledge
  • Attacking novel weak cryptography with no prior knowledge
Day 2
90m
  • SUNBURST Domain-Generation Algorithm
  • Base64 vs. Base32
  • Endianness issues (Esab32)
  • SUNBURST monoalphabetic substitution cipher
90m
  • Parsing of domain lists
  • First-pass decode of SUNBURST DGA
90m
  • Accurate prediction of Esab32 vs substitution
  • Decoding GUID
90m
  • Chaining Esab32 fragments
  • Dealing with errors
  • Correlating DGA domains for final decode

Setup

You will need a computer with a modern web browser, and a Linux command line (preferably bash). If you need a suggestion on which Linux distribution, our team typically recommends Ubuntu, but this isn’t a requirement.

Many participants choose to use a dedicated VM for the class. Whether you do the same or bring a laptop with Linux installed as the native OS is up to you. In the past some participants have run into issues installing software at the event due to corporate policies, slow hotel wifi, or other challenges. We will be unable to help overcome most of these issues, so we strongly encourage you to complete your setup prior to your departure.

In any case, please have the following packages pre-installed PRIOR to the start of class:

  • wireshark
  • tcpflow
  • tcpdump
  • python3
  • A C build toolchain:
    • apt install build-essential on Ubuntu / Debian / Mint
    • yum groupinstall 'Development Tools' on Red Hat / CentOS
  • The Fluffy Suite (Install from source using the linked instructions)

Other Operating Systems

If you really know what you’re doing, you can complete this class with MacOS or Windows. Be prepared to figure out your OS quirks on your own, however. Windows users should be prepared to write a lot of code, as our command-line recipes won’t work at all in Windows.

A further note about modern Apple laptops: Recent Apple laptops (those with Apple processors) use an ARM architecture. Many participants have arrived planning to install virtualization software and use common VMs only to find that this does not work, as many common VMs are not ARM compatible. All of the above software should be available either as a Mac package or able to be built and installed on ARM. Homebrew can be helpful for this, and Kali does offer an installer image specifically for Apple Silicon (ARM64). It should be noted that both MacOS and kali default to zsh - however, bash is our recommended shell, so we encourage you to make that switch. If you are wondering why, consider trying the commands we demonstrate in both shells and observing the difference. As previously noted, our staff will not have the time to help you work through this setup, so please complete it prior to arrival.