Network Archaeology

What to expect

We begin with several fundamental CS concepts: binary network protocols, elementary cryptanalysis, and back engineering (reverse engineering/cleanroom design). We also have an intense but occasional focus on mathematics and information theory.

People who are not deep into mathematics and computer science will still benefit from this class, but those who have a passion for theoretical CS, information theory, or mathematics, will gain the most.

The class is presented as a series of technical challenges, each teaching a concept by allowing students to invent different approaches and try them out, seeing which work well and which do not. Challenges build on each other, until, by the final challenge, students are writing custom binary protocol decoder software from scratch.

Network Archaeology is a self-paced lab class, with intermittent instructor lectures. Some participants may find the first dozen or so labs easy: they are encouraged to proceed through as quickly as they like. The instructors lead occasional “how-to” lectures, starting with the first lab, eventually bringing the class to the same point. Between lectures, instructors traverse the room helping people with labs.

This class is taught using the Linux command line. The instructor will use command-line tools, to create increasingly powerful tools, but participants can make decent progress using Wireshark (local install) and Cyber Chef (web-based tool).

Each lab exercise either introduces new concepts or builds on previously presented concepts. Very few people make it through every exercise, and there is no expectation that anyone will “finish” in two days. Many Network Archaeology attendees come back to Cyber Fire to take this class a second or even third time.

Technical Content expert

Audience

  • Computer scientists
  • Mathematicians
  • Information theorists
  • Incident investigators
  • Software engineers
  • Applied mathematicians
  • System administrators
  • Site reliability engineers

Applicability

Network Archaeology teaches students how to approach unknown data that no existing tool can handle. People expecting to walk away with a recipe book will be disappointed. Our goal is for you to gain insight about how network protocols work, how encryption works, and what common techniques can be used to “break” malware protocols.

Network Archaeology is broadly interesting to anyone who wants a better understanding of the process of network packet forensic techniques. Even if you don’t intend to engage in this activity in your job, going through the instructor-led exercises will provide insight into challenges facing your organization.

Objectives

  • Understand byte structure of TCP/IP and DNS
  • Understand the theory of Base64 encoding
  • Recognize Base64 and hexadecimal encodings
  • Recognize and differentiate popular TCP/IP protocols
  • Know how to carve transferred files out of ICMP, DNS, and HTTP packet captures
  • Understand practical single- and multi-byte XOR encoding/decoding
  • Know techniques to approach guessed plaintext attacks against basic obfuscation
  • Understand cryptographic keyspace
  • Understand the significance of entropy in cryptanalysis
  • Be able to use protocol design knowledge and deobfuscation attacks to decode undocumented and obfuscated network protocols
  • Recognize and be able to leverage multiple tool chains to achieve the same result
  • Understand the fundamental structure and encoding techniques used in the SUNBURST DGA

Typical Agenda

Day 0
90m
  • Base arithmetic
  • Introduction to Network Protocols
90m
  • Byte structure of TCP/IP
  • Encoding schemes
90m
  • Examining packet captures
  • Extracting transferred data from packet captures
90m
  • Attack techniques against weak encryption
  • Helpful tools for Network Archaeology
Day 1
90m
  • Entropy as it relates to cryptography
  • Application-layer protocol tunneling
90m
  • Using sequencing meta-information to reconstruct transferred information
90m
  • Analysis and decoding of novel binary protocols with no prior knowledge
90m
  • Attacking novel compression with no prior knowledge
  • Attacking novel weak cryptography with no prior knowledge
Day 2
90m
  • SUNBURST Domain-Generation Algorithm
  • Base64 vs. Base32
  • Endianness issues (Esab32)
  • SUNBURST monoalphabetic substitution cipher
90m
  • Parsing of domain lists
  • First-pass decode of SUNBURST DGA
90m
  • Accurate prediction of Esab32 vs substitution
  • Decoding GUID
90m
  • Chaining Esab32 fragments
  • Dealing with errors
  • Correlating DGA domains for final decode

Setup

You will need a computer with a modern web browser, and a Linux command line. We recommend Ubuntu, either as your native OS, or in a virtual machine.

You should have the following packages pre-installed:

  • wireshark
  • tcpflow
  • tcpdump
  • python3
  • A C build toolchain:
    • apt install build-essential on Ubuntu / Debian / Mint
    • yum groupinstall 'Development Tools' on Red Hat / CentOS

During class, we will walk you through the installation of:

We will not be able to help anyone configure their computer, so please arrive with a properly set up machine.

Other Operating Systems

If you really know what you’re doing, you can complete this class with MacOS or Windows. Be prepared to figure out your OS quirks on your own, however. Windows users should be prepared to write a lot of code, as our command-line recipes won’t work at all in Windows.