Service Clients & Automation for Wargames: Protocol RE, Robust Solvers, and FormulaOne + Maze Tie-ins

Service Clients & Automation for Wargames

In FormulaOne/Maze (and many modern CTFs), you’re not just “solving a puzzle”—you’re speaking a protocol. This tutorial is a field guide to recon (pcap + filters), protocol reverse engineering, client building (Python sockets / asyncio / pwntools), traffic interception (mitmproxy), and robust automation (timeouts, retries, assertions, logging). The goal: stable solvers that survive flaky I/O, weird encodings, and grumpy services.

If you’re following my series, peek the official pages: OverTheWire — FormulaOne and Maze.

Table of Contents

• 1) Mindset & outcomes
• 2) Network fundamentals (pcap + filters + anatomy)
• 3) Toolbelt (CLI): nc/socat/openssl/curl/tshark
• 4) Protocol RE workflow (from pcap to spec)
• 5) Clients in Python: sockets, selectors, asyncio
• 6) pwntools automation patterns
• 7) Interception & replay: mitmproxy + addons
• 8) Robustness: timeouts, retries, state machines, logging
• 9) Case patterns you’ll meet (and templates)
• 10) FormulaOne + Maze tie-ins (non-spoiler)
• Appendix A — One-liners & snippets
• Appendix B — Mini-labs (safe, local)
• Appendix C — TLS, certs & local MITM
• Appendix D — Field checklist (printable)
• Resource Library (Videos & Reading)

1) Mindset & outcomes

• Talk like a client. Identify request/response boundaries, encodings, framing, and error codes.
• Determinize the flow. Make prompts and end-conditions explicit (recvuntil, checksums, terminators).
• Automate early. Build a client as you recon; don’t wait for “full spec.”
• Log everything. Save pcaps, transcripts, and solver versions per attempt.

Orientation video Wireshark (fundamentals) — great when you’re new to captures

Further references

• Display-Filter Quickref (you’ll use this constantly)
• Tshark CLI (scriptable Wireshark)
• IANA Port Registry (sanity checks for port meanings)

2) Network fundamentals (pcap + filters + anatomy)

What to extract from a trace:

• Transport: TCP or UDP? Retries? Fragmentation?
• Framing: line-based (\n/\r\n), length-prefixed, or delimiter-based?
• Encoding: ASCII, hex, base64, JSON, binary structs (endianness!).
• State: challenge → response → flag; or multi-round handshake.

Core filters (Wireshark/TShark display filters):

tcp.stream eq 3
ip.addr == 10.0.2.15 && tcp.port == 31337
frame contains "Welcome"
tls.handshake.type == 1    # ClientHello

Capture filters (pcap/BPF syntax; use at capture time):

tcp port 31337 or udp port 31337
host 10.0.2.15 and not port 22

Video Socket basics with Python — mental model for what you’re scripting

Further references

• Wireshark User’s Guide • Display Filter Reference
• Capture Filter syntax (pcap-filter)
• “Service Names & Port Numbers” (IANA)

3) Toolbelt (CLI): nc/socat/openssl/curl/tshark

• nc / ncat: banner grab, quick sanity.
• socat: swiss-army relay, also for UDP, PROXY, and weird TTYs.
• openssl s_client: peek inside TLS endpoints.
• curl: HTTP(S) w/ headers, cookies, and binary upload/download.
• tshark: headless Wireshark—perfect for CI and scripted parsing.

# Handshake a TLS service
openssl s_client -connect host:443 -servername host </dev/null 2>/dev/null | head

# Scriptable capture
tshark -r traffic.pcapng -Y 'tcp.stream==2 && frame contains "flag"' -T fields -e frame.number -e data

Tip: For line-based puzzles, rlwrap -cAr nc HOST PORT gives history and colors; stdbuf -oL disables buffering in pipelines.

4) Protocol RE workflow (from pcap to spec)

1. Segment flows by tcp.stream/udp.stream.
2. Annotate: each request/response, delimiters, magic bytes, checksums.
3. Hypothesize a grammar: tokens, numbers (endian/size), records.
4. Validate with tshark/pyshark extraction → small script to replay.
5. Freeze a mini spec (README.md): message formats + state diagram.

Video PyShark + Wireshark dissectors = programmable parsing

Further references

• PyShark (Python wrapper around TShark)
• Scapy (craft/replay packets)
• Wireshark Display Filter Reference

5) Clients in Python: sockets, selectors, asyncio

Blocking sockets — simplest baseline, easiest to debug.

#!/usr/bin/env python3
import socket, sys

def recvuntil(s, token: bytes, max_len=1<<20, timeout=5):
    s.settimeout(timeout)
    buf = b''
    while token not in buf:
        chunk = s.recv(4096)
        if not chunk: break
        buf += chunk
        if len(buf) > max_len: raise RuntimeError("too much data")
    return buf

host, port = sys.argv[1], int(sys.argv[2])
s = socket.create_connection((host, port), timeout=5)
banner = recvuntil(s, b'\n')
s.sendall(b'HELLO\n')
print(recvuntil(s, b'OK\n').decode('utf-8', 'replace'))

selectors — multiplex multiple sockets without threads.

import selectors, socket
sel = selectors.DefaultSelector()
def connect(addr):
    s = socket.create_connection(addr); s.setblocking(False)
    sel.register(s, selectors.EVENT_READ|selectors.EVENT_WRITE, data={'buf':b''})
    return s

asyncio streams — concise, great for timeouts & backoff.

import asyncio
async def client(host, port):
    reader, writer = await asyncio.open_connection(host, port)
    await reader.readline()              # banner
    writer.write(b'PING\n'); await writer.drain()
    line = await asyncio.wait_for(reader.readline(), timeout=3)
    print(line)
    writer.close(); await writer.wait_closed()
asyncio.run(client('127.0.0.1', 31337))

Video Hands-on sockets refresher (great for mental model)

Further references

• Python socket, selectors, asyncio Streams HOWTO
• RealPython sockets guide (modern patterns)

6) pwntools automation patterns

Why pwntools? recvuntil/sendlineafter, easy remote/local toggle, ELF helpers, tube abstractions, and logging.

Template: local ↔ remote + asserts

#!/usr/bin/env python3
from pwn import *
context.log_level = 'info'
exe = './solver'     # your helper, optional
HOST, PORT = os.getenv('HOST','localhost'), int(os.getenv('PORT','31337'))

def start():
  return remote(HOST, PORT) if args.REMOTE else process([exe]) if os.path.exists(exe) else remote(HOST, PORT)

io = start()
io.recvuntil(b'> ')
io.sendline(b'1')
line = io.recvline(timeout=3)
assert b'OK' in line, f"unexpected: {line!r}"
# ... more rounds
io.interactive()

Stability tricks

• context.timeout global default; add per-recv guard rails.
• Wrap steps with assertions; fail fast & retry.
• Use with context.local(log_level='error') for noisy steps.

Video Overflow walkthrough (demonstrates the value of scripts & guards)

Further references

• pwntools docs • ROPgadget (later)
• one_gadget (libc constraints; later)

7) Interception & replay: mitmproxy + addons

When a service uses HTTP(S)/WebSockets, mitmproxy = live view, modification, and scripting.

Quick start

# Install and run
mitmproxy -p 8080
# Point your client at HTTP proxy http://127.0.0.1:8080 and install mitm CA (docs)

Addon skeleton (Python)

# save as addons/log_all.py, run: mitmdump -s addons/log_all.py
from mitmproxy import http
def request(flow: http.HTTPFlow):  # or response(flow)
    if flow.request.host.endswith("challenge.local"):
        flow.request.headers["X-Auto"] = "1"

Video Capture, analyze, and debug HTTPS traffic with mitmproxy

Further references

• mitmproxy docs (proxy modes, certs, addons)
• “Wireshark & SSL/TLS” HOWTO (pair with mitm)

8) Robustness: timeouts, retries, state machines, logging

• Timeouts: per-read and overall; exponential backoff on reconnect.
• State machine: explicit states (HELLO → CHALLENGE → ANSWER → DONE).
• Idempotency: safe to re-run on partial progress.
• Telemetry: JSONL logs per run (ts, state, rx, tx, duration).
• Artifacts: save pcaps and transcripts on failure.

import time, json, pathlib
LOG = pathlib.Path('runs')/f'{int(time.time())}.jsonl'
def log(ev, **kw):
    LOG.parent.mkdir(exist_ok=True)
    with LOG.open('a') as f: f.write(json.dumps({'t':time.time(),'ev':ev,**kw})+'\n')

9) Case patterns you’ll meet (and templates)

A) Line-based math quiz

• recvuntil(b': ') → parse integers → compute → sendline(str(ans).encode()).
• Watch for Unicode (UTF-8) and CRLF (\r\n).

B) Length-prefixed binary messages

• Read 4 bytes (LE) → struct.unpack('<I', hdr)[0] → read exact body.
• Add short-read loop to gather full body.

C) Base64 / hex dance

• Detect via prompts or alphabet; prefer Python’s base64 / binascii.

D) Checksummed payloads (CRC32)

• Compute with binascii.crc32(data) & 0xffffffff, respect endianness.

E) TLS endpoints (custom app on 443)

• Try openssl s_client for sanity; if custom framing, then Python ssl.create_default_context() + wrap the socket.

10) FormulaOne + Maze tie-ins (non-spoiler)

• FormulaOne: many tasks act like quirky network daemons—client discipline (read prompts, exact formatting, timeouts) wins.
• Maze: mixes RE + exploitation; your pcap-first habit + deterministic clients pay off when services are flaky or multi-stage.
• Build one solver skeleton and reuse it per service.

Appendix A — One-liners & snippets

Banner & timing

nc -v host 31337
time ( printf 'HELLO\n' | nc host 31337 )

socat relays

# TCP→TCP relay (debug with -v)
socat -v TCP-LISTEN:9000,reuseaddr,fork TCP:host:31337
# UDP client
socat - UDP:host:31337

openssl TLS peek

openssl s_client -connect host:443 -servername host </dev/null | openssl x509 -noout -issuer -subject

tshark extract

tshark -r cap.pcapng -Y 'tcp.stream==1' -T fields -e frame.number -e data

PyShark “first flag-looking line”

import pyshark
cap = pyshark.FileCapture('cap.pcapng', display_filter='tcp && frame contains "FLAG"')
print(cap[0].frame_info.number)

Scapy craft/replay

from scapy.all import IP,TCP,Raw,send
pkt = IP(dst="host")/TCP(dport=31337,flags="PA")/Raw(load=b"HELLO\n")
send(pkt, verbose=0)

pwntools toggles

io = remote(HOST, PORT) if args.REMOTE else process([exe])
io.sendlineafter(b'> ', b'1')

Appendix B — Mini-labs (safe, local)

Do these on your own VM; use a throwaway network namespace if possible.

Lab B1 — Record → RE → Replay

1. Start an echo-ish server (Python):

# server.py
import socket, threading, base64
def h(conn):
    conn.sendall(b'Welcome\n')
    while True:
        d = conn.recv(4096)
        if not d: break
        if d.strip() == b'ENC':
            conn.sendall(base64.b64encode(b'secret-data')+b'\n')
        else:
            conn.sendall(b'OK\n')
    conn.close()
s = socket.socket(); s.bind(('0.0.0.0',31337)); s.listen()
print('listening 31337')
while True:
    c,_=s.accept()
    threading.Thread(target=h,args=(c,),daemon=True).start()

1. Capture with tcpdump -i lo -w cap.pcapng tcp port 31337.
2. RE with Wireshark/PyShark; write a client that sends ENC and parses base64.

Lab B2 — Line-based → length-prefixed pivot

Modify the server to send len(4 bytes LE) || body; adjust client to read exact bytes.

Lab B3 — TLS wrap

Wrap server with stunnel or run a simple Python ssl server; connect with openssl s_client then your Python TLS client.

Appendix C — TLS, certs & local MITM

• Local TLS for labs: self-signed certs (OpenSSL).
• Client pinning gotchas: some apps pin certs; mitmproxy may need upstream certs or be impossible—fall back to pcap at the OS layer.
• HTTP(S) services: mitmproxy intercept → modify → replay flows; write small addons to automate boring clicks / header surgery.

Mini how-to

# Self-signed cert
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -subj "/CN=localhost" -days 365
# mitmproxy as reverse proxy to local target
mitmproxy --mode reverse:http://127.0.0.1:8080 -p 9000

Appendix D — Field checklist (printable)

Resource Library (Videos & Reading)

Videos (curated)

• Wireshark fundamentals: Wireshark Tutorial for Beginners — Chris Greer
• Sockets (Python): Python Socket Programming — Corey Schafer
• PyShark analysis: Analyzing Captures in Python with PyShark — Dor Green / PyCon IL
• MitM for HTTP(S): Capture & Debug HTTPS with mitmproxy — Tutorial
• PyShark quick intro: EASY PyShark Tutorial — ByteByteGo-like walk-through
• MITM deep dive playlist: Learn mitmproxy — QAInsights Playlist

Reading / Tools

• Wireshark User’s Guide — docs
• Display Filter Reference — dfref
• TShark — User’s Guide sections
• pcap/BPF filter syntax — pcap-filter manpage
• PyShark — PyPI • Intro guide
• Scapy — docs
• mitmproxy — docs (modes, certs, addons)
• pwntools — docs
• Python sockets & multiplexing — socket • selectors • asyncio streams
• IANA port registry — Service Names & Ports

Final note

Think pcap → spec → tiny client → robust solver. Once you can speak the service’s language deterministically, everything else (RE, exploitation, crypto mini-puzzles) becomes a lot less scary. Tie this playbook to FormulaOne and Maze and you’ll ship solvers that just… work.

Thanks for reading!

Until next time — Otsumachi!! 💖☄️✨