The Engine of Detection: Building a Deep-Packet Inspection Sniffer from Scratch

Day 11: Module 1 — Project 4. After building high-level scanners, I went one level deeper: building a production-grade packet sniffer from absolute scratch. No Scapy, no libpcap—just Python raw sockets and manual binary parsing. I built a deep-packet inspection engine that tracks TCP states, decodes binary DNS questions, and detects ARP spoofing in real-time.

📍 Journey Navigation Prev: Day 10 — Module 1 Capstone: Building Tools That See Like an Attacker | Day 1 | Day 3 | Day 5 | | Day 6 | Day 7 | Day 8 | Day 9 | Day 10 |

Status: Day 11 of 90-day AI Security journey — Module 1 Project 4 (Completed) Projects Shipped: DPI Packet Sniffer (sniffer.py) GitHub: applied-ai-security-projects

Beyond the API: Why Building a Sniffer Matters

Yesterday, I spent the time building scanners—tools that send packets and interpret the responses. Today was about the other side of that coin: building the "engine" that sits quietly on the network and overhears everything.

If you use tools like Scapy or tcpdump, packet sniffing feels like magic. You tell the tool to start, and it gives you a clean list of packets. But when you're transitioning into AI security—where you'll eventually be building models to detect anomalies—you cannot afford to treat the data source as a black box. You need to understand how a raw byte-stream from a network card becomes a structured HTTP request.

So, I built Project 04: Packet Sniffer.

It’s a production-grade DPI (Deep-Packet Inspection) engine written in pure Python. No pip install scapy. No import libpcap. Just the Python standard library, raw sockets, and several hundred lines of binary decoding logic.

The Challenge: The "Hard Way" of Raw Sockets

Most Python networking happens at the Transport Layer (TCP/UDP) using higher-level abstractions. But to build a sniffer, you have to go down to the Data Link Layer.

In Linux, this means using socket.AF_PACKET.

# To capture everything (including Ethernet headers), you need CAP_NET_RAW privileges
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))

When you call sock.recvfrom(65535), you don't get a "packet object." You get a raw bytes object—a long string of hexadecimal numbers. To turn that into something human-readable, you have to manually parse it using struct.unpack.

Decoding the Ethernet Header

Every frame starts with 14 bytes: 6 for Destination MAC, 6 for Source MAC, and 2 for the EtherType.

import struct

def parse_ethernet(raw_data):
    # ! = Network (big-endian) byte order
    # 6s = 6-byte string (MAC address)
    # H = Unsigned short (2-byte EtherType)
    dst_mac, src_mac, ethertype = struct.unpack('!6s6sH', raw_data[:14])
    return dst_mac, src_mac, ethertype

This is where you realize how fragile network protocols are. If you’re off by a single byte, the entire decoding stack fails. You have to be precise with the bitmasking and the offsets.

The Feature Set: What Makes a Sniffer "Deep"

A basic sniffer prints source and destination IPs. A DPI sniffer understands the meaning behind the traffic. My implementation handles:

1. The Multi-Layer Parsing Stack

The sniffer follows the OSI model. It decodes the Ethernet frame, identifies it as IPv4, decodes the IP header, identifies the protocol (TCP/UDP/ICMP), and then dives into the application layer (DNS/HTTP). Each layer is parsed into a Python dataclass, making it easy to work with in the detection engines.

2. A Stateful TCP Machine

Stateless sniffers see individual packets. They see a SYN, then they see an ACK. My sniffer maintains a state table for every unique 4-tuple (source/dest IP and port). It tracks the RFC 793 connection lifecycle:

LISTEN → SYN_SENT → ESTABLISHED → FIN_WAIT → CLOSED. It knows when a connection is actually "alive," much like a netstat command.

3. DNS Intelligence & Label Compression

DNS packets are complex. They use a system called label compression where, instead of repeating a domain name like google.com, the packet uses a pointer to a previous occurrence of that name to save space.

Building the _parse_dns_name() function was the most technically challenging part. It requires following pointers (marked by 0xC0) recursively, while preventing infinite loops (in case of a malicious packet designed to crash a sniffer).

def _parse_dns_name(data, offset):
    # Following 0xC0 pointers to decode compressed DNS labels
    # while tracking jumps to prevent infinite loops...

Security in Transit: Building Detectors Into the Core

Because the sniffer sees every packet across all layers, it’s the perfect place to build real-time security detection.

ARP Spoofing Detector

The sniffer maintains a mapping of IPs to MAC addresses. If it sees an ARP reply claiming that an IP (like the gateway) now belongs to a different MAC address than what was previously recorded, it fires a high-severity alert. This detects the very attacks I practiced on Day 7 and 8.

Port Scan Detector

I implemented a sliding-window counter. If a single source IP probes more than 15 unique destination ports within a 5-second window, the sniffer flags a "Port Scan Detection" alert. This isn't a static rule—it's active, stateful monitoring.

Talking to Wireshark: The PCAP Writer

One of the most useful features is the integrated PCAP Writer. By manually writing the libpcap global header (0xa1b2c3d4) and prepending a 16-byte header to every captured packet (timestamp, original length, included length), I can save my captures to a .pcap file that Wireshark can open natively.

This closes the loop between "I built my own tool" and "I can use professional tools to analyze my work."

Reflections: Speed is Evasion

Building this tool reinforced a lesson from Day 10: Performance equals capability.

In my sniffer.py, the stats dashboard runs in a background thread, refreshing every 3 seconds to show throughput (pkts/s, KB/s) and top talkers. All analyzers (Connection Tracker, ARP Watcher, Port Scan Detector) use threading.Lock() to prevent race conditions.

If the sniffer drops packets because it’s too slow, the detection engine becomes useless. In the world of AI security, "data loss" is "detection gap."

Project 4: Summary

By building this sniffer, I’ve moved from being a "user" of network tools to being a "builder" of the underlying engine. I now understand exactly what it takes to:

Parse binary protocol headers precisely.
Maintain state across thousands of ephemeral connections.
Detect multi-layer attacks in real-time.
Expose that data in industry-standard formats.

This is the foundation for the next phase of the journey: moving into more complex attack chains and, eventually, building AI models that sit on top of this exact data stream.