Building the End-to-End Pipeline

The previous posts covered individual layers — modulation (Part 2), protocol framing (Part 3), and frequency detection (Part 4). This post shows how they connect into a complete transmit and receive pipeline, how the codebase is structured, and how the CLI and library API expose the system.

System Flow

The full data path through NearWave:

graph LR
    A[Data] --> B[FEC Encode]
    B --> C[Frame]
    C --> D[Modulate]
    D --> E[Audio Output]
    E --> F[Microphone]
    F --> G[Detect Preamble]
    G --> H[Demodulate]
    H --> I[FEC Decode]
    I --> J[Data]

The top half (A→E) is the transmit pipeline. The bottom half (F→J) is the receive pipeline. They share no state — the only link between them is the audio signal traveling through air.

Transmit Pipeline

The sender takes raw bytes and produces audio:

data → [crypto] → CRC32 → build header → Hamming encode → frame → modulate → audio

Step by step:

Encryption (optional) — If EnableEncrypt is set, the payload is AES-GCM encrypted. The encryption key must be pre-shared; NearWave doesn’t negotiate keys over the audio channel. Handled by /internal/crypto/.
CRC32 — Computed over the (possibly encrypted) payload. Stored in the frame header. Handled by /internal/fec/.
Header construction — The 9-byte header is assembled: payload length, modulation type, CRC32, original bit length. Handled by /internal/protocol/.
Hamming(7,4) encoding — Applied separately to the header and payload. Expands data by 75%. Handled by /internal/fec/.
Frame assembly — Preamble (32 bits) + encoded header + encoded payload + footer (16 bits). Handled by /internal/protocol/.
Modulation — The frame bitstream is converted to audio samples. Each group of bits maps to a frequency via the tone map, generating sine waves with guard intervals. Handled by /internal/modulation/.
Audio output — Samples are either written to a WAV file (/internal/wav/) or streamed to the speaker via PortAudio (/internal/audio/).

The sender orchestration lives in /internal/sender/. It calls each layer in sequence and produces the final audio output.

Receive Pipeline

The receiver takes audio and produces bytes:

audio → preamble detect → demodulate symbols → Hamming decode → verify CRC → [decrypt] → data

Step by step:

Audio input — Samples come from a WAV file (/internal/wav/) or a live microphone via PortAudio (/internal/audio/). Live input uses a ring buffer to handle continuous capture while processing proceeds.
Preamble detection — The receiver demodulates symbols continuously and watches for the 32-bit preamble pattern. On match, it transitions to frame-reading mode. Handled by /internal/protocol/.
Header demodulation and decoding — The next symbols are demodulated and Hamming-decoded to recover the 9-byte header. The receiver now knows the payload length, modulation type, CRC32, and original bit count.
Payload demodulation and decoding — Exactly the number of symbols specified by the header are demodulated and Hamming-decoded. Handled by /internal/modulation/ and /internal/fec/.
CRC32 verification — The decoded payload’s CRC32 is compared against the value from the header. Mismatch means the frame is discarded. Handled by /internal/fec/.
Decryption (optional) — If encryption was enabled, AES-GCM decryption is applied. Handled by /internal/crypto/.

The receiver orchestration lives in /internal/receiver/.

Package Architecture

Each pipeline stage maps to a package with clear boundaries:

/internal/
    sender/         Orchestrates transmit pipeline
    receiver/       Orchestrates receive pipeline
    protocol/       Frame construction, preamble detection
    modulation/     BFSK, MFSK tone generation and demodulation
    fec/            Hamming(7,4) encode/decode, CRC32
    dsp/            Goertzel filter, Hann windowing, bandpass
    audio/          PortAudio streaming, ring buffer
    wav/            WAV file read/write
    config/         Profile definitions (reliable, fast, ultrasonic)
    crypto/         AES-GCM encrypt/decrypt

Dependencies flow downward. sender/ depends on protocol/, modulation/, fec/, and crypto/. modulation/ depends on nothing except config/. dsp/ depends on nothing. No circular dependencies.

Ring Buffer

Live microphone input is continuous — samples arrive at 44100 Hz regardless of what the receiver is doing. The ring buffer in /internal/audio/ decouples capture from processing:

PortAudio writes samples into the ring buffer in a callback
The receiver reads from the ring buffer at its own pace
The buffer is sized to hold several seconds of audio, accommodating processing latency

This means the receiver can take time to decode a frame header without losing subsequent audio data. The ring buffer is lock-free for the single-producer, single-consumer case.

WAV Mode vs. Real-Time Mode

NearWave supports two modes with identical processing paths:

WAV mode — Data is encoded into a .wav file and decoded from a .wav file. This is deterministic, reproducible, and used for testing. No audio hardware needed.

Real-time mode — Data is played through the speaker and captured from the microphone via PortAudio. This is the production path for actual device-to-device transfer.

The sender and receiver don’t know which mode they’re in — they work with sample arrays. The I/O layer (wav/ or audio/) handles the source/destination.

CLI Design

The CLI (/cmd/nsdt/) exposes four commands:

send — Encode data to a WAV file:

go run ./cmd/nsdt send --input message.txt --output message.wav

receive — Decode data from a WAV file:

go run ./cmd/nsdt receive --input message.wav --output decoded.txt

play — Encode and play through speakers (requires PortAudio):

go run ./cmd/nsdt play --input message.wav

listen — Capture from microphone and decode (requires PortAudio):

go run ./cmd/nsdt listen --output received.txt

Flags apply to all commands:

Flag	Description
`--profile`	`reliable` (default), `fast`, `ultrasonic`
`--encrypt`	AES encryption key (16/24/32 bytes)
`--debug`	Enable debug logging

Library API

The public API in nsdt.go wraps the pipeline in two functions:

// Encode data to a WAV file
err := nsdt.EncodeToWav(data, "message.wav", cfg)

// Decode data from a WAV file
decoded, err := nsdt.DecodeFromWav("message.wav", cfg)

Configuration is explicit:

cfg := nsdt.DefaultConfig()             // Reliable profile
cfg.Profile = "fast"                     // Switch profile
cfg.EnableEncrypt = true                 // Enable encryption
cfg.EncryptionKey = []byte("16-byte-key!")

No global state. No init functions. The config struct carries all parameters, and the encode/decode functions are stateless.

Testing Strategy

The WAV round-trip is the primary integration test:

func TestRoundTrip(t *testing.T) {
    data := []byte("test payload")
    cfg := nsdt.DefaultConfig()

    err := nsdt.EncodeToWav(data, "test.wav", cfg)
    require.NoError(t, err)

    decoded, err := nsdt.DecodeFromWav("test.wav", cfg)
    require.NoError(t, err)
    require.Equal(t, data, decoded)
}

This exercises the full pipeline — FEC, framing, modulation, WAV I/O, demodulation, FEC decode, CRC check — without audio hardware. Tests run in CI with go test ./....

Unit tests cover individual packages: Hamming encode/decode correctness, Goertzel magnitude accuracy, preamble detection against synthetic bitstreams, CRC32 validation.

Debug Mode

The --debug flag enables verbose output at each pipeline stage:

Detected frequencies per symbol window
Raw bitstream before and after FEC
Preamble detection confidence score
Frame header fields after decoding
CRC32 match/mismatch

This is essential for diagnosing real-world issues — is the problem in modulation, detection, or FEC? Debug output pinpoints the layer.

The final post covers performance characteristics, profile tradeoffs, and the practical constraints of transmitting data over sound in real environments.