Overview

Vesta is a CPU inference engine for ternary neural networks — models whose weights are restricted to -1, 0, or +1. It has no dependencies: no GPU, no PyTorch, no TensorFlow, no BLAS. Just C++ and hand-written SIMD kernels that compile and run directly on the CPU.

Core Idea

Ternary weights compress extremely well: 64 of them fit into two 64-bit integers, one bitmask marking every +1 position and the other marking every -1 position. A position with neither bit set is a 0. That representation turns matrix-vector multiplication into bitwise operations instead of floating-point multiplies, and it’s what makes the inference core fast without any hardware acceleration.

On a Snapdragon X laptop with ARM NEON, a 1024x1024 ternary matrix-vector multiply finishes in about half a millisecond — roughly 4 billion operations per second, about 5.7x faster than the scalar fallback. The SIMD and scalar paths produce identical checksums, so the speedup isn’t coming from cutting corners.

How It Works

Packing — a Python packer quantizes float32 weights to ternary using error-compensated rounding (the rounding error from one weight carries into the next, preserving the overall signal) and writes two bit-planes per row into a .bits file.

Loading — the C++ loader reads the .bits header, allocates 64-byte aligned memory, and maps the bit-planes directly. No file I/O happens during inference after that.

Matvec — the SIMD kernel processes each row in 64-column chunks, one positive bitmask and one negative bitmask per chunk. It loads several input values at once, uses a bitwise compare to build a pick mask, and conditionally adds or subtracts — with no data-dependent branches in the hot loop.

Transformer decode — a token is embedded, passed through one or more transformer layers (RMSNorm, ternary Q/K/V projections, rotary position embeddings from a precomputed table, a KV cache, multi-head causal attention, a feedforward layer with ReLU, and residual connections), then projected back to vocabulary space by the LM head.

Features

Four SIMD backends — ARM NEON, AVX2, AVX-512, and a scalar fallback, all verified to produce identical checksums; force the scalar path with VESTA_BACKEND=scalar to confirm the speedup is real
KV cache — single-head and multi-head variants for causal decoding, resettable between sequences
Multi-head attention — scaled dot-product attention with causal masking, computed independently per head
RoPE — rotary position embeddings precomputed into lookup tables, no transcendental calls during inference
Three tokenizers behind one facade — word-level, char-level, and byte-level BPE, with deterministic selection and explicit errors instead of silent fallbacks
A pure-Python trainer — standard-library only, trains a tiny transformer and exports a complete .bits model directory, proving the pipeline end to end
A benchmark harness — every benchmark runs with warmup and repeats, reporting median, p95, throughput in GOP/s, and a checksum to catch incorrect kernels

Benchmarks

Measured on a Snapdragon X laptop with ARM NEON, compiled with -O3:

Benchmark	Median (ms)	GOP/s	Status
Linear 1024x1024	0.54	3.92	ok
Linear (scalar reference)	3.06	0.69	ok
Batched 4x1024	1.84	4.57	ok
Decode 128x32	1.27	4.97	ok
MHA decode 128x4h	1.31	4.80	ok
Transformer block (256)	0.15	5.14	ok

Checksums match exactly between the SIMD and scalar paths across every benchmark.

Philosophy

The project stays intentionally narrow: if a feature doesn’t help ternary weights run faster, it belongs in a Python script outside the core, not in the inference engine itself. The core is one header, avoids std::vector in hot loops, and keeps memory cache-aligned throughout.

Status

Not a full language model — the tokenizer and training script are intentionally small — but the inference core is real and produces verified-correct results on both ARM and x86.