Been building a from-scratch LLM inference engine in C99 for about a year. No Python, no BLAS, no CUDA - just GCC + make. Finally have numbers worth sharing.
BitNet b1.58-2B-4T on Intel Xeon Emerald Rapids (T=4, PGO+LTO):
- Project Zero: 36.25 tok/s
- bitnet.cpp: ~19.8 tok/s
- 1.83x faster, verified on OpenBenchmarking: https://openbenchmarking.org/result/2606207-SHIF-PROJECT42
The gap comes from a native packed-ternary representation rather than converting through GGUF quantization. Different internal formats, same model weights.
Dense model results vary a lot by platform - on an older i5-5250U macOS machine with T=4 it beats llama.cpp by around 50%, but on the more modern i5-11300H Linux machine llama.cpp wins by about 7%, which I think comes down to AVX-512 kernel tuning I haven’t finished yet.
The other thing that distinguishes it: one binary handles both BitNet ternary and standard GGUF. llama.cpp can’t load the native BitNet format; dedicated BitNet engines mostly skip GGUF. Hadn’t seen anything that did both so that’s where I focused.
Honest gap: DeepSeek Q4_K_S is about 7x behind llama.cpp on Linux. Missing a native quantized matmul kernel - using a float dequant path. Open issue in Discussion #1 if anyone’s interested in working on it.
Pre-built x86-64 Linux binary in Releases, or build with: git clone + make. Zero runtime deps.