Document ID: docs/research/TERNARY_INFERENCE_CROSS_ARCH_BREAK_EVEN_REPORT.md
Date: April 2026
Status: Frozen / STOP (Closed via Synthesis Report TERNARY_RESEARCH_CLOSURE_AND_NEXT_HYPOTHESIS.md)
Target Subsystem: experiments/packed_ternary_matvec/
CTest Targets: t81_packed_ternary_matvec_oracle_test, t81_packed_ternary_matvec_arm64_qemu_test
This study establishes the measurable, empirical break-even conditions between 2-bit packed ternary matrix-vector inference ($0.25 \text{ bytes/weight}$) and INT8 vectorized inference ($1.0 \text{ byte/weight}$) across x86_64 (AVX2/FMA) and ARM64 (NEON) SIMD architectures.
Following PR #548, which introduced a VPSHUFB SIMD-native decoding kernel on x86_64, we audited correctness, implemented adversarial boundary/pattern verification against an independent double-precision oracle, evaluated scaled quantized inference conditions, and implemented a portable ARM64 NEON candidate.
Across all measured matrix sizes ($64 \times 256$ to $4096 \times 4096$), working-set sizes (L1, L2, L3, DRAM), and vector architectures (AVX2, NEON):
vqtbl1_s8) requires 16-bit and 32-bit unpack/zip shuffling overhead that keeps 2-bit packed execution significantly slower than native INT8 NEON (vmlaq_f32 / vdot_s32). A custom software 2-bit bit-packed CPU kernel is not technically justified as a portable inference standard.| Parameter | x86_64 Environment | ARM64 Environment (QEMU User Emulation) |
|---|---|---|
| CPU Model | Intel(R) Xeon(R) Processor @ 2.30GHz (4 cores) | QEMU User Emulation (qemu-aarch64 8.2.2) |
| Compiler | GCC 13.3.0 (gcc-13 Ubuntu 13.3.0-6ubuntu2) |
Cross GCC 13.3.0 (aarch64-linux-gnu-g++) |
| Compilation Flags | -O3 -mavx2 -mfma |
-O3 -march=armv8-a+simd |
| Cache Architecture | L1d 128 KiB, L2 1 MiB, L3 45 MiB | Virtualized QEMU environment |
Note on ARM64 Environment: Metrics collected under qemu-aarch64 measure software dynamic binary translation latency and QEMU host dispatch rather than physical ARM silicon cycle counts. These metrics validate algorithmic correctness and instruction sequence validity, but must not be presented as physical ARM hardware execution speeds.
All kernels (scalar FP32/INT8, AVX2 FP32/INT8, NEON FP32/INT8, 1-byte conditional/branchless, 2-bit scalar on-the-fly, pre-unpacked batch, SWAR, 4-trit LUT, AVX2 SIMD, NEON SIMD) were audited and verified against matvec_oracle_f64 (double-precision reference):
random, all_pos (+1), all_neg (-1), all_zero (0), alternating (+1, -1), checkerboard (+1, -1, 0), and boundary_isolated.10 ($0\text{x}02$) decodes safely to $0$ across all kernels without faulting or contaminating accumulations.| Condition / Dimension | Packed Ternary (2-bit SIMD) | INT8 Vectorized (1-byte SIMD) | Winning Format | Empirical Cause / Technical Barrier |
|---|---|---|---|---|
| Execution Speed (L1/L2 Cache) | $11.70 \text{ GFLOP/s}$ ($2.80 \mu s$) | $16.04 \text{ GFLOP/s}$ ($2.04 \mu s$) | INT8 | L1 cache bandwidth is abundant; 2-bit bit-shuffle decode tax slows execution. |
| Execution Speed (L3 Bound) | $12.77 \text{ GFLOP/s}$ ($41.07 \mu s$) | $18.42 \text{ GFLOP/s}$ ($28.46 \mu s$) | INT8 | L3 cache bandwidth is sufficient; INT8 _mm256_cvtepi8_epi32 operates in single cycle. |
| Execution Speed (DRAM Bound) | $12.91 \text{ GFLOP/s}$ ($2.59 \text{ ms}$) | $17.15 \text{ GFLOP/s}$ ($1.96 \text{ ms}$) | INT8 | On x86 CPUs, DRAM read bandwidth savings ($4\times$) are offset by decode instruction count. |
| Total Weight Memory Footprint | $0.25 \text{ bytes/weight}$ ($4.19 \text{ MB}$ for $4096^2$) | $1.00 \text{ byte/weight}$ ($16.78 \text{ MB}$ for $4096^2$) | Packed Ternary | Direct 2-bit bit packing ($4 \text{ trits / byte}$). |
| Weight Read Bandwidth Needed | $1.63 \text{ GB/s}$ ($4096^2$) | $8.59 \text{ GB/s}$ ($4096^2$) | Packed Ternary | $4.19 \text{ MB}$ weight buffer read vs $16.78 \text{ MB}$ INT8 buffer read per MatVec pass. |
| Single-Use (Pack + Run) Cost | $209.18 \text{ ms}$ ($4096^2$) | $1.96 \text{ ms}$ ($4096^2$) | INT8 | On-the-fly packing cost ($102.69 \text{ ms}$) dominates single-use inference. |
| ARM64 NEON Speed | $0.06 \text{ GFLOP/s}$ (Emulated) | $0.07 \text{ GFLOP/s}$ (Emulated) | INT8 | NEON vqtbl1_s8 table lookups require multi-instruction vzip unpacking. |
| Workload Tier | Size ($M \times K$) | Kernel Variant | Latency ($\mu s$) | GFLOP/s | Read BW (GB/s) | Footprint |
|---|---|---|---|---|---|---|
| Cache-Resident | $64 \times 256$ | 1b. FP32 AVX2 | $1.57$ | $20.89$ | $42.60$ | $65.5 \text{ KB}$ |
| 2b. INT8 AVX2 (Scaled) | $2.04$ | $16.04$ | $8.65$ | $16.4 \text{ KB}$ | ||
| 4d. Ternary 2-bit LUT | $10.90$ | $3.01$ | $0.49$ | $4.1 \text{ KB}$ | ||
| 4e. Ternary 2-bit AVX2 (Scaled) | $2.80$ | $11.70$ | $1.92$ | $4.1 \text{ KB}$ | ||
| Medium (L3) | $512 \times 512$ | 1b. FP32 AVX2 | $28.53$ | $18.37$ | $36.89$ | $1.05 \text{ MB}$ |
| 2b. INT8 AVX2 (Scaled) | $28.46$ | $18.42$ | $9.36$ | $262.1 \text{ KB}$ | ||
| 4d. Ternary 2-bit LUT | $180.55$ | $2.90$ | $0.39$ | $65.5 \text{ KB}$ | ||
| 4e. Ternary 2-bit AVX2 (Scaled) | $41.07$ | $12.77$ | $1.70$ | $65.5 \text{ KB}$ | ||
| Memory-Bound (DRAM) | $4096 \times 4096$ | 1b. FP32 AVX2 | $5794.41$ | $5.79$ | $11.59$ | $67.10 \text{ MB}$ |
| 2b. INT8 AVX2 (Scaled) | $1956.76$ | $17.15$ | $8.59$ | $16.78 \text{ MB}$ | ||
| 4d. Ternary 2-bit LUT | $11791.05$ | $2.85$ | $0.36$ | $4.19 \text{ MB}$ | ||
| 4e. Ternary 2-bit AVX2 (Scaled) | $2598.21$ | $12.91$ | $1.63$ | $4.19 \text{ MB}$ |
For $16,777,216$ parameters ($4096 \times 4096$):
Pre-packing weights during model compilation or loading is mandatory for prepacked inference ($2.59 \text{ ms}$ vs $209.18 \text{ ms}$). However, even with prepacked weights, 2-bit AVX2 decoding consumes extra ALU cycles per vector compared to INT8 vector sign-extension (_mm256_cvtepi8_epi32).
Evaluating row scale factors ($y_i = \text{scale}i \cdot \sum{j} w_{i,j} x_j$) confirms that applying a single scalar multiplication per row adds negligible overhead ($<0.5\%$) to both INT8 and 2-bit SIMD kernels. The bottleneck remains the ratio of bit-shuffle decode instructions to arithmetic accumulation instructions inside the inner loop.