I built this to challenge the assumption that we are strictly bound by the "Memory Wall." My hypothesis was that modern consumer silicon (like Apple M-series) has enough spare compute to decompress weights procedurally faster than it can read them from RAM.

The Architecture: It uses a "Predator-Prey" method:

Predators: We identify and preserve the high-magnitude outliers (the "Alpha" weights) in FP16.

Prey: The remaining weights are compressed into ternary masks {-1, 0, 1} and a block-wise scalar.

Reconstruction: A bitwise kernel reconstructs the layer in L2 cache during the forward pass.

Results: I validated this on Llama-3-8B (Layer 20, SwiGLU).

Compression: ~3.0 bits per weight (effective).

Fidelity: 0.915 Cosine Similarity (weights) / 0.912 (outputs).

Size: Brings an 8B model down to ~3GB.

The repo is currently a "Proof of Engine" in Python/MLX. The math works, but to realize the theoretical speed gains (125 t/s), I am working on porting the decompression kernels to Metal/CUDA.

Happy to answer questions about the compression logic or the "Predator" selection algorithm!