PrismML Shrinks Qwen3.8 27B Model to Just 5.9 GB

PrismML, a Caltech spinout backed by Google, Samsung, Khosla Ventures, and Cerberus, has launched Ternary Bonsai 2 27B under an Apache 2.0 license. This model is a highly compressed version of Qwen3.8 27B, utilizing ternary quantization to map weights to -1, 0, or +1 with group-wise FP16 scaling. Operating at 1.76 effective bits per weight, the model fits into a 5.9 GB footprint, which is more than nine times smaller than the full-precision original. Despite this drastic reduction, it retains 98.2 percent of its base performance, scoring 83.9 overall on benchmarks compared to the base model's 85.4.

The compression preserves capabilities unevenly across different tasks. While math and coding benchmarks saw minor declines of 0.49 and 0.59 points respectively, and instruction-following actually rose by 1.41 points, vision and agentic tool use experienced larger drops, including a 2.17-point decline in agentic performance. This release improves upon the original Bonsai 27B launched two months prior, which only retained 95 percent of base performance. The new version is designed to handle long-horizon agentic behavior, computer-use loops, and multimodal tasks with a theoretical context limit of 262K tokens.

For practitioners, this compression allows running a 27-billion-parameter model on consumer hardware typically restricted to smaller models. PrismML provides custom CUDA kernels for NVIDIA GPUs and MLX kernels for Apple devices. The model achieves generation speeds of 143 tokens per second on an NVIDIA GeForce RTX 5090 and 44 tokens per second on an Apple M5 Max. On an RTX 4090, it consumes 0.581 mWh per token, which the company claims is 40 percent more energy-efficient than a standard 8B full-precision model. However, developers must account for additional memory overhead from activations and key-value caches, which makes the maximum context window impractical on some local devices.