Post
2,028
Ternary Bonsai 2 27B: 2.7× faster on a Mac, 5.9 GB on DGX Spark ⚡
gbx v0.8.2:
• 114 tok/s on M3 Ultra, 2.7× the official MLX runtime
• 62 tok/s on DGX Spark, 2.1× llama.cpp
• 27B in 5.9 GB of memory
Weights are just −1, 0, +1.
GreenBit Ternary Models, coming soon.
🤗 Model: GreenBitAI/Ternary-Bonsai-2-27B-gbx
⚙️ gbx: https://github.com/GreenBitAI/gbx-lm/releases/tag/v0.8.2
gbx v0.8.2:
• 114 tok/s on M3 Ultra, 2.7× the official MLX runtime
• 62 tok/s on DGX Spark, 2.1× llama.cpp
• 27B in 5.9 GB of memory
Weights are just −1, 0, +1.
GreenBit Ternary Models, coming soon.
🤗 Model: GreenBitAI/Ternary-Bonsai-2-27B-gbx
⚙️ gbx: https://github.com/GreenBitAI/gbx-lm/releases/tag/v0.8.2
