All Tools
A
LLMFreeOpen Source
AQLM
Extreme 2-bit LLM compression via additive quantization
Apache-2.0
ABOUT
Standard quantization methods hit a quality wall below 3 bits per parameter, limiting compression on memory-constrained hardware. AQLM (Additive Quantization of Language Models) breaks through this barrier by representing each weight as a sum of vectors from multiple learned codebooks, rather than a single scalar value. This achieves 2 bits per parameter with significantly less accuracy loss than GPTQ or AWQ at comparable compression ratios. AQLM is particularly effective for deploying LLMs on edge devices, mobile hardware, or any environment where GPU memory is tightly constrained.
INSTALL
pip install aqlmINTEGRATION GUIDE
1. Compress a 13B Llama model to 2-bit and fit it on a single consumer GPU with 8GB VRAM
2. Deploy quantized LLMs on mobile devices or edge hardware with extreme memory constraints
3. Compare AQLM 2-bit compression against GPTQ 3-bit for a production model serving pipeline
4. Run a 7B model entirely in GPU memory on an RTX 3060 or similar 6-8 GB card
5. Quantize model ensembles for deployment on resource-constrained cloud instances
TAGS
quantizationcompressionllminferenceoptimizationmodel-compressioncodebook