Aleph-Alpha-Research/1BW-Llama-30B-1Bit-SFT

LLaMA 30.4B - 1-bit Quantized

📚 Paper • 🏠 GitHub

This is one of the checkpoints supplementing the paper 1-Bit-Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization. Instructions on how to use the model for inference can be found in the corresponding repository.

⚠️ IMPORTANT: This model is intended for research purposes only. It is provided as-is without warranties for production use.

Model Details

• Architecture: LLaMA
• Size: 30.4B (30,416,566,272 parameters)
• Quantization: 1-bit k-means with per-64 block absmean scaling
• Centroids: Defined per Layer

Directory Structure

.
├── config.json                  # HuggingFace model config
├── quantization_config.json     # Quantization parameters
├── generation_config.json       # Default generation settings
├── tokenizer.json              # Tokenizer files
└── model.safetensors           # Weights (quantized linear layers + bf16 embeddings/norms)

Weight Format

Linear layers are quantized:

• *.weight_packed: Packed uint8 indices (8 elements per byte)
• *.scales: Per-block scale factors (bfloat16)
• *.centroids: K-means centroids for this layer (float32)

Embeddings and norms are stored in bfloat16.

Citation

If you find this work helpful, please cite:

@inproceedings{
maskey2026bit,
title={1-Bit Wonder: Improving {QAT} Performance in the Low-Bit Regime through K-Means Quantization},
author={Sohir Maskey and Constantin Eichenberg and Johannes Messner and Douglas Orr},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=6EGpWhH2yx}
}