--- license: apache-2.0 tags: - model-garden - quantization - mlx - gguf - apple-silicon - asr - embeddings - kv-cache-quantization library_name: transformers pipeline_tag: text-generation language: - en --- > [!TIP] > **KV-cache quantization needs no fork (2026):** upstream llama.cpp / Ollama > cover it natively. Use `-ctk q8_0 -ctv q8_0` (~half KV memory, perplexity > +0.002–0.05) or `-ctk q4_0 -ctv q4_0` (~quarter memory, ≈7.6% perplexity > increase). In Ollama: `OLLAMA_KV_CACHE_TYPE=q8_0` with > `OLLAMA_FLASH_ATTENTION=1`. Keep K and V types symmetric to stay on the > fused Flash-Attention path. Since April 2026 mainline llama.cpp also > applies Hadamard rotation to KV activations > ([PR #21038](https://github.com/ggml-org/llama.cpp/pull/21038)), which > greatly improves low-bit KV quality (opt-out: `LLAMA_ATTN_ROT_DISABLE=1`). # majentik — Model Garden Quantized open-weight models for Apple Silicon and llama.cpp, released only after a coherence smoke gate passes on real hardware. Every repo keeps the upstream tokenizer, architecture and license; the only thing we change is how the weights are stored. > 402 model repositories · 14 datasets · 13 collections · snapshot 2026-09-16 ## How releases are made 1. **Pin upstream.** Every lane pins the exact upstream revision it quantized from and refuses to run if the source drifts. 2. **Quantize with standard tooling.** `mlx-lm` / `mlx-vlm` / `mlx-audio` for MLX, `llama.cpp` for GGUF. Group size 32 for ≤4-bit MLX tiers. We never publish a tier at or above the source's bits-per-weight. 3. **Smoke gate.** Each tier must produce coherent greedy output on a fixed prompt set before upload. Tiers that fail are **held back** and listed as such on the family card instead of being shipped — expect gaps in the tier ladder for small or fragile models. 4. **Evidence.** Gate records are published in [`majentik/garden-quant-bench`](https://huggingface.co/datasets/majentik/garden-quant-bench). Repos in the [Verified MLX releases](https://huggingface.co/collections/majentik/verified-mlx-releases-smoke-gated-6a5679b153c6e810855d4ddb) collection link their evidence directly. ## Lanes | Lane | Repos | Runtime | Tiers | | --- | ---: | --- | --- | | **MLX** | 264 | `mlx-lm`, `mlx-vlm`, `mlx-audio` on Apple Silicon | 2 / 3 / 4 / 5 / 6 / 8-bit, MXFP4, bf16 references, LoRA adapters | | **GGUF** | 129 | `llama.cpp` (one-shot: `llama-completion -no-cnv`), Ollama, LM Studio | Q2_K … Q8_0, IQ4_XS, MXFP4 | | **FP8 / ONNX / other** | 7 | vLLM, onnxruntime | Family-specific | Repo naming is unbranded: `majentik/-MLX-` and `majentik/-GGUF-`. ## Families Counts are repos on the Hub as of the snapshot date. | Family | Repos | Notes | | --- | ---: | --- | | Gemma 4 | 91 | E2B / E4B / 12B / 26B-A4B / 31B, base + instruct, MLX + GGUF ([collection](https://huggingface.co/collections/majentik/gemma-4-quantized-gguf-mlx-6a56797c108d798040f3fc8a)) | | Nemotron 3 / 3.5 / Cascade 2 | 55 | Nano 4B, Nano 30B-A3B, Nano Omni 30B (audio+vision), Super 120B-A12B, Lightning 30B, Cascade 2 ([collection](https://huggingface.co/collections/majentik/nemotron-quantized-6a5679bbeb8b5285afe9830a)) | | Qwen 3.6 | 25 | 27B dense + 35B-A3B MoE, full vision tower ([collection](https://huggingface.co/collections/majentik/qwen-36-quantized-6a56798eefe10fe2cf50e6b0)) | | Qwen 3.5 / 3.8 | 22 | 27B, 122B-A10B, 397B-A17B; Qwen3.8-27B ([collection](https://huggingface.co/collections/majentik/qwen-35-quantized-6a567999b3603603c963dc84)) | | Qwen agents & coders | 14 | Qwen3-Coder-Next, Qwen-AgentWorld-35B-A3B | | MiniCPM 5 | 25 | 1B base / instruct / SFT edge models | | Ornith 1.0 / 1.5 | 16 | 9B, 35B, 35B-A3B reasoning MoE | | Embeddings | 29 | Qwen3-Embedding 0.6B/4B/8B, UEmbed 2B/4B/9B, nomic-embed-text-v2-moe, form-embed ([UEmbed](https://huggingface.co/collections/majentik/uembed-multimodal-embeddings-mlx-dense-text-6a7a65b0030aefa4397275ab), [ONNX](https://huggingface.co/collections/majentik/embeddings-onnx-6a5679cdeaaccc7c6a7e8f0b)) | | Speech — ASR & TTS | 37 | MERaLiON-3 3B/10B, Qwen3-ASR, Voxtral Mini/Realtime/TTS, MOSS-Transcribe, cohere-transcribe-arabic, Kokoro, fishaudio-s2-pro, Audio8-TTS, Gemma-4-E4B MERaLiON speech LoRAs ([ASR on Apple Silicon](https://huggingface.co/collections/majentik/asr-on-apple-silicon-mlx-6a5679aa5039bc57c9e03e1a), [TTS on Apple Silicon](https://huggingface.co/collections/majentik/tts-on-apple-silicon-mlx-6aa973c6bd194270585e8b9d), [MERaLiON](https://huggingface.co/collections/majentik/meralion-singapore-speech-models-on-mlx-6a5679a0854fb642ca194cc5)) | | gpt-oss | 10 | 20B + 120B rebuilt GGUFs / MLX ([collection](https://huggingface.co/collections/majentik/gpt-oss-rebuilt-ggufs-6a5679af3e5e112d1c06c217)) | | LFM 2.5 | 10 | 2.6B dense, 8B-A1B MoE | | harrier-oss | 10 | 270M / 0.6B / 27B | | Mistral | 11 | Medium 3.5 128B, Small 4 119B, Leanstral | | Vision & agents | 29 | Muse Glimmer 30B ([collection](https://huggingface.co/collections/majentik/meta-muse-glimmer-mlx-6a7a65ad187894ba895a50f6)), UI-Mate 27B, BigBang v1, Unlimited-OCR, Qwen-Image-Bench, GELab-Zero ([OCR & DocAI](https://huggingface.co/collections/majentik/ocr-and-document-ai-mlx-6a5679ad40e489b44ce64a80)); gaze estimation: [MobileGaze-MobileOne-S0-MLX-fp32](https://huggingface.co/majentik/MobileGaze-MobileOne-S0-MLX-fp32) (MIT, Gaze360, parity-gated hand port, added after the snapshot) | | Other LLMs | 16 | KAT-Coder V2.5, Shieldstral 3B, MiniMax M2.7, DeepSeek-V4-Flash, Qwen2.5-1.5B DWQ reference, cga-gpt | ## Datasets Calibration and evaluation sets used by the lanes, mirrored so results are reproducible: `ultrachat-calib`, `c4-calib`, `ultra-fineweb-calib`, `ultradata-{sft,math}-calib`, `tulu-3-sft-mixture`, `wikitext-2-ppl`, `gsm8k`, `ifeval`, `fleurs-ar-asr`, `WildASR`, `SASRBench-v1`, `magpie-reasoning-qwen25-7b`, and the gate ledger `garden-quant-bench`. ## Pick a starting point - **MacBook, 16 GB** → Gemma 4 E4B-it MLX 4-bit, or LFM2.5-8B-A1B MLX 4-bit. - **MacBook, 32–64 GB** → Qwen3.6-35B-A3B MLX 4-bit, Nemotron-3-Nano-30B-A3B MLX 4-bit. - **Mac Studio, 128 GB+** → Qwen3.5-122B-A10B MLX 4-bit, Mistral-Medium-3.5-128B MLX 4-bit, gpt-oss-120b. - **Speech on Apple Silicon** → MERaLiON-3-3B-ASR MLX 8-bit (Singapore English / Malay / Chinese / Tamil), Qwen3-ASR MLX. - **Gaze estimation on Apple Silicon** → MobileGaze-MobileOne-S0 MLX fp32 (1.2M params, yaw/pitch in degrees from a 448×448 face crop, ~4 ms/frame on M5 Max). - **Long context on llama.cpp** → any GGUF tier plus `-ctk q8_0 -ctv q8_0` (see tip above). ## What is *not* in each repo - **Training data.** Quantization-only releases; the base model's training data and disclaimers are upstream's. - **Benchmarks for every axis.** We publish the smoke gate result, and where measured, WikiText-2 perplexity or family-specific evals (KL vs. bf16, WER for ASR). Missing an axis you need? Open a discussion on the repo. ## About the RotorQuant / TurboQuant labels Older repos in this org carry `RotorQuant` or `TurboQuant` in their names. These are historical **release labels, not distinct quantization algorithms**: for any given tier both labelled repos hold byte-identical weights produced with the standard MLX / llama.cpp quantizers, and no brand-specific speedup is claimed or measured. New releases are unbranded. The associated llama.cpp fork is unmaintained; use upstream KV-cache quantization as described in the tip at the top. ## Who we are majentik publishes these to keep our own fleet running cheaply on commodity Apple hardware and to close the gap between a research release and "can I actually run this tonight". Issues, quant requests and benchmark PRs are welcome via the **Community** tab on the closest repo. ## Versioning Each repo tracks `upstream@base-model-revision × quant-lane`. When upstream ships a new base revision we re-run the lane and bump the repo. Card-only changes do not bump the version. ## License Each repo inherits the **base model's license**, not this organization-level license. Check the `license` field in the repository's card before deploying.