A native Apple-silicon conversion of inclusionAI/Ling-3.0-flash, quantized with oMLX's optimized mixed-precision oQ5 recipe and packaged for oMLX and compatible MLX-LM runtimes.
Original model · InclusionAI on ModelScope · OpenRouter · MLX-LM
## About this conversion This repository contains an **oMLX-optimized mixed-precision oQ5 conversion** of Ling 3.0 Flash. Ling is a 124B-total / 5.1B-active hybrid-linear mixture-of-experts model. The conversion preserves the upstream tokenizer and chat template and includes the custom MLX architecture adapter required for Kimi Delta Attention, gated MLA, and sparse MoE layers. | Item | Value | | --- | --- | | Base model | [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash) | | Format | MLX safetensors | | Quantization | oQ5 affine mixed precision, group size 64 | | Base precision | 5-bit | | Protected modules | 130 at 6-bit and 325 at 8-bit | | Conversion/runtime stack | oMLX with MLX-LM 0.31.3 / MLX 0.32.0 | | Weight shards | 16 | | Weight size | 86.62 GB (80.67 GiB) | | Maximum configured context | 262,144 tokens | | Architecture | `bailing_hybrid` | > [!IMPORTANT] > This model uses the included `bailing_hybrid.py` custom MLX adapter. In oMLX, enable **Trust Remote Code** for this model before loading it. Standalone MLX-LM requires a build whose `mlx_lm.load` supports the `trust_remote_code` argument and repository-provided `model_file` adapters; stock PyPI `mlx-lm 0.31.3` does not provide that loader path. ## Apple-silicon performance This checkpoint was load-tested, generation-tested, and benchmarked on: | Hardware | Configuration | | --- | --- | | Host | Mac Studio | | Chip | Apple M3 Ultra | | CPU | 32 cores (24 performance + 8 efficiency) | | Unified memory | 256 GB | | Runtime | oMLX-bundled MLX-LM 0.31.3 / MLX 0.32.0 | A warmed local test produced: | Measurement | Result | | --- | ---: | | Decode (median) | **73.10 tokens/s** | | Individual decode runs | 73.23 / 73.10 / 73.09 tokens/s | | Reported peak memory | **87.44 GB** | | Timed runs | 3 × 256 generated tokens | | Warm-up | 256 generated tokens | | Prompt | 39 tokens after chat templating | The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance. ## Recommended use with oMLX 1. Download the model into the oMLX model directory: ```bash hf download TensorFold/Ling-3.0-flash-oQ5 \ --local-dir ~/.omlx/models/TensorFold/Ling-3.0-flash-oQ5 ``` 2. Refresh the oMLX model registry. 3. Open the model settings and enable **Trust Remote Code**. 4. Load `Ling-3.0-flash-oQ5` and use the normal chat UI or OpenAI-compatible endpoint. Example request: ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OMLX_API_KEY" \ -d '{ "model": "Ling-3.0-flash-oQ5", "messages": [{"role": "user", "content": "Explain why hybrid linear attention is useful."}], "temperature": 0.6, "top_p": 0.95, "max_tokens": 512 }' ``` Thinking mode is enabled by the upstream chat template by default. Disable it with the model's chat-template configuration when a direct answer is preferred. ## Compatible standalone MLX-LM builds With an MLX-LM build that supports repository-provided model adapters: ```bash mlx_lm.generate \ --model TensorFold/Ling-3.0-flash-oQ5 \ --trust-remote-code \ --prompt "Explain why hybrid linear attention is useful." \ --max-tokens 512 \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 ``` Python usage with the same capability: ```python from mlx_lm import load, generate model, tokenizer = load( "TensorFold/Ling-3.0-flash-oQ5", trust_remote_code=True, tokenizer_config={"trust_remote_code": True}, ) messages = [ {"role": "user", "content": "Explain sparse mixture-of-experts routing."} ] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=False, enable_thinking=False, ) print(generate(model, tokenizer, prompt=prompt, max_tokens=512)) ``` Only enable remote code after reviewing the included adapter files. ## Architecture Ling 3.0 Flash alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts. | Architecture detail | Upstream value | | --- | ---: | | Total / active parameters | 124B / 5.1B | | Transformer layers | 35 KDA + 7 gated MLA | | Dense layers | 2 | | Routed / shared experts | 512 / 1 | | Active routed experts | 8 | | Attention heads | 32 | | Hidden size | 2,560 | | Expert intermediate size | 768 | | Dense intermediate size | 6,144 | | Vocabulary size | 157,184 | | Context training schedule | 8K → 32K → 256K | The included MLX adapter uses MLX-LM primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is excluded from ordinary causal generation; this release does **not** claim MTP or DSpark speculative-decoding support. ## Upstream model highlights InclusionAI describes Ling 3.0 Flash as a hybrid reasoning model for software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding. The upstream defaults are: ```text thinking: enabled temperature: 0.6 top_p: 0.95 top_k: 20 ``` For benchmark methodology, scores, intended use, limitations, and framework-specific deployment instructions, see the [original InclusionAI model card](https://huggingface.co/inclusionAI/Ling-3.0-flash). ## Conversion and validation notes - Source weights: upstream BF16 checkpoint. - Quantization mode: affine, group size 64. - oQ5 uses a 5-bit base with 455 sensitive modules kept at higher precision. - Both `quantization` and `quantization_config` preserve the per-module recipe. - The upstream chat template is included unchanged. - All 2,067 converted tensors and all 16 indexed shards were checked locally. - The custom adapter was loaded with explicit trust and exercised through end-to-end generation. - Quantization can reduce output quality relative to BF16; use a higher-precision variant when quality matters more than memory use. This is a community conversion, not an official InclusionAI release. Validate quality and numerical behavior on representative workloads before production use. ## License and attribution The upstream model declares the **MIT License** in its Hugging Face metadata. A standard MIT licence copy is included in this repository. All model design, training, benchmark, and upstream documentation credit belongs to InclusionAI and the original contributors. The mixed-precision conversion, Apple-silicon validation, compatibility packaging, and model card are provided by [TensorFold](https://huggingface.co/TensorFold). ## Choose for your Mac [64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b) Published peak memory: **87.44 GB**; estimated starting tier: **128GB**, leaving about **40 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request. ### Runtime and evidence The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results. ### Quick start and demo prompt ```bash hf download TensorFold/Ling-3.0-flash-oQ5 --local-dir ./models/Ling-3.0-flash-oQ5 ``` Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading. Try this in a new chat with a 128-token output limit: ```text Explain why the sky looks blue in three short sentences. ``` This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available. [Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold)