Blame for README.md

---
library_name: mlx
license: mit
base_model: inclusionAI/Ling-3.0-flash
base_model_relation: quantized
pipeline_tag: text-generation
language:
  - en
  - zh
tags:
  - mlx
  - mlx-lm
  - omlx
  - bailing_hybrid
  - mixture-of-experts
  - linear-attention
  - quantized
  - apple-silicon
  - custom_code
  - oq5
---

<p align="center">
  <a href="https://tensorfold.dev">
    <img src="https://huggingface.co/spaces/TensorFold/README/resolve/main/tensorfold-logo.png" alt="TensorFold" width="160">
  </a>
</p>

<p align="center">
  <img src="https://mdn.alipayobjects.com/huamei_qa8qxu/afts/img/A*4QxcQrBlTiAAAAAAQXAAAAgAemJ7AQ/original" width="100" alt="Ling logo">
</p>


<p align="center">
  <img src="https://img.shields.io/badge/InclusionAI-Ling-1677FF?style=for-the-badge" alt="InclusionAI Ling">
  <img src="https://img.shields.io/badge/Apple_Silicon-MLX-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Apple silicon MLX">
  <img src="https://img.shields.io/badge/TensorFold-oMLX-6E56CF?style=for-the-badge&logo=huggingface&logoColor=white" alt="TensorFold oMLX">
</p>


<h1 align="center">Ling 3.0 Flash — oQ5</h1>


<p align="center">
  A native Apple-silicon conversion of <a href="https://huggingface.co/inclusionAI/Ling-3.0-flash">inclusionAI/Ling-3.0-flash</a>, quantized with oMLX's optimized mixed-precision oQ5 recipe and packaged for oMLX and compatible MLX-LM runtimes.
</p>


<p align="center">
  <a href="https://huggingface.co/inclusionAI/Ling-3.0-flash">Original model</a> ·
  <a href="https://modelscope.cn/organization/inclusionAI">InclusionAI on ModelScope</a> ·
  <a href="https://openrouter.ai/inclusionai/ling-3.0-flash:free">OpenRouter</a> ·
  <a href="https://github.com/ml-explore/mlx-lm">MLX-LM</a>
</p>


## About this conversion


This repository contains an **oMLX-optimized mixed-precision oQ5 conversion** of Ling 3.0 Flash. Ling is a 124B-total / 5.1B-active hybrid-linear mixture-of-experts model. The conversion preserves the upstream tokenizer and chat template and includes the custom MLX architecture adapter required for Kimi Delta Attention, gated MLA, and sparse MoE layers.


| Item | Value |
| --- | --- |
| Base model | [`inclusionAI/Ling-3.0-flash`](https://huggingface.co/inclusionAI/Ling-3.0-flash) |
| Format | MLX safetensors |
| Quantization | oQ5 affine mixed precision, group size 64 |
| Base precision | 5-bit |
| Protected modules | 130 at 6-bit and 325 at 8-bit |
| Conversion/runtime stack | oMLX with MLX-LM 0.31.3 / MLX 0.32.0 |
| Weight shards | 16 |
| Weight size | 86.62 GB (80.67 GiB) |
| Maximum configured context | 262,144 tokens |
| Architecture | `bailing_hybrid` |


> [!IMPORTANT]
> This model uses the included `bailing_hybrid.py` custom MLX adapter. In oMLX, enable **Trust Remote Code** for this model before loading it. Standalone MLX-LM requires a build whose `mlx_lm.load` supports the `trust_remote_code` argument and repository-provided `model_file` adapters; stock PyPI `mlx-lm 0.31.3` does not provide that loader path.


## Apple-silicon performance


This checkpoint was load-tested, generation-tested, and benchmarked on:


| Hardware | Configuration |
| --- | --- |
| Host | Mac Studio |
| Chip | Apple M3 Ultra |
| CPU | 32 cores (24 performance + 8 efficiency) |
| Unified memory | 256 GB |
| Runtime | oMLX-bundled MLX-LM 0.31.3 / MLX 0.32.0 |


A warmed local test produced:


| Measurement | Result |
| --- | ---: |
| Decode (median) | **73.10 tokens/s** |
| Individual decode runs | 73.23 / 73.10 / 73.09 tokens/s |
| Reported peak memory | **87.44 GB** |
| Timed runs | 3 × 256 generated tokens |
| Warm-up | 256 generated tokens |
| Prompt | 39 tokens after chat templating |


The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance.


## Recommended use with oMLX


1. Download the model into the oMLX model directory:


```bash
hf download TensorFold/Ling-3.0-flash-oQ5 \
  --local-dir ~/.omlx/models/TensorFold/Ling-3.0-flash-oQ5
```


2. Refresh the oMLX model registry.
3. Open the model settings and enable **Trust Remote Code**.
4. Load `Ling-3.0-flash-oQ5` and use the normal chat UI or OpenAI-compatible endpoint.


Example request:


```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "Ling-3.0-flash-oQ5",
    "messages": [{"role": "user", "content": "Explain why hybrid linear attention is useful."}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 512
  }'
```


Thinking mode is enabled by the upstream chat template by default. Disable it with the model's chat-template configuration when a direct answer is preferred.


## Compatible standalone MLX-LM builds


With an MLX-LM build that supports repository-provided model adapters:


```bash
mlx_lm.generate \
  --model TensorFold/Ling-3.0-flash-oQ5 \
  --trust-remote-code \
  --prompt "Explain why hybrid linear attention is useful." \
  --max-tokens 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20
```


Python usage with the same capability:


```python
from mlx_lm import load, generate


model, tokenizer = load(
    "TensorFold/Ling-3.0-flash-oQ5",
    trust_remote_code=True,
    tokenizer_config={"trust_remote_code": True},
)
messages = [
    {"role": "user", "content": "Explain sparse mixture-of-experts routing."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```


Only enable remote code after reviewing the included adapter files.


## Architecture


Ling 3.0 Flash alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts.


| Architecture detail | Upstream value |
| --- | ---: |
| Total / active parameters | 124B / 5.1B |
| Transformer layers | 35 KDA + 7 gated MLA |
| Dense layers | 2 |
| Routed / shared experts | 512 / 1 |
| Active routed experts | 8 |
| Attention heads | 32 |
| Hidden size | 2,560 |
| Expert intermediate size | 768 |
| Dense intermediate size | 6,144 |
| Vocabulary size | 157,184 |
| Context training schedule | 8K → 32K → 256K |


The included MLX adapter uses MLX-LM primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is excluded from ordinary causal generation; this release does **not** claim MTP or DSpark speculative-decoding support.


## Upstream model highlights


InclusionAI describes Ling 3.0 Flash as a hybrid reasoning model for software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding.


The upstream defaults are:


```text
thinking: enabled
temperature: 0.6
top_p: 0.95
top_k: 20
```


For benchmark methodology, scores, intended use, limitations, and framework-specific deployment instructions, see the [original InclusionAI model card](https://huggingface.co/inclusionAI/Ling-3.0-flash).


## Conversion and validation notes


- Source weights: upstream BF16 checkpoint.
- Quantization mode: affine, group size 64.
- oQ5 uses a 5-bit base with 455 sensitive modules kept at higher precision.
- Both `quantization` and `quantization_config` preserve the per-module recipe.
- The upstream chat template is included unchanged.
- All 2,067 converted tensors and all 16 indexed shards were checked locally.
- The custom adapter was loaded with explicit trust and exercised through end-to-end generation.
- Quantization can reduce output quality relative to BF16; use a higher-precision variant when quality matters more than memory use.


This is a community conversion, not an official InclusionAI release. Validate quality and numerical behavior on representative workloads before production use.


## License and attribution


The upstream model declares the **MIT License** in its Hugging Face metadata. A standard MIT licence copy is included in this repository.


All model design, training, benchmark, and upstream documentation credit belongs to InclusionAI and the original contributors. The mixed-precision conversion, Apple-silicon validation, compatibility packaging, and model card are provided by [TensorFold](https://huggingface.co/TensorFold).



<!-- TensorFold-chooser-start -->
## Choose for your Mac

[64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b)

Published peak memory: **87.44 GB**; estimated starting tier: **128GB**, leaving about **40 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

### Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

### Quick start and demo prompt

```bash
hf download TensorFold/Ling-3.0-flash-oQ5 --local-dir ./models/Ling-3.0-flash-oQ5
```

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

```text
Explain why the sky looks blue in three short sentences.
```

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

[Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold)
<!-- TensorFold-chooser-end -->