Instructions to use neuralll/GLM-5.3-Flash-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use neuralll/GLM-5.3-Flash-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf neuralll/GLM-5.3-Flash-MTP-GGUF
# Run inference directly in the terminal:
llama cli -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf neuralll/GLM-5.3-Flash-MTP-GGUF
# Run inference directly in the terminal:
llama cli -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf neuralll/GLM-5.3-Flash-MTP-GGUF
# Run inference directly in the terminal:
./llama-cli -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf neuralll/GLM-5.3-Flash-MTP-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Use Docker

docker model run hf.co/neuralll/GLM-5.3-Flash-MTP-GGUF

Llama new
LM Studio
Jan
Ollama
How to use neuralll/GLM-5.3-Flash-MTP-GGUF with Ollama:
```
ollama run hf.co/neuralll/GLM-5.3-Flash-MTP-GGUF
```
Unsloth Desktop

How to use neuralll/GLM-5.3-Flash-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "neuralll/GLM-5.3-Flash-MTP-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use neuralll/GLM-5.3-Flash-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/neuralll/GLM-5.3-Flash-MTP-GGUF
```

Lemonade

How to use neuralll/GLM-5.3-Flash-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull neuralll/GLM-5.3-Flash-MTP-GGUF

Run and chat with the model

lemonade run user.GLM-5.3-Flash-MTP-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use neuralll/GLM-5.3-Flash-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default neuralll/GLM-5.3-Flash-MTP-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use neuralll/GLM-5.3-Flash-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf neuralll/GLM-5.3-Flash-MTP-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "neuralll/GLM-5.3-Flash-MTP-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash MTP draft head (GGUF, Q4_K)

The GLM-5.3-Flash multi-token-prediction (MTP / NextN) block as a small standalone GGUF (4.3 GiB), for speculative decoding next to a GLM-5.3-Flash model you already have. No need to download a whole model again just to get MTP.

llama-server -m GLM-5.3-Flash-GSQ-RCO-3.0bit-q4kattn.gguf \
    -md GLM-5.3-Flash-MTP-Q4_K.gguf --spec-type draft-mtp

The draft borrows the token embeddings and output head of the main model, so only the MTP block itself is stored here. The main model verifies every drafted token: the draft's quantization changes how often drafts are accepted, never the output.

Requirements

The neurall/llama.cpp fork, release b11327 or newer. Stock llama.cpp does not run GLM-5.3-Flash MTP yet (PR #27917), and loading an MTP-only file as -md is a fork addition.
A main GLM-5.3-Flash GGUF with the glm5-next architecture name, e.g. pfeifferj/GLM-5.3-Flash-GSQ-RCO-GGUF (3.0-bit, 3.5-bit) or neuralll/GLM-5.3-Flash-GSQ-RCO-3.0bit-Q4Kattn-GGUF. unsloth's GLM-5.3-Flash GGUFs use a different architecture name (glm5next) and don't load in the fork.

Will it be faster?

It depends on how much of the model the GPUs hold. Every drafted token has to be verified, and each token picks its own experts; experts that run on the CPU cost about as much per drafted token as per generated one.

Measured on 2x RTX 3090 (48 GB) + 125 GB DDR4 with the fork's expert cache, 1500-token chat reply, GLM-5.3-Flash 3.0-bit Q4_K attention (106 GB, 2.3x the VRAM):

	decode t/s
without MTP	20.2
with this MTP head (best draft depth, 1)	17.6 to 18.4

Draft acceptance is ~70-75% at depth 1, but here ~30% of expert work runs on the CPU and the draft's VRAM shrinks the expert cache, so MTP is slower on this setup. Expect a gain when most of the model fits in VRAM (more or bigger GPUs, smaller quants). For comparison, on the same box Qwen3.8-Flash-Next (1.9x the VRAM, 95% of experts cached) gets 1.21x from MTP.

How it was made

The NextN block (blk.45.*, 29 tensors) is taken byte for byte from unsloth/GLM-5.3-Flash-GGUF UD-Q4_K_XL (attention Q8_0, experts Q4_K/Q5_K), fetched with HTTP range requests instead of downloading the model. Metadata comes from the GSQ-RCO main model, with block_count 46, nextn_predict_layers 1 and the MTP layer's per-layer values from unsloth's file. Scripts in this repo (also in the fork under tools/moe-bench/, run from a fork checkout): gguf_remote.py (read a remote GGUF header) and glm_splice_mtp.py (--mtp-only writes this file; without it, it writes a full model with the MTP block).

Credits and license

Model and MTP weights: zai-org/GLM-5.3-Flash, MIT.
Quantization of the MTP block: unsloth/GLM-5.3-Flash-GGUF, MIT.
GLM-5.3-Flash MTP support in llama.cpp: PR #27917 (timkhronos).

MIT license, see LICENSE.

Downloads last month: 914

GGUF

Model size

7B params

Architecture

glm5-next

Hardware compatibility

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for neuralll/GLM-5.3-Flash-MTP-GGUF

Base model

zai-org/GLM-5.3-Flash

Quantized

(213)

this model