startlux-models/ReaDy-Qwen3.8-27B

ReaDy · Qwen3.8-27B

Recurrent states as dynamic context for speculative drafting.

Paper · Code & quickstart · Target model

This repository contains the full ReaDy draft-model weights and its rank-8 recurrent-state reader, as described in ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models. ReaDy augments a DSpark drafter with position-specific reads from the target model's committed recurrent states, and injects the retrieved context as a residual update to the draft representation.

These are draft weights, not a standalone 27B chat model. Inference requires the separate Qwen3.8-27B target, its tokenizer, and the pinned SGLang integration in the code repository. Load this checkpoint through the supplied serve.py; it is not an AutoModel.from_pretrained release and requires no Hugging Face remote code.

Download

hf download startlux-models/ReaDy-Qwen3.8-27B --local-dir checkpoint
hf download Qwen/Qwen3.8-27B --local-dir models/Qwen3.8-27B

Checkpoint layout

File	Contents
model.safetensors	Full draft backbone and prediction heads
config.json	Native SGLang draft configuration
reader/reader.safetensors	Recurrent-state reader parameters
reader/reader_config.json	Reader dimensions and injection point
SHA256.json	SHA-256 checksums verified by the launcher

The draft weight SHA-256 is 746f7d94f77d1d032614d2c0af805bf08cbe22b4a019b7ba84e103d36139c3ac.

Run

From the ReaDy code repository, with its documented runtime environment prepared:

python scripts/prepare_engine.py --destination third_party/sglang
python serve.py \
  --target models/Qwen3.8-27B \
  --checkpoint checkpoint \
  --sglang-source third_party/sglang/python \
  --reader-backend triton \
  --flashinfer-cubins /path/to/flashinfer-0.6.17-cubins \
  --gpu 0 --port 30000

Set --flashinfer-cubins to your matching 0.6.17 cubin directory and select a GPU reserved for your process. The default reader backend is triton, enabling the custom fused reader operations. Pass --reader-backend reference for the PyTorch implementation. Both use SGLang for target verification and request execution.

Configuration and validation

• Target recurrent layers: 30 and 62 (zero-based).
• State per request: [2, 48, 128, 128], with value before key.
• Rank: 8 for the value-map output and query factorization; the target state remains 128 × 128.
• Residual injection: draft layer 4, after attention normalization.
• Draft block: 7 tokens; hidden size: 5,120; reader bottleneck: 128.
• Tested runtime: Linux, Python 3.12, one NVIDIA B200, BF16 target/draft/KV and FP32 recurrent state.
• Validated serving mode: greedy decoding, concurrency 1, context limit 16,384, natural EOS.

The code repository includes exact core package versions, the pinned engine revision, and VALIDATION.json. Reference and Triton backends passed the recorded kernel and end-to-end equivalence checks. Those checks describe the tested configuration; other hardware, batching modes, and stochastic decoding require separate validation.

Dependencies

Target and base-drafter weights retain their upstream licensing requirements. This repository does not grant additional rights to third-party weights. SGLang and the target model are downloaded separately; refer to the code repository for runtime requirements.