Qwen3-Embedding-0.6B-FP8

FP8-quantized version of Qwen/Qwen3-Embedding-0.6B.

This model was quantized with llm-compressor using FP8 dynamic activation quantization for the Qwen3 embedding backbone.

Quantization details

• Base model: Qwen/Qwen3-Embedding-0.6B
• Quantization tool: llm-compressor
• Saved format: compressed-tensors
• Quantization scheme: FP8_DYNAMIC
• Targets: Linear
• Ignored modules: none

Quantization recipe

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

recipe = QuantizationModifier(
    targets="Linear",
    scheme="FP8_DYNAMIC",
)

oneshot(model=model, recipe=recipe)

model.save_pretrained("Qwen3-Embedding-0.6B-FP8")
tokenizer.save_pretrained("Qwen3-Embedding-0.6B-FP8")