Instructions to use ReasoningShield/ReasoningShield-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ReasoningShield/ReasoningShield-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ReasoningShield/ReasoningShield-1B")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ReasoningShield/ReasoningShield-1B", dtype="auto")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ReasoningShield/ReasoningShield-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ReasoningShield/ReasoningShield-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReasoningShield/ReasoningShield-1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ReasoningShield/ReasoningShield-1B

SGLang

How to use ReasoningShield/ReasoningShield-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ReasoningShield/ReasoningShield-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReasoningShield/ReasoningShield-1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ReasoningShield/ReasoningShield-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReasoningShield/ReasoningShield-1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ReasoningShield/ReasoningShield-1B with Docker Model Runner:
```
docker model run hf.co/ReasoningShield/ReasoningShield-1B
```

Update pipeline tag to text-classification and add paper link

by nielsr HF Staff - opened Oct 16, 2025

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+41

-38

Files changed (1) hide show

README.md +41 -38

README.md CHANGED Viewed

@@ -1,16 +1,18 @@
 ---
-license: apache-2.0
 language:
 - en
 metrics:
 - accuracy
 - precision
 - recall
 - f1
-base_model:
-- meta-llama/Llama-3.2-3B-Instruct
-pipeline_tag: text-generation
-library_name: transformers
 tags:
 - llama
 - safe
@@ -18,12 +20,11 @@ tags:
 - safety
 - moderation
 - classifier
-datasets:
-- ReasoningShield/ReasoningShield-Dataset
 ---
-#  🤗 Model Card for *ReasoningShield*
 <div align="center">
   <img src="images/ReasoningShield.svg" alt="ReasoningShield" style="width: 200px; height: auto;">
@@ -59,12 +60,12 @@ datasets:
 ---
-##  🛡 1. Model Overview
 ***ReasoningShield*** is the first specialized safety moderation model tailored to identify hidden risks in intermediate reasoning steps in Large Reasoning Models (LRMs). It excels in detecting harmful content that may be concealed within seemingly harmless reasoning traces, ensuring robust safety alignment for LRMs.
 - **Key Features** :
-  - **Strong Performance**:  It sets a CoT Moderation **SOTA** with over 91% average F1 on open-source LRM traces, outperforming LlamaGuard-4 by 36% and GPT-4o by 16%.
   - **Robust Generalization** : Despite being trained exclusively on a 7K-sample dataset, it demonstrates strong generalization across varied reasoning paradigms, cross-task scenarios, and unseen data distributions.
@@ -72,7 +73,7 @@ datasets:
   - **Efficient Design** : Built on compact base models, it requires low GPU memory (e.g., 2.3GB for 1B version), enabling cost-effective deployment on resource-constrained devices.
-- **Base Model**:  https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct & https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
 ---
@@ -85,10 +86,10 @@ datasets:
 </div>
-- The model is trained on a high-quality dataset of  7,000 (Query, CoT) pairs. Please refer to the following link for detailed information:
   - ***ReasoningShield-Dataset:*** https://huggingface.co/datasets/ReasoningShield/ReasoningShield-Dataset
-- **Risk Categories** :
   - Violence
   - Hate & Toxicity
@@ -125,7 +126,7 @@ datasets:
 #### Stage 2: Direct Preference Optimization Training
-- **Objective** :  Refining the model's performance on hard negative samples constructed from the ambiguous cases and enhancing its robust generalization.
 - **Dataset Size** : 2,642 hard negative samples.
 - **Batch Size** : 2
 - **Gradient Accumulation Steps** : 8
@@ -138,32 +139,32 @@ These two-stage training procedures significantly enhance ***ReasoningShield's**
 ## 🏆 3. Performance Evaluation
-***ReasoningShiled*** achieves **state-of-the-art** performance on CoT Moderation. **Bold** denotes the best results and <ins>underline</ins> the second best.  ***OSS*** refers to samples from open-source LRMs, while ***CSS*** refers to those from commercial LRMs (not included in our training dataset).  Moreover, samples from BeaverTails and Jailbreak are also excluded from our training dataset for testing the generalization capability.
 <div align="center">
-| **Model**               | **Size** | **AIR (OSS)** | **AIR (CSS)** | **SALAD (OSS)** | **SALAD (CSS)** | **BeaverTails (OSS)** | **BeaverTails (CSS)** | **Jailbreak (OSS)** | **Jailbreak (CSS)** | **Avg (OSS)** | **Avg (CSS)** |
 | :---------------------: | :------: | :-----------: | :-----------: | :-------------: | :-------------: | :-------------------: | :-------------------: | :-----------------: | :-----------------: | :-----------: | :-----------: |
-| **Moderation API**      |          |               |               |                 |                 |                       |                       |                     |                     |               |               |
-| Perspective             | -        | 0.0           | 0.0           | 0.0             | 11.9            | 0.0                   | 0.0                   | 0.0                 | 0.0                 | 0.0           | 5.2           |
-| OpenAI Moderation       | -        | 45.7          | 13.2          | 61.7            | 66.7            | 64.9                  | 29.2                  | 70.9                | 41.1                | 60.7          | 44.8          |
-| **Prompted LLM**        |          |               |               |                 |                 |                       |                       |                     |                     |               |               |
-| GPT-4o                  | -        | 70.1          | 47.4          | 75.3            | 75.4            | 79.3                  | 60.6                  | 82.0                | 68.7                | 76.0          | 65.6          |
-| Qwen-2.5                | 72B      | 79.1          | 59.8          | 82.1            | **86.0**        | 81.1                  | 61.5                  | 84.2                | 71.9                | 80.8          | 74.0          |
-| Gemma-3                 | 27B      | 83.2          | 71.6          | 80.2            | 78.3            | 79.2                  | **68.9**              | 86.6                | 73.2                | 81.6          | 74.4          |
-| Mistral-3.1             | 24B      | 65.0          | 45.3          | 77.5            | 73.4            | 73.7                  | 55.1                  | 77.3                | 54.1                | 73.0          | 60.7          |
-| **Finetuned LLM**       |          |               |               |                 |                 |                       |                       |                     |                     |               |               |
-| LlamaGuard-1            | 7B       | 20.3          | 5.7           | 22.8            | 48.8            | 27.1                  | 18.8                  | 53.9                | 5.7                 | 31.0          | 28.0          |
-| LlamaGuard-2            | 8B       | 63.3          | 35.7          | 59.8            | 40.0            | 63.3                  | 47.4                  | 68.2                | 28.6                | 62.4          | 38.1          |
-| LlamaGuard-3            | 8B       | 68.3          | 33.3          | 70.4            | 56.5            | 77.6                  | 30.3                  | 78.5                | 20.5                | 72.8          | 42.2          |
-| LlamaGuard-4            | 12B      | 55.0          | 23.4          | 46.1            | 49.6            | 57.0                  | 13.3                  | 69.2                | 16.2                | 56.2          | 33.7          |
-| Aegis-Permissive        | 7B       | 56.3          | 51.0          | 66.5            | 67.4            | 65.8                  | 35.3                  | 70.7                | 33.3                | 64.3          | 53.9          |
-| Aegis-Defensive         | 7B       | 71.2          | 56.9          | 76.4            | 67.8            | 73.9                  | 27.0                  | 75.4                | 53.2                | 73.6          | 54.9          |
-| WildGuard               | 7B       | 58.8          | 45.7          | 66.7            | 76.3            | 68.3                  | 51.3                  | 79.6                | 55.3                | 67.6          | 62.1          |
-| MD-Judge                | 7B       | 71.8          | 44.4          | 83.4            | 83.2            | 81.0                  | 50.0                  | 86.8                | 56.6                | 80.1          | 66.0          |
-| Beaver-Dam              | 7B       | 50.0          | 17.6          | 52.6            | 36.6            | 71.1                  | 12.7                  | 60.2                | 36.0                | 58.2          | 26.5          |
-| **ReasoningShield (Ours)** | 1B    | <ins>94.2</ins> | <ins>83.7</ins> | <ins>91.5</ins> | 80.5            | <ins>89.0</ins>       | 60.0                  | <ins>90.1</ins>     | <ins>74.2</ins>     | <ins>89.4</ins> | <ins>77.7</ins> |
-| **ReasoningShield (Ours)** | 3B    | **94.5**      | **86.7**      | **94.0**        | <ins>84.8</ins> | **90.4**              | <ins>64.6</ins>       | **92.3**            | **76.2**            | **91.8**      | **81.4**      |
 </div>
@@ -204,7 +205,8 @@ model_thinking = "Intermediate reasoning steps here"  # The content can be repla
 messages = [
     {"role": "system", "content": reasoningshield_prompt},
-    {"role": "user", "content": f"Query: {question}\nThought: {model_thinking}"}
 ]
 prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
@@ -247,7 +249,8 @@ model_thinking = "Intermediate reasoning steps here" # The content can be replac
 messages = [
     {"role": "system", "content": reasoningshield_prompt},
-    {"role": "user", "content": f"Query: {question}\nThought: {model_thinking}"}
 ]
 prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

 ---
+base_model:
+- meta-llama/Llama-3.2-3B-Instruct
+datasets:
+- ReasoningShield/ReasoningShield-Dataset
 language:
 - en
+library_name: transformers
+license: apache-2.0
 metrics:
 - accuracy
 - precision
 - recall
 - f1
+pipeline_tag: text-classification
 tags:
 - llama
 - safe
 - safety
 - moderation
 - classifier
 ---
+# 🤗 Model Card for *ReasoningShield*
+This repository contains the ReasoningShield model presented in the paper [ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models](https://huggingface.co/papers/2505.17244).
 <div align="center">
   <img src="images/ReasoningShield.svg" alt="ReasoningShield" style="width: 200px; height: auto;">
 ---
+## 🛡 1. Model Overview
 ***ReasoningShield*** is the first specialized safety moderation model tailored to identify hidden risks in intermediate reasoning steps in Large Reasoning Models (LRMs). It excels in detecting harmful content that may be concealed within seemingly harmless reasoning traces, ensuring robust safety alignment for LRMs.
 - **Key Features** :
+  - **Strong Performance**: It sets a CoT Moderation **SOTA** with over 91% average F1 on open-source LRM traces, outperforming LlamaGuard-4 by 36% and GPT-4o by 16%.
   - **Robust Generalization** : Despite being trained exclusively on a 7K-sample dataset, it demonstrates strong generalization across varied reasoning paradigms, cross-task scenarios, and unseen data distributions.
   - **Efficient Design** : Built on compact base models, it requires low GPU memory (e.g., 2.3GB for 1B version), enabling cost-effective deployment on resource-constrained devices.
+- **Base Model**: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct & https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
 ---
 </div>
+- The model is trained on a high-quality dataset of 7,000 (Query, CoT) pairs. Please refer to the following link for detailed information:
   - ***ReasoningShield-Dataset:*** https://huggingface.co/datasets/ReasoningShield/ReasoningShield-Dataset
+- **Risk Categories** :
   - Violence
   - Hate & Toxicity
 #### Stage 2: Direct Preference Optimization Training
+- **Objective** : Refining the model's performance on hard negative samples constructed from the ambiguous cases and enhancing its robust generalization.
 - **Dataset Size** : 2,642 hard negative samples.
 - **Batch Size** : 2
 - **Gradient Accumulation Steps** : 8
 ## 🏆 3. Performance Evaluation
+***ReasoningShiled*** achieves **state-of-the-art** performance on CoT Moderation. **Bold** denotes the best results and <ins>underline</ins> the second best. ***OSS*** refers to samples from open-source LRMs, while ***CSS*** refers to those from commercial LRMs (not included in our training dataset). Moreover, samples from BeaverTails and Jailbreak are also excluded from our training dataset for testing the generalization capability.
 <div align="center">
+| **Model** | **Size** | **AIR (OSS)** | **AIR (CSS)** | **SALAD (OSS)** | **SALAD (CSS)** | **BeaverTails (OSS)** | **BeaverTails (CSS)** | **Jailbreak (OSS)** | **Jailbreak (CSS)** | **Avg (OSS)** | **Avg (CSS)** |
 | :---------------------: | :------: | :-----------: | :-----------: | :-------------: | :-------------: | :-------------------: | :-------------------: | :-----------------: | :-----------------: | :-----------: | :-----------: |
+| **Moderation API** | | | | | | | | | | | |
+| Perspective | - | 0.0 | 0.0 | 0.0 | 11.9 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 5.2 |
+| OpenAI Moderation | - | 45.7 | 13.2 | 61.7 | 66.7 | 64.9 | 29.2 | 70.9 | 41.1 | 60.7 | 44.8 |
+| **Prompted LLM** | | | | | | | | | | | |
+| GPT-4o | - | 70.1 | 47.4 | 75.3 | 75.4 | 79.3 | 60.6 | 82.0 | 68.7 | 76.0 | 65.6 |
+| Qwen-2.5 | 72B | 79.1 | 59.8 | 82.1 | **86.0** | 81.1 | 61.5 | 84.2 | 71.9 | 80.8 | 74.0 |
+| Gemma-3 | 27B | 83.2 | 71.6 | 80.2 | 78.3 | 79.2 | **68.9** | 86.6 | 73.2 | 81.6 | 74.4 |
+| Mistral-3.1 | 24B | 65.0 | 45.3 | 77.5 | 73.4 | 73.7 | 55.1 | 77.3 | 54.1 | 73.0 | 60.7 |
+| **Finetuned LLM** | | | | | | | | | | | |
+| LlamaGuard-1 | 7B | 20.3 | 5.7 | 22.8 | 48.8 | 27.1 | 18.8 | 53.9 | 5.7 | 31.0 | 28.0 |
+| LlamaGuard-2 | 8B | 63.3 | 35.7 | 59.8 | 40.0 | 63.3 | 47.4 | 68.2 | 28.6 | 62.4 | 38.1 |
+| LlamaGuard-3 | 8B | 68.3 | 33.3 | 70.4 | 56.5 | 77.6 | 30.3 | 78.5 | 20.5 | 72.8 | 42.2 |
+| LlamaGuard-4 | 12B | 55.0 | 23.4 | 46.1 | 49.6 | 57.0 | 13.3 | 69.2 | 16.2 | 56.2 | 33.7 |
+| Aegis-Permissive | 7B | 56.3 | 51.0 | 66.5 | 67.4 | 65.8 | 35.3 | 70.7 | 33.3 | 64.3 | 53.9 |
+| Aegis-Defensive | 7B | 71.2 | 56.9 | 76.4 | 67.8 | 73.9 | 27.0 | 75.4 | 53.2 | 73.6 | 54.9 |
+| WildGuard | 7B | 58.8 | 45.7 | 66.7 | 76.3 | 68.3 | 51.3 | 79.6 | 55.3 | 67.6 | 62.1 |
+| MD-Judge | 7B | 71.8 | 44.4 | 83.4 | 83.2 | 81.0 | 50.0 | 86.8 | 56.6 | 80.1 | 66.0 |
+| Beaver-Dam | 7B | 50.0 | 17.6 | 52.6 | 36.6 | 71.1 | 12.7 | 60.2 | 36.0 | 58.2 | 26.5 |
+| **ReasoningShield (Ours)** | 1B | <ins>94.2</ins> | <ins>83.7</ins> | <ins>91.5</ins> | 80.5 | <ins>89.0</ins> | 60.0 | <ins>90.1</ins> | <ins>74.2</ins> | <ins>89.4</ins> | <ins>77.7</ins> |
+| **ReasoningShield (Ours)** | 3B | **94.5** | **86.7** | **94.0** | <ins>84.8</ins> | **90.4** | <ins>64.6</ins> | **92.3** | **76.2** | **91.8** | **81.4** |
 </div>
 messages = [
     {"role": "system", "content": reasoningshield_prompt},
+    {"role": "user", "content": f"Query: {question}
+Thought: {model_thinking}"}
 ]
 prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 messages = [
     {"role": "system", "content": reasoningshield_prompt},
+    {"role": "user", "content": f"Query: {question}
+Thought: {model_thinking}"}
 ]
 prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)