Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,98 @@
+---
+license: apache-2.0
+tags:
+  - misinformation
+  - content-safety
+  - fineweb
+  - text-classification
+  - modernbert
+datasets:
+  - ratishsp/fineweb-edu-misinfo
+language:
+  - en
+base_model: answerdotai/ModernBERT-base
+---
+
+# FineWeb-Edu Misinformation Classifier
+
+A ModernBERT-base classifier trained to detect misinformation in web text, specifically content that passes educational 
+quality filters despite being misleading or harmful. Trained on 200K documents from [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) annotated by 
+Llama 4 Maverick (meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8).
+
+## Models
+
+This repo contains two models:
+
+### Binary (`binary/`)
+
+Classifies documents as **misinfo** or **benign**.
+
+| | Precision | Recall | F1 | Support |
+|---|---|---|---|---|
+| misinfo | 0.83 | 0.89 | 0.86 | 3,885 |
+| benign | 0.97 | 0.95 | 0.96 | 15,663 |
+| **accuracy** | | | **0.94** | 19,548 |
+
+### Multiclass (`multiclass/`)
+
+Classifies documents into 5 misinformation categories + benign.
+
+| | Precision | Recall | F1 | Support |
+|---|---|---|---|---|
+| climate_denial | 0.79 | 0.91 | 0.84 | 539 |
+| health_misinfo | 0.78 | 0.90 | 0.83 | 1,014 |
+| pseudoscience | 0.82 | 0.86 | 0.84 | 1,618 |
+| hate_extremism | 0.65 | 0.70 | 0.67 | 226 |
+| conspiracy_propaganda | 0.55 | 0.74 | 0.63 | 488 |
+| benign | 0.97 | 0.94 | 0.96 | 15,663 |
+| **accuracy** | | | **0.92** | 19,548 |
+
+## Training details
+
+- **Base model**: [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) (149M parameters)
+- **Training data**: 156,383 examples (from [ratishsp/fineweb-edu-misinfo](https://huggingface.co/datasets/ratishsp/fineweb-edu-misinfo))
+- **Validation**: 19,548 examples
+- **Test**: 19,548 examples
+- **Epochs**: 3
+- **Batch size**: 8 per GPU, 8 GPUs (AMD MI250X on LUMI)
+- **Learning rate**: 2e-5
+- **Warmup**: 10% of total steps
+- **Weight decay**: 0.01
+- **Max sequence length**: 8,192 tokens
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+# Binary model
+tokenizer = AutoTokenizer.from_pretrained("ratishsp/fineweb-edu-misinfo-classifier", subfolder="binary")
+model = AutoModelForSequenceClassification.from_pretrained("ratishsp/fineweb-edu-misinfo-classifier", subfolder="binary")
+
+text = "Your document text here..."
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=8192)
+with torch.no_grad():
+    logits = model(**inputs).logits
+    prediction = torch.argmax(logits, dim=-1).item()
+    label = model.config.id2label[prediction]
+    print(label)  # "misinfo" or "benign"
+```
+
+## Limitations
+
+- Annotations were produced by an LLM (Llama 4 Maverick), not human annotators. Inter-annotator agreement with Claude Sonnet 4.6 on 600 documents: binary kappa = 0.862, multiclass kappa = 0.842.
+- The model was trained on content from known problematic domains and random FineWeb-Edu samples. It may not generalize well to misinformation styles not represented in the training data.
+- The conspiracy_propaganda (F1 = 0.63) and hate_extremism (F1 = 0.67) categories have lower performance, likely due to less training data and more ambiguous boundaries.
+
+## Citation
+
+```bibtex
+@misc{puduppully2026fineweb-edu-misinfo,
+  author = {Puduppully, Ratish},
+  title = {FineWeb-Edu Misinformation Classifier},
+  year = {2026},
+  publisher = {HuggingFace},
+  url = {https://huggingface.co/ratishsp/fineweb-edu-misinfo-classifier}
+}
+```