sergiopaniego/huggingface-static-1a5eab-bucket
266 kB
Sergio Paniego PRO
sergiopaniego
AI & ML interests
None yet
Recent Activity
updated a dataset about 1 hour ago
huggingface-projects/Deep-RL-Course-Certification updated a dataset about 6 hours ago
agents-course/final-certificates updated a dataset about 6 hours ago
agents-course/course-certificates-of-excellenceOrganizations
Posts 85
Post
1252
Earlier this month, Apple introduced Simple Self-Distillation: a fine-tuning method that improves models on coding tasks just by sampling from the model and training on its own outputs with plain cross-entropy
And… it's already supported in TRL, built by Kashif Rasul. you can really feel the pace of development in the team 🐎
Paper by Ruixiang ZHANG, He Bai, Huangjie Zheng, Navdeep Jaitly, Ronan Collobert, Yizhe Zhang at Apple 🍎
How it works: the model generates completions at a training-time temperature (T_train) with top_k/top_p truncation, then fine-tunes on them with plain cross-entropy. no labels or verifier needed
You can try it right away with this ready-to-run example (Qwen3-4B on rStar-Coder):
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd.py
or benchmark a checkpoint with the eval script:
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd_eval.py
One neat insight from the paper: T_train and T_eval compose into an effective T_eff = T_train × T_eval, so a broad band of configs works well. even very noisy samples still help
Want to dig deeper?
Paper: Embarrassingly Simple Self-Distillation Improves Code Generation (2604.01193)
Trainer docs: https://huggingface.co/docs/trl/main/en/ssd_trainer
And… it's already supported in TRL, built by Kashif Rasul. you can really feel the pace of development in the team 🐎
Paper by Ruixiang ZHANG, He Bai, Huangjie Zheng, Navdeep Jaitly, Ronan Collobert, Yizhe Zhang at Apple 🍎
How it works: the model generates completions at a training-time temperature (T_train) with top_k/top_p truncation, then fine-tunes on them with plain cross-entropy. no labels or verifier needed
You can try it right away with this ready-to-run example (Qwen3-4B on rStar-Coder):
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd.py
or benchmark a checkpoint with the eval script:
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd_eval.py
One neat insight from the paper: T_train and T_eval compose into an effective T_eff = T_train × T_eval, so a broad band of configs works well. even very noisy samples still help
Want to dig deeper?
Paper: Embarrassingly Simple Self-Distillation Improves Code Generation (2604.01193)
Trainer docs: https://huggingface.co/docs/trl/main/en/ssd_trainer
Articles 16
Article
884
Welcome Gemma 4: Frontier multimodal intelligence on device
In-depth technical articles and research pieces published by Hugging Face
-
The Ultra-Scale Playbook
🌌3.82kThe ultimate guide to training LLM on large GPU Clusters
-
The Smol Training Playbook
📚3.14kThe secrets to building world-class LLMs
-
Evaluation Guidebook
📝312Explore LLM benchmark trends over time
-
FineVision: Open Data is All You Need
📝223A new open-source dataset for training VLMs
Blog: https://huggingface.co/blog/sergiopaniego/bringing-carla-to-openenv-trl/
In-depth technical articles and research pieces published by Hugging Face
-
The Ultra-Scale Playbook
🌌3.82kThe ultimate guide to training LLM on large GPU Clusters
-
The Smol Training Playbook
📚3.14kThe secrets to building world-class LLMs
-
Evaluation Guidebook
📝312Explore LLM benchmark trends over time
-
FineVision: Open Data is All You Need
📝223A new open-source dataset for training VLMs
spaces 135
pinned
Running on Zero
Agents
Featured
115
VLM Object Understanding
🦀
Explore object detection, visual grounding, keypoint Detecti
pinned
Runtime error
Agents
11
Qwen2-VL-7B
🔥
Ask questions about charts in images
pinned
Running on Zero
Agents
SmolVLM-trl-dpo-rlaif-v
🔥
Generate text from an image and question
pinned
Runtime error
Agents
SmolVLM-trl-sft-ChartQA
🔥
Ask questions about charts in images
Running
Huggingface Static 1a5eab
🎯
View and monitor key metrics with an interactive dashboard
Running
Huggingface Static 71a48c
🎯
Explore and monitor your data with an interactive dashboard
models 123
sergiopaniego/reasoning-gym-chain-sum-Qwen3-1.7B
Text Generation • 2B • Updated • 616
sergiopaniego/carla-vlm-gemma-test
Updated
sergiopaniego/carla-vlm-qwen35-test
Updated
sergiopaniego/carla-vlm-gemma
Updated
sergiopaniego/carla-vlm-qwen35
Updated
sergiopaniego/nemotron-3-sft
Updated
sergiopaniego/Qwen3-0.6B-carla-trolley-escape
0.8B • Updated • 5
sergiopaniego/tiny-aya-global-SFT
Updated
sergiopaniego/nemo3-sft-bnb
Updated
sergiopaniego/rloo_tldr_test
Updated
datasets 7
sergiopaniego/ttt-scripted-smoke
Viewer • Updated • 20 • 38
sergiopaniego/browsergym-grpo-functiongemma-270m-it-dataset
Viewer • Updated • 105 • 44 • 1
sergiopaniego/sample_videos
Viewer • Updated • 2 • 14
sergiopaniego/difficult_prompts
Viewer • Updated • 38 • 15
sergiopaniego/ourworldindata_example
Viewer • Updated • 13 • 226 • 1
sergiopaniego/faiss_embeddings
Updated • 19
sergiopaniego/CarlaFollowLanePreviousV
Viewer • Updated • 59.6k • 18