Hugging Face's logo
Activity Feed Request to join this org

AI & ML interests

None defined yet.

Recent Activity
oopereΒ  authored a paper about 22 hours ago
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
oopereΒ  submitted a paper 1 day ago
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
christopherΒ  new activity 3 days ago
blog-explorers/README:[Support] Community Articles
View all activity
sergiopaniegoΒ 
posted an update 2 days ago
view post
Post
2472
Simon Willison (@simonw ) has asked every new model to draw a pelican riding a bicycle for some time now

you look at the drawing and you know. but there is no number, so nothing can train against it, no?

I turned this idea into an rl env in OpenEnv. now, you can eval any model against it, and train against it with TRL

read the details!πŸ€“

https://huggingface.co/blog/sergiopaniego/pelican-env-openenv
  • 2 replies
Β·
AviSoori1xΒ 
authored 2 papers 3 days ago
sergiopaniegoΒ 
posted an update 3 days ago
christopherΒ 
in blog-explorers/README 3 days ago

[Support] Community Articles

πŸ”₯πŸš€ 2
109
#5 opened over 2 years ago by
victor
juiceb0xc0deΒ 
in blog-explorers/README 3 days ago

[Support] Community Articles

πŸš€πŸ€ 2
109
#5 opened over 2 years ago by
victor
sergiopaniegoΒ 
posted an update 5 days ago
view post
Post
2839
quick reminder! 🚨

tomorrow (Tuesday, July 28), we're back with Class 3 of the Training Agents live series

🧠 what: reinforcement learning for training agents (GRPO): how it works, how to implement it in TRL, and end-to-end examples
πŸ—“οΈ when: Tuesday, July 28 - πŸ•” 5:00 PM CEST / 8:30 PM IST
πŸ“ where: Live on @huggingface 's X, YouTube, and LinkedIn

live: https://www.youtube.com/watch?v=ztdTed5egrM

class 1: https://x.com/SergioPaniego/status/2069382207618379813
class 2: https://x.com/SergioPaniego/status/2075180665184686187
  • 1 reply
Β·
sergiopaniegoΒ 
posted an update 8 days ago
view post
Post
178
you can now train your own coding agents with trl + openenv, starting with opencode

we just added end-to-end support for training agent harnesses:

> TRL: a loop-owning training path (AsyncGRPOTrainer + HarnessRolloutWorker) that launches the agent in an OpenEnv session, reads back its trace, reconstructs the training samples, and trains with AsyncGRPO
> OpenEnv: the OpenCode harness environment plus a transparent proxy that forwards the agent's model calls and records each turn's token ids and logprobs

you train the actual opencode agent as is, it runs its own loop and tools and the policy learns from the exact tokens it produced

we're shipping a self-contained example: local subprocess sandbox, DeepCoder problems, validated on Qwen3-8B.

> example: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/opencode.py
> docs: https://huggingface.co/docs/trl/main/openenv

and we're working actively on both sides so expect more πŸ€“
  • 1 reply
Β·
sergiopaniegoΒ 
posted an update 9 days ago
sergiopaniegoΒ 
posted an update 10 days ago
view post
Post
211
join us next Tuesday, July 28, for Class 3 of the Training Agents live series!

we'll dive into reinforcement learning for agent training, covering the intuition behind GRPO, how it works, and how to implement it in TRL with practical, e2e examples

see you there 🀠

live: https://www.youtube.com/live/ztdTed5egrM

> in case you missed class 1:
https://x.com/SergioPaniego/status/2069382207618379813
> and in case you missed class 2: https://x.com/SergioPaniego/status/2075180665184686187
sergiopaniegoΒ 
posted an update 24 days ago
view post
Post
7721
Frontier models use distillation as a step of their post-training pipelines.

In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.

I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026

It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
  • 3 replies
Β·
AdinaYΒ 
submitted a paper to Daily Papers about 1 month ago
sergiopaniegoΒ 
posted an update about 1 month ago
sergiopaniegoΒ 
posted an update about 1 month ago
sergiopaniegoΒ 
posted an update about 2 months ago
view post
Post
339
GLM-5.2 is open and comes with competitive performance against opus 4.8

day-0 in transformers + vllm + sglang, mit license πŸ€—

on the post-training side: critic-based ppo for variable-length agentic rollouts (ppo is back!) + an online anti-reward-hacking module that feeds the agent dummy info when it tries to cheat
AdinaYΒ 
submitted 2 papers to Daily Papers about 2 months ago
sergiopaniegoΒ 
posted an update about 2 months ago
view post
Post
3968
OpenEnv has a new home: github.com/huggingface/OpenEnv

Starting today, it's coordinated by a committee that includes Meta-PyTorch, Reflection, Unsloth, Modal, Prime Intellect, Nvidia, Mercor, Fleet AI, and Hugging Face

frontier labs train their models and their harnesses together. Claude knows Claude Code. GPT-5.5 knows Codex. that's not an accident, it's training. open-source models deserve the same magic, but pulling that off requires infrastructure that belongs to everyone, not one lab

OpenEnv is that layer. one api, any harness, any trainer, any environment

Rewards and training loops stay in TRL, Unsloth, wherever you already work. OpenEnv is the socket they all plug into

Get involved!

Full announcement: https://huggingface.co/blog/openenv-agentic-rl
sergiopaniegoΒ 
posted an update about 2 months ago
view post
Post
360
Frontier agents are this good partly because the model was trained inside the very harness it ships with.

NVIDIA's new paper "Polar: Agentic RL on Any Harness at Scale" brings that recipe to the open: it turns coding harnesses like Codex, Claude Code, Qwen Code or Pi into RL training environments without touching their internals.

The core idea: every agent, however complex or closed, talks to a model through an API, so they put a proxy there. The harness runs exactly like in production while the proxy records prompts, sampled token ids and logprobs. Trajectories get rebuilt outside, token faithful, so gradients hit the exact tokens the policy sampled.

The gains are consistent across all four harnesses. Same Qwen3.5-4B, plain GRPO, evaluated on SWE-Bench Verified:

Codex 3.8 β†’ 26.4 (+22.6)
Claude Code 29.8 β†’ 34.6 (+4.8)
Qwen Code 34.6 β†’ 35.2 (+0.6)
Pi 34.2 β†’ 40.4 (+6.2)

The biggest gains appear on unfamiliar execution paths, Codex being the clearest case. The takeaway: you are not just training a model, you are training the model + harness system.

Two engineering pieces make it work at scale. Async worker pools isolate container boots (CPU), agent execution (GPU) and long tail test runs, so slow runtimes never block the GPUs. And prefix merging stitches hundreds of captured API calls back into contiguous traces: 5.4x faster trainer updates and rollout GPUs at 88% utilization.

It also doubles as an SFT data factory: 504 test verified agent traces from a 122B teacher, multi-turn conversations averaging 104 messages each, coming to the Hub under Apache 2.0 (release pending review).

Paper authors: Binfeng Xu, Hao Zhang, Shaokun Zhang, Songyang Han, Mingjie Liu, Jian Hu, Shizhe Diao, Zhenghui Jin, Yunheng Zou, Michael Demoret, Jan Kautz and Yi Dong.

> Paper: Polar: Agentic RL on Any Harness at Scale (2605.24220)
> Code: https://github.com/NVIDIA-NeMo/ProRL-Agent-Server
> Training data: NovaSky-AI/SkyRL-v0-293-data
sergiopaniegoΒ 
posted an update about 2 months ago
view post
Post
310
The recording from our talk: "From Responses To Trajectories: Multi-Turn and Multi-Environment RL" from PyTorch Conf Europe is live!

@kashif and I covered the latest advances in multi-turn GRPO in TRL: trajectories, tool use, envs, and agentic post-training at scale

https://www.youtube.com/watch?v=rPBeXFntJSU