Cosmos3-Edge

Mixture-of-Transformers architecture of Cosmos 3 (Figure 5 of the technical report). The understanding pathway reads the text prompt, the generation pathway reads the video latents and the actions, and the two meet in a shared attention.

NVIDIA Cosmos3-Edge in the LeRobot format. This is the base checkpoint for fine-tuning as a robot policy. Taking the camera images and a language instruction as input, the policy predicts the future video frames together with the next chunk of actions.

Item	Value
Model developer	NVIDIA
Original model	nvidia/Cosmos3-Edge
Policy type	cosmos3_edge
Backbone	Mixture-of-Transformers with a dense 2B transformer, 28 layers, hidden size 2048
Video tokenizer	Wan2.2 VAE
Parameters	4,563,688,396 (4.6B), of which 1,423,383,232 (1.4B) train by default
Weights	bfloat16, the timestep embedding in float32
License	OpenMDW 1.1

Fine-tuning

Install LeRobot from source with the Cosmos3 extras (the Cosmos3 policy is added in huggingface/lerobot#5134):

pip install -e ".[cosmos3,training]"

Cosmos3 Edge can be fine-tuned in full on one 80 GB GPU (about 47 GB for 32 samples per GPU with gradient checkpointing), for example on LIBERO-10:

lerobot-train \
  --policy.type=cosmos3_edge \
  --policy.pretrained_path=lerobot/Cosmos3-Edge \
  --policy.load_action_modules=false \
  --policy.gradient_checkpointing=true \
  --policy.repo_id=${HF_USER}/Cosmos3-Edge-Policy-LIBERO_10 \
  --dataset.repo_id=lerobot/LIBERO_10-Cosmos3 \
  --accelerator.mixed_precision=bf16 \
  --batch_size=32 \
  --steps=16000 \
  --output_dir=outputs/train/cosmos3_edge_libero_10

• --policy.load_action_modules=false starts the action modules from their initialization, as the native post-training does.
• The cameras, the action dims and the frame rate come from the dataset.
• config.json keeps dtype: float32 for float32 master weights in training. For inference and evaluation pass --policy.dtype=bfloat16.

The Cosmos3 documentation gives the full fine-tuning recipes (multi GPU, LIBERO, DROID), the dataset requirements and the evaluation.

Fine-tuned checkpoints

Checkpoint	Training data
lerobot/Cosmos3-Edge-Policy-LIBERO	lerobot/LIBERO-Cosmos3
lerobot/Cosmos3-Edge-Policy-LIBERO_10	lerobot/LIBERO_10-Cosmos3

Citation

@techreport{nvidia2026cosmos3,
  title  = {Cosmos 3: Omnimodal World Models for physical AI},
  author = {{NVIDIA Cosmos Team}},
  year   = {2026},
  url    = {https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf}
}

License

The NVIDIA model materials use OpenMDW 1.1. Retain the license and the notices of NOTICE when redistributing them. The NVIDIA model card describes the intended use and the limitations of the original model.