TomerBo/RadarGen_600M_512px_TS

RadarGen: Automotive Radar Point Cloud Generation from Cameras

Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany

💡 TL;DR: RadarGen generates sparse radar point clouds from multi-view camera images.

Abstract

We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery. RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form that encodes spatial structure together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Conditioning on images makes the approach broadly compatible, in principle, with existing visual datasets and simulation frameworks, offering a scalable direction for multimodal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step toward unified generative simulation across sensing modalities.

Overview

This repository contains the pre-trained weights for RadarGen, trained on the MAN TruckScenes dataset.

Quick Start

Run inference on TruckScenes using the notebook provided on GitHub: notebooks/inference_truckscenes.ipynb

Or use the Python API:

from radargen.inference import RadarGenInference
from radargen.datasets import get_adapter

# Load dataset adapter
adapter = get_adapter("truckscenes", trucksc=trucksc_obj)

# Initialize model
model = RadarGenInference(
    adapter=adapter,
    config_path="configs/RadarGen_600M_512px_TS_inference.yaml",
    checkpoint_path="hf://TomerBo/RadarGen_600M_512px_TS/RadarGen_600M_512px_TS.safetensors"
)

# Generate point cloud from two consecutive frames
pcl = model.from_sample_data(sample_t0, sample_t1)

Check out our Github Repo for more information.

License

This model inherits its license from SANA weights (NSCL v2-custom) and the MAN TruckScenes dataset (CC BY-NC-SA 4.0). Non-commercial use only.

📖 BibTeX

If you find our work useful, please consider starring ⭐ the repository and citing our paper:

@article{borreda2025radargen,
      title={RadarGen: Automotive Radar Point Cloud Generation from Cameras},
      author={Borreda, Tomer and Ding, Fangqiang and Fidler, Sanja and Huang, Shengyu and Litany, Or},
      journal={arXiv preprint arXiv:2512.17897},
      year={2025}
}