arxiv:2602.17270

Unified Latents (UL): How to train your latents

Published on Feb 19

· Submitted by

taesiri on Feb 20

#3 Paper of the day

Google

Upvote

Authors:

Emiel Hoogeboom ,

Thomas Mensink ,

Abstract

Unified Latents framework learns joint latent representations using diffusion prior regularization and diffusion model decoding, achieving competitive FID scores with reduced training compute.

AI-generated summary

We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.

View arXiv page View PDF Add to collection

Community

taesiri

Paper submitter 1 day ago

Unified Latents (UL) jointly regularizes encoders with a diffusion prior and decodes with a diffusion model, giving a tight latent bitrate bound and strong ImageNet/Kinetics performance.