Paper

Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training

arXiv:2602.10314v2 Announce Type: replace Abstract: Masked Diffusion Models (MDMs) have emerged as a promising approach for generative modeling in discrete spaces. By generating sequences in any order and allowing for parallel decoding, they enable fast inference and strong performance on non-causal tasks. However, this flexibility comes with a training complexity trade-off: MDMs train on an exponentially large set of masking patterns, which is not only computationally expensive, but also creates a train--test mismatch between the random masks used in training and the highly structured masks…

Authors:

Related discussions