Researchers have released CORA-Diff, a training-free method that accelerates how diffusion language models generate text, alongside open-source code that reproduces its results (arXiv paper).
The slowdown starts with how diffusion language models work. They revise many token positions at once, but most practical decoders still run a fixed number of denoising steps, so positions that already settled keep getting recomputed in repeated dense forward passes. Earlier accelerators attacked this with learned filters, modified scoring functions, dependency models, or cache-specific tricks that change the model or its outputs.
CORA-Diff takes a different route. It keeps the model’s original acceptance rule and adds a second gate based on confidence and persistence, applied only to the tokens that rule still leaves open. Tokens the gate accepts stay in the context window, and decoding for that block ends once no positions remain unresolved. The authors argue, with paired trajectory measurements, that high-confidence persistent predictions are the ones most likely to match the final dense-decoding result.
The benchmark runs tell a clear story. In a matched Learn2PD-style LLaDA evaluation across eight task-length settings, the authors measured CORA-Diff turning in the fastest runtime every time, and on five of those settings its task scores equalled or surpassed ordinary dense decoding; the single largest score penalty was 1.22 points. Next to EOS-aware dense decoding it was 2.70x faster on GSM8K and 3.32x faster on HumanEval, climbing to 13.14x under the fixed 1024/1024 horizon isolation test, and it transferred to Dream without retuning at 3.18x to 3.53x (arXiv paper).
The method is already usable outside the paper (GitHub repository). The released code ships the CORA-Diff implementation next to three baselines — Prophet, KLASS, and DAPD — under one shared evaluation interface, with scripts to reproduce the main table and a smoke test for quick verification.
For teams weighing diffusion against autoregressive models, the result reframes the inference-cost question. A recent survey of eight diffusion language models across eight benchmarks maps the broader tradeoffs that methods like CORA-Diff now aim to shrink (zBrandCo coverage). If the 13x isolation result from the published paper holds in production pipelines (arXiv paper), diffusion text generation could move from a research curiosity to a practical option for latency-sensitive serving.
