Edit-Based Refinement for Parallel Masked Diffusion Language Models

This repository contains the Stage 3 checkpoint for ME-DLM, as presented in the paper Edit-Based Refinement for Parallel Masked Diffusion Language Models.

Authors: Houxing Ren, Mingjie Zhan, Zimu Lu, Ke Wang, Yunqiao Yang, Haotian Hou, Junting Pan, Hongsheng Li.

๐Ÿ“„ Paper โ€ข ๐Ÿ  Repo โ€ข ๐Ÿค– Models

Introduction

ME-DLM is a lightweight edit-based refinement framework for masked diffusion language models. It first generates a complete response through parallel diffusion decoding, then refines the output with minimal edit operations such as replacement, deletion, and insertion, conditioned on the full sequence. By using edit distance as deterministic training supervision, ME-DLM improves sequence-level consistency while preserving the decoding efficiency of diffusion models. Built on LLaDA, it achieves consistent gains on HumanEval and GSM8K while using only one-eighth of the total diffusion steps.

Models

Model Checkpoint
ME-DLM Stage 1 ๐Ÿค— HF Link
ME-DLM Stage 2 ๐Ÿค— HF Link
ME-DLM Stage 3 ๐Ÿค— HF Link

Citation

@article{ren2025edit,
  title={Edit-Based Refinement for Parallel Masked Diffusion Language Models},
  author={Ren, Houxing and Zhan, Mingjie and Lu, Zimu and Ke Wang and Yang, Yunqiao and Hou, Haotian and Pan, Junting and Li, Hongsheng},
  journal={arXiv preprint arXiv:2605.09603},
  year={2025}
}

Acknowledgments

We thank the following amazing projects that truly inspired us:

Downloads last month
26
Safetensors
Model size
9B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for renhouxing/ME-DLM-Stage2

Finetuned
(7)
this model

Paper for renhouxing/ME-DLM-Stage2