Diffusion Models in language : Until now 2025/april. Do a table with the year and month…
- User
Diffusion Models in language : Until now 2025/april. Do a table with the year and month of publication, name of the paper, the resume of the contribuition to the fild of the most importante papers, link to code (if exist) and link to playground (if exist).
- Assistant
Okay, here is a table summarizing some of the most important papers on diffusion models applied to language, up to approximately April 2025.
Note:
- "Importance" is subjective, but this list focuses on papers that introduced foundational concepts for discrete/language diffusion, demonstrated key capabilities (like controllability), or proposed influential methods.
- Diffusion models for images (like DDPM by Ho et al., 2020) were foundational for the technique but are not listed here unless they specifically addressed discrete data relevant to language. However, they are crucial background.
- The field is rapidly evolving, so many other valuable papers exist.
- Playgrounds/Demos are less common for these specific research papers compared to code repositories.
Diffusion Models in Language - Key Papers (Until ~April 2025)
Year / Month Paper Title Contribution Resume Code Link Playground/Demo Link 2015 / Jun Deep Unsupervised Learning using Nonequilibrium Thermodynamics Foundational: Introduced the core concept of diffusion probabilistic models for generative modeling. Mentioned potential for discrete data, laying theoretical groundwork. Yes (Theano) No 2021 / Feb Score-Based Generative Modeling through Stochastic Differential Equations (SDE) Foundational Framework: Unified previous diffusion/score-matching methods under the umbrella of SDEs. Discussed applicability to discrete data types, providing a continuous-time perspective. Yes (Multiple Implementations) No 2021 / Jul Structured Denoising Diffusion Models in Discrete State-Spaces Discrete Diffusion: One of the first key works directly tackling diffusion for discrete data (like text tokens) by defining diffusion processes over discrete state spaces (e.g., categorical distributions). Yes (JAX) No 2022 / May Diffusion-LM Improves Controllable Text Generation Breakthrough Application & Controllability: Showed a practical way to apply diffusion to text by operating in the continuous embedding space. Demonstrated significantly improved controllability over classifiers or PPLM for imposing constraints (syntax, keywords). Highly influential. Yes (PyTorch) No 2022 / Oct DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models Seq2Seq Adaptation: Adapted the diffusion framework for sequence-to-sequence tasks (like translation, summarization), often using partial noising or conditioning on the source sequence. Explored non-autoregressive generation benefits. Yes (PyTorch) No 2022 / Oct DiffuEdit: Diffusion-based Testing Editing with Reinforcement Learning Text Editing Application: Leveraged the iterative refinement nature of diffusion models specifically for the task of text editing, allowing controlled modifications while maintaining fluency. Yes (PyTorch) No 2022 / Nov Continuous Diffusion for Categorical Data Alternative Discrete Approach: Proposed a method to adapt continuous Gaussian diffusion to categorical data by carefully designing the forward (noising) and reverse (denoising) processes, linking it to variational autoencoders. Yes (PyTorch) No 2023 / Feb Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning Novel Embedding Technique: Introduced "Analog Bits," a method to represent discrete tokens as continuous vectors suitable for standard Gaussian diffusion, mapping integers to points on a hypersphere. Also incorporated self-conditioning. Yes (PyTorch) No ~Ongoing (Various Papers) (Refinements, Scaling, Specific Tasks) Subsequent works continue to refine efficiency, explore different noising/denoising schedules, scale models, improve training stability, and apply diffusion to more niche language tasks (e.g., code generation, molecule generation from text). (Varies) (Varies) This table provides a snapshot of the key developments. Remember that research builds incrementally, and many other papers have contributed significantly to understanding and improving diffusion models for language.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.