Skip to content
The Chat Archive
gemini-2.5-pro-preview-03-25· Google LMArena Human Preference 140K

Diffusion Models in language : Until now 2025/april. Do a table with the year and month…

  1. User

    Diffusion Models in language : Until now 2025/april. Do a table with the year and month of publication, name of the paper, the resume of the contribuition to the fild of the most importante papers, link to code (if exist) and link to playground (if exist).

  2. Assistant

    Okay, here is a table summarizing some of the most important papers on diffusion models applied to language, up to approximately April 2025.

    Note:

    • "Importance" is subjective, but this list focuses on papers that introduced foundational concepts for discrete/language diffusion, demonstrated key capabilities (like controllability), or proposed influential methods.
    • Diffusion models for images (like DDPM by Ho et al., 2020) were foundational for the technique but are not listed here unless they specifically addressed discrete data relevant to language. However, they are crucial background.
    • The field is rapidly evolving, so many other valuable papers exist.
    • Playgrounds/Demos are less common for these specific research papers compared to code repositories.

    Diffusion Models in Language - Key Papers (Until ~April 2025)

    Year / MonthPaper TitleContribution ResumeCode LinkPlayground/Demo Link
    2015 / JunDeep Unsupervised Learning using Nonequilibrium ThermodynamicsFoundational: Introduced the core concept of diffusion probabilistic models for generative modeling. Mentioned potential for discrete data, laying theoretical groundwork.Yes (Theano)No
    2021 / FebScore-Based Generative Modeling through Stochastic Differential Equations (SDE)Foundational Framework: Unified previous diffusion/score-matching methods under the umbrella of SDEs. Discussed applicability to discrete data types, providing a continuous-time perspective.Yes (Multiple Implementations)No
    2021 / JulStructured Denoising Diffusion Models in Discrete State-SpacesDiscrete Diffusion: One of the first key works directly tackling diffusion for discrete data (like text tokens) by defining diffusion processes over discrete state spaces (e.g., categorical distributions).Yes (JAX)No
    2022 / MayDiffusion-LM Improves Controllable Text GenerationBreakthrough Application & Controllability: Showed a practical way to apply diffusion to text by operating in the continuous embedding space. Demonstrated significantly improved controllability over classifiers or PPLM for imposing constraints (syntax, keywords). Highly influential.Yes (PyTorch)No
    2022 / OctDiffuSeq: Sequence to Sequence Text Generation with Diffusion ModelsSeq2Seq Adaptation: Adapted the diffusion framework for sequence-to-sequence tasks (like translation, summarization), often using partial noising or conditioning on the source sequence. Explored non-autoregressive generation benefits.Yes (PyTorch)No
    2022 / OctDiffuEdit: Diffusion-based Testing Editing with Reinforcement LearningText Editing Application: Leveraged the iterative refinement nature of diffusion models specifically for the task of text editing, allowing controlled modifications while maintaining fluency.Yes (PyTorch)No
    2022 / NovContinuous Diffusion for Categorical DataAlternative Discrete Approach: Proposed a method to adapt continuous Gaussian diffusion to categorical data by carefully designing the forward (noising) and reverse (denoising) processes, linking it to variational autoencoders.Yes (PyTorch)No
    2023 / FebAnalog Bits: Generating Discrete Data using Diffusion Models with Self-ConditioningNovel Embedding Technique: Introduced "Analog Bits," a method to represent discrete tokens as continuous vectors suitable for standard Gaussian diffusion, mapping integers to points on a hypersphere. Also incorporated self-conditioning.Yes (PyTorch)No
    ~Ongoing(Various Papers)(Refinements, Scaling, Specific Tasks) Subsequent works continue to refine efficiency, explore different noising/denoising schedules, scale models, improve training stability, and apply diffusion to more niche language tasks (e.g., code generation, molecule generation from text).(Varies)(Varies)

    This table provides a snapshot of the key developments. Remember that research builds incrementally, and many other papers have contributed significantly to understanding and improving diffusion models for language.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.