Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-flash· GoogleHugging Face LMArena Human Preference 140K

Tell me about how LLM Watermarking works.

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with chatgpt-4o-latest-20250326’s answer.
  1. User

    Tell me about how LLM Watermarking works.

  2. Gemini Assistant

    LLM watermarking is a technique designed to embed a detectable, secret signal into text generated by a Large Language Model (LLM) at the time of generation. The goal is to later identify whether a piece of text originated from a specific watermarked LLM.

    Unlike traditional image watermarking where you directly manipulate pixels, text is discrete. You can't just add "noise" without changing the words. LLM watermarking works by subtly biasing the LLM's token (word/subword) choices during the text generation process.

    Here's a breakdown of how it generally works:

    Core Idea: Biasing Token Probabilities

    At each step of text generation, an LLM predicts the probability distribution over its entire vocabulary for the next token. Normally, it samples a token based on these probabilities (e.g., picking the most likely token, or using methods like nucleus sampling).

    Watermarking interferes with this process by making certain tokens more or less likely to be chosen, based on a secret key and the preceding context.

    The Embedding Process (During Generation):

    1. Secret Key & Pseudorandom Function (PRF):

      • The LLM's owner/operator uses a secret key (or a seed).
      • At each generation step, this key, combined with a function of the immediately preceding tokens (the context window), is used to deterministically classify the next possible tokens in the vocabulary.
    2. "Green List" / "Red List" Token Partitioning:

      • Based on the PRF output for the current context, the LLM's entire vocabulary is pseudo-randomly partitioned into two sets:
        • "Green List" tokens: These are the tokens that the watermarking algorithm prefers the LLM to choose.
        • "Red List" tokens: These are the tokens the algorithm discourages the LLM from choosing.
      • Crucially, this partitioning is context-dependent. The "green" tokens for one word might be different for the next word, even if they're semantically similar. This makes the watermark robust and less predictable.
    3. Logit Modification (Biasing):

      • After the LLM calculates its initial predicted probabilities (logits) for the next token, the watermarking algorithm intervenes:
        • It increases the logits for tokens on the "Green List."
        • It decreases the logits (or penalizes) for tokens on the "Red List."
      • This subtle adjustment steers the LLM towards selecting "green" tokens more often than it would naturally, without drastically altering the text's meaning or fluency. The strength of this bias is a tunable parameter – too strong, and the text becomes unnatural; too weak, and the watermark is undetectable.
    4. Sampling:

      • After the biased logits are calculated, the LLM then samples a token from this modified probability distribution. This chosen token is now "watermarked" according to the secret key and context.

    This process repeats for every token generated, embedding a subtle, statistical signal throughout the output text.

    The Detection Process:

    1. Re-partitioning & Counting:

      • To check if a piece of text is watermarked, a detector (which possesses the same secret key and PRF used during generation) processes the text token by token.
      • For each token, it uses the preceding context and the secret key to re-create the "Green List" and "Red List" that would have been in effect at that generation step.
      • It then counts how many of the actual tokens in the text fall into the "Green List."
    2. Statistical Analysis:

      • If the text were not watermarked, the number of "green" tokens chosen would roughly match what's expected by random chance (e.g., if 50% of tokens are green, about 50% of tokens in a random text would be green).
      • If the text is watermarked, the count of "green" tokens will be significantly higher than random chance.
      • A statistical test (e.g., calculating a Z-score) is performed to determine the probability that the observed "green" token count occurred by chance. If this probability is very low (e.g., p-value < 0.05), the text is flagged as watermarked.

    Key Characteristics & Trade-offs:

    • Imperceptibility: The watermark should not visibly alter the quality, coherence, or fluency of the generated text. This is a trade-off with detectability.
    • Robustness: The watermark should ideally survive common text transformations like paraphrasing, summarization, minor edits, or even translation. This is the hardest challenge, as these operations can easily destroy the subtle statistical signal.
    • Detectability: The watermark should be reliably detectable with a low rate of false positives (marking human text as AI) or false negatives (missing an AI-generated text).
    • Computational Overhead: The embedding process should be efficient enough not to significantly slow down text generation.
    • Key Management: The secret key must be kept secure to prevent unauthorized detection or removal.

    Use Cases:

    • Attribution: Identifying text generated by a specific model or organization.
    • Misinformation Detection: Helping to flag AI-generated content that could be used for malicious purposes (e.g., deepfakes of text).
    • Plagiarism & Academic Integrity: Detecting if students are submitting AI-generated essays.
    • Content Moderation: Identifying synthetic content on platforms.

    Challenges and Limitations:

    • Robustness against adversarial attacks: Simple paraphrasing tools or even manual editing can often remove the watermark. More sophisticated methods like fine-tuning an LLM on watermarked text can also degrade the signal.
    • Generalizability: A watermark embedded by one model might not be detectable by another if they don't share the same key or algorithm.
    • Ethical Concerns: Who decides what gets watermarked? Could it be used for censorship or to stifle legitimate AI uses? There are also privacy concerns if watermarks contain identifying information.
    • False Positives/Negatives: No system is perfect, and there's always a risk of misclassification.

    LLM watermarking is an active area of research, with ongoing efforts to improve robustness, imperceptibility, and efficiency.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.