Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-sonnet-4-20250514· AnthropicHugging Face LMArena Human Preference 140K

Hey, I'm doing up a workshop on evaluation metrics in LLM training. Assuming some of the…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-flash’s answer.
  1. User

    Hey, I'm doing up a workshop on evaluation metrics in LLM training. Assuming some of the audience have minimal background in math and struggle with equations, explain in simple and concise terms: accuracy rate vs recall rate vs precision rate vs false positive rate vs overkill rate vs escape rate vs leakage rate. Like are any of these the same thing and in what scenario would any of these specific metrics be useful for? Give accurate formulas too

  2. Claude Assistant

    Here's a breakdown of these evaluation metrics in simple terms:

    Accuracy Rate measures how often your model gets it right overall. It's the percentage of all predictions that were correct. Formula: (Correct Predictions) / (Total Predictions)

    Precision Rate asks: "When the model says YES, how often is it actually right?" It focuses on avoiding false alarms. Formula: (True Positives) / (True Positives + False Positives)

    Recall Rate asks: "Of all the actual YES cases, how many did the model catch?" It focuses on not missing important cases. Formula: (True Positives) / (True Positives + False Negatives)

    False Positive Rate measures how often the model incorrectly says YES when the answer should be NO. Formula: (False Positives) / (False Positives + True Negatives)

    Overkill Rate and False Positive Rate are the same thing - just different names used in different contexts.

    Escape Rate and False Negative Rate are the same - they measure how often the model misses actual positive cases. Formula: (False Negatives) / (False Negatives + True Positives). This is actually 1 minus the Recall Rate.

    Leakage Rate isn't a standard evaluation metric - it typically refers to data contamination where training data accidentally appears in test data, which invalidates your results.

    When to use which:

    • Accuracy for balanced datasets where all errors are equally bad
    • Precision when false alarms are costly (like spam detection - you don't want real emails marked as spam)
    • Recall when missing cases is dangerous (like medical diagnosis - you don't want to miss diseases)
    • False Positive Rate in security contexts where you need to minimize false alarms

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.