Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-3-7-sonnet-20250219-thinking-32k· AnthropicHugging Face LMArena Human Preference 140K

they're already sorted. [1, 0, 1, 1, 1, 0, 0, 0]

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-235b-a22b-no-thinking’s answer.
  1. User

    they're already sorted. [1, 0, 1, 1, 1, 0, 0, 0]

  2. Claude Assistant

    If the input is already sorted as [1, 0, 1, 1, 1, 0, 0, 0], then we can simplify the function since we don't need to sort by scores. We just need to calculate the Average Precision directly from this ranked list.

    import numpy as np
    
    mean_average_precision = lambda sorted_relevance: (
        np.sum(
            np.cumsum(sorted_relevance) / np.arange(1, len(sorted_relevance) + 1) 
            * sorted_relevance
        ) / np.sum(sorted_relevance)
    ) if np.sum(sorted_relevance) > 0 else 0
    

    Let's apply this to your example:

    # For the sorted array [1, 0, 1, 1, 1, 0, 0, 0]
    sorted_relevance = np.array([1, 0, 1, 1, 1, 0, 0, 0])
    ap = mean_average_precision(sorted_relevance)
    print(f"Average Precision: {ap}")
    

    This will calculate the Average Precision by summing the precision at each position where a relevant item (1) appears, divided by the total number of relevant items.

    Would you like me to explain or break down the code?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.