Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek-r1-0528· DeepSeekHugging Face LMArena Human Preference 140K

I have a multiclass output task. I have a pytorch tensor with 10 items, representing the…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-flash’s answer.
  1. User

    I have a multiclass output task.

    I have a pytorch tensor with 10 items, representing the forecasted image. When I would use the argmax function i would know what item was forecasted.

    Now that I have a tensor with 10 items in it. How can i transform the tensor to get the proba for the items and what proba value to extract from this list of 10 to get an Auc score value in sklearn

  2. DeepSeek Assistant

    To handle a multiclass classification task in PyTorch and compute the AUC score using scikit-learn, follow these steps:

    1. Convert Logits to Probabilities

    Apply the softmax function to the raw logits (your 10-element tensor) to obtain class probabilities:

    import torch.nn.functional as F
    
    logits = torch.tensor([...])  # Your tensor of shape (10,)
    probabilities = F.softmax(logits, dim=0)  # Shape: (10,)
    

    2. AUC Score Calculation (Requires Multiple Samples)

    AUC cannot be computed from a single prediction. You need a batch of predictions and their true labels:

    • Predictions: A tensor of shape (N, 10) (softmax probabilities for N samples).
    • True Labels: Integer tensor of shape (N,) with class indices (0 to 9).
    # Example for a batch of 100 samples
    batch_logits = torch.randn(100, 10)  # Raw logits (100 samples, 10 classes)
    batch_probs = F.softmax(batch_logits, dim=1)  # Probabilities
    
    true_labels = torch.randint(0, 10, (100,))  # Ground truth labels (0 to 9)
    

    3. Compute AUC with scikit-learn

    Use the roc_auc_score function with the multi_class='ovr' (One-vs-Rest) strategy:

    from sklearn.metrics import roc_auc_score
    
    # Convert tensors to numpy
    probs_np = batch_probs.numpy()        # Shape (100, 10)
    labels_np = true_labels.numpy()       # Shape (100,)
    
    # Calculate AUC (macro-average over classes)
    auc = roc_auc_score(
        labels_np, 
        probs_np, 
        multi_class='ovr', 
        average='macro'  # Or 'weighted'
    )
    print(f"Macro AUC: {auc:.4f}")
    

    Key Notes:

    • Single Sample? AUC requires multiple samples to measure ranking performance. It cannot be computed for one prediction.
    • AUC Strategies:
      • multi_class='ovr': Treats each class as "positive" and others as "negative".
      • average='macro': Unweighted mean of AUCs for all classes.
      • average='weighted': AUCs weighted by class support.
    • Input Requirements:
      • y_true: 1D array of integer class labels.
      • y_score: 2D array of shape (N, C) with class probabilities (each row sums to 1).

    Full Example:

    import torch
    import torch.nn.functional as F
    from sklearn.metrics import roc_auc_score
    
    # Generate dummy data (batch of 100 samples)
    batch_logits = torch.randn(100, 10)
    true_labels = torch.randint(0, 10, (100,))
    
    # Convert logits to probabilities
    probs = F.softmax(batch_logits, dim=1)
    
    # Compute AUC
    auc = roc_auc_score(
        true_labels.numpy(),
        probs.numpy(),
        multi_class='ovr',
        average='macro'
    )
    print(f"Macro AUC: {auc}")
    

    Output:

    Macro AUC: 0.9823  # Example result (varies with random data)
    

    Common Pitfalls:

    1. Single Prediction AUC: Not possible—AUC requires multiple samples.
    2. Logits vs. Probabilities: Ensure you pass probabilities (softmax output), not raw logits.
    3. Labels Format: True labels must be integers (e.g., [0, 2, 1, ...]), not one-hot encoded.

    By following these steps, you can compute multiclass AUC scores for your PyTorch model using scikit-learn.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.