Skip to content
TheChatArchiveThe Chat Archive
Mistralmixtral-8x7b-instruct-v0.1· Mistral AIHugging Face Arena Human Preference 55K

Consider the following MCQ question for a quiz on machine learning at MSc level. Come up…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with vicuna-13b’s answer.
  1. User

    Consider the following MCQ question for a quiz on machine learning at MSc level. Come up with additional wrong but plausible sounding answers. "Training and Testing/Validation must use different data sets:

    because estimating the accuracy using the same set used for training would very likely overestimate the true accuracy on unseen data
    because estimating the accuracy using the same set used for training would very likely underestimate the true accuracy on unseen data
    "
    
  2. Mistral Assistant

    Here are some additional wrong but plausible sounding answers for the given MCQ question:

    • Because using the same data set for training and testing would not provide any error estimate
    • Because using the same data set for training and testing would not allow for model selection
    • Because using the same data set for training and testing would not provide an unbiased estimate of the true accuracy
    • Because using the same data set for training and testing would not allow for the identification of overfitting
    • Because using the same data set for training and testing would not provide a realistic measure of the model's performance on new data.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.