alpaca-13b· Stanford
Arena Human Preference 55K
How should I calculate the sample size required for an experiment where I compare two…
- User
How should I calculate the sample size required for an experiment where I compare two models on the same set of data?
- Assistant
The formula for calculating the sample size required for an experiment where you compare two models on the same set of data is n = m+p, where m is the number of successes in the smaller sample, and p is the probability of success in the smaller sample.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.