Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)
I am looking for a way to calculate how similar each number is compared to every other…
In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
- User
I am looking for a way to calculate how similar each number is compared to every other number in a set. Here's an example:
[61, 62, 59, 60, 63, 62, 58]would have a high "similarity score", where as[12, 96, 71, 66, 108, 23, 5, 59]would have a low "similarity score".Is there a term for this, and how would I calculate it?
- Assistant
The term you are looking for is "standard deviation", the measure of variance between a set of values.
To find the standard deviation
- Find the mean.
- For each data point, find the square of its distance to the mean.
- Sum the values from Step 2.
- Divide by the number of data points.
- Take the square root.
To calculate standard deviation of an 1D array in Julia:
n=std([61, 62, 59, 60, 63, 62, 58])
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →