Skip to content
The Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)

I am looking for a way to calculate how similar each number is compared to every other…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    I am looking for a way to calculate how similar each number is compared to every other number in a set. Here's an example: [61, 62, 59, 60, 63, 62, 58] would have a high "similarity score", where as [12, 96, 71, 66, 108, 23, 5, 59] would have a low "similarity score".

    Is there a term for this, and how would I calculate it?

  2. Assistant

    The term you are looking for is "standard deviation", the measure of variance between a set of values.

    To find the standard deviation

    1. Find the mean.
    2. For each data point, find the square of its distance to the mean.
    3. Sum the values from Step 2.
    4. Divide by the number of data points.
    5. Take the square root.

    To calculate standard deviation of an 1D array in Julia:

    n=std([61, 62, 59, 60, 63, 62, 58])
    

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.