computationalanalysissocial_roles.pdfPDFI'm creating a replicant of this paper on my own…
- User
computational_analysis_social_roles.pdfPDFI'm creating a replicant of this paper on my own data. Which Python library can I use to calculate the cosine similarity of two contributions? Is this simple tf-idf?
- Assistant
Yes — for replicating the method in that paper, you can use cosine similarity on tf-idf vectors of contributions. Here’s how it fits together:
The paper uses Latent Semantic Analysis (LSA), which is built on tf-idf.
Each contribution is represented as a vector (from tf-idf, or after dimensionality reduction with SVD in LSA).
Cosine similarity is then calculated between these vectors to measure the semantic similarity of contributions .
In Python, you have a couple of standard library choices:
scikit-learn
TfidfVectorizer to get tf-idf vectors.
cosine_similarity from sklearn.metrics.pairwise to compute similarity.
TruncatedSVD if you want to replicate the LSA step used in the paper.
pythonCopyEditfrom sklearn.feature_extraction.text import TfidfVectorizer from sklearn.metrics.pairwise import cosine_similarity from sklearn.decomposition import TruncatedSVD
Example contributions
docs = ["First contribution text", "Second contribution text"]
Step 1: TF-IDF
vectorizer = TfidfVectorizer() X = vectorizer.fit_transform(docs)
Step 2 (optional): LSA with SVD
svd = TruncatedSVD(n_components=100) # pick dimensions X_reduced = svd.fit_transform(X)
Step 3: Cosine similarity
sim_matrix = cosine_similarity(X_reduced)
gensim
Provides easy tools for tf-idf, LSA, and cosine similarity.
So yes — it is tf-idf (optionally reduced with LSA), followed by cosine similarity. 👉 Do you want me to give you a full Python workflow that replicates the paper’s similarity pipeline step by step (preprocessing → tf-idf → LSA → cosine similarity)?
- User
Yes
- Assistant
Perfect 👍 — let’s build a step-by-step Python workflow that follows the approach in the paper:
🔹 Workflow: Replicating Contribution Similarity (like in the paper) We’ll go through:
Preprocessing text
TF-IDF vectorization
LSA (dimensionality reduction with SVD)
Cosine similarity between contributions
-
Import libraries pythonCopyEditimport re import numpy as np from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.decomposition import TruncatedSVD from sklearn.metrics.pairwise import cosine_similarity
-
Example contributions (replace with your data) pythonCopyEditcontributions = [ "I think personality disorders are caused by genetics.", "Some disorders are influenced by environmental factors.", "I agree that genetics plays a big role.", "Environmental stress is also very important." ]
-
Preprocessing (basic cleanup) pythonCopyEditdef preprocess(text): text = text.lower() text = re.sub(r"[^a-z\s]", "", text) # keep only letters return text
contributions_clean = [preprocess(c) for c in contributions]
-
TF-IDF Vectorization pythonCopyEditvectorizer = TfidfVectorizer(stop_words="english") X_tfidf = vectorizer.fit_transform(contributions_clean)
-
LSA with Truncated SVD The paper used Latent Semantic Analysis (LSA), which is TF-IDF reduced with SVD. pythonCopyEditn_components = 100 # choose dimensionality (often 100–500) svd = TruncatedSVD(n_components=n_components, random_state=42) X_lsa = svd.fit_transform(X_tfidf)
-
Cosine Similarity pythonCopyEditsimilarity_matrix = cosine_similarity(X_lsa) print(similarity_matrix)
This will output a matrix where similarity_matrix[i, j] = cosine similarity between contribution i and j.
✅ Notes
If your dataset is small, you may want a lower n_components (e.g. 50).
If you skip step 5 (SVD), you’re just doing TF-IDF + cosine similarity.
If you want to replicate the given–new or cross-cohesion measures from the paper, you’ll need to calculate similarity with lags (e.g., contribution vs. previous ones). I can show that too.
👉 Do you want me to also extend this to show how to compute responsivity / social impact (like they did in the Group Communication Analysis framework), or just keep it at raw similarity matrices?
-
- User
Help me continue this:
""" import pandas as pd from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.metrics.pairwise import cosine_similarity from sklearn.decomposition import TruncatedSVD
""" This script provides a framework for analyzing group discussions. Calculates the aspects from: https://link.springer.com/article/10.3758/s13428-018-1102-z """
TODO decide if to implement a class for variable storage
def calculate_participation(discussion_data: pd.DataFrame, participant: str) -> dict[float, list[int]]: """ Calculate participation score for a specific participant in a discussion. The logic: the discussion is split into contributions: c1, c2, ..., cn t is referred to as time: 1 <= t <= n k is the number of participants in the discussion. The participation function: p_a(t) = 1 if participant contributes at time t else 0 The participation function for a participant is the the sequence of contributions: p_a = (p_a(1), p_a(2), ..., p_a(n)) The number of contributions a participant makes is denoted as |p_a|, which is the sum of the sequence: |p_a| = p_a(1) + p_a(2) + ... + p_a(n) The sample mean participation of a participant is defined as: p_a_mean = |p_a| / n
Eventually, the participation score is calculated as: participation_score = p_a_mean - 1/k Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Participation score """ # Get the sequence of all authors contributors_lst = discussion_data['author'].tolist() # Total number of contributions in the discussion n_contributors = len(contributors_lst) # Number of unique participants in the discussion k = discussion_data['author'].nunique() # Create the participation sequence p_a for this participant # p_a(t) = 1 if participant contributes at time t, else 0 p_a = [1 if contributor == participant else 0 for contributor in contributors_lst] # Calculate |p_a| - total number of contributions by this participant p_a_total = sum(p_a) # Calculate sample mean participation p_a_mean = p_a_total / n_contributors # Calculate the participation score participation_score = p_a_mean - (1 / k) return {'participation_score': participation_score, 'participation_seq': p_a}def calc_cos(contribution_1: str, contribution_2: str) -> float: """ Calculate the cosine similarity between two contributions.
Args: contribution_1: The first contribution text contribution_2: The second contribution text Returns: float: Cosine similarity score between the two contributions """ passdef calc_semantic_similarity(discussion_data: pd.DataFrame, contribution_1: str, contribution_2: str) -> float: """ calculate the semantic similarity between two contributions: if contribution c_t was made by a and c_u was made by b: s_a,b(t, u) = p_a(t)*p_b(u)*cos(d_t, d_u) else: s_a,b(t, u) = 0
:param discussion_data: :param contribution_1: :param contribution_2: :return: """ # TODO hold a class with these values instead of calculating multiple times author_1 = discussion_data[discussion_data.text == contribution_1].author.values[0] author_2 = discussion_data[discussion_data.text == contribution_2].author.values[0] time_1 = discussion_data[discussion_data.text == contribution_1].contribution_id.values[0] time_2 = discussion_data[discussion_data.text == contribution_2].contribution_id.values[0] p_a_t = calculate_participation(discussion_data, author_1)['participation_seq'][time_1-1] p_b_u = calculate_participation(discussion_data, author_2)['participation_seq'][time_2-1] return p_a_t*p_b_u*calc_cos(contribution_1, contribution_2)def calculate_overall_responsivity(discussion_data: pd.DataFrame, participant: str, discussion_id: str) -> float: """ Calculate overall responsivity score for a specific participant in a discussion.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Overall responsivity score """ # TODO: Implement responsivity calculation logic # Example: Could be based on response times, response rates to others, etc. passdef calculate_internal_cohesion(discussion_data: pd.DataFrame, participant: str, discussion_id: str) -> float: """ Calculate internal cohesion score for a specific participant in a discussion.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Internal cohesion score """ # TODO: Implement internal cohesion calculation logic # Example: Could be based on consistency of messaging, topic coherence, etc. passdef calculate_social_impact(discussion_data: pd.DataFrame, participant: str, discussion_id: str) -> float: """ Calculate social impact score for a specific participant in a discussion.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Social impact score """ # TODO: Implement social impact calculation logic # Example: Could be based on influence on others, responses received, etc. passdef calculate_newness(discussion_data: pd.DataFrame, participant: str, discussion_id: str) -> float: """ Calculate newness score for a specific participant in a discussion.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Newness score """ # TODO: Implement newness calculation logic # Example: Could be based on novel topics introduced, unique vocabulary, etc. passdef calculate_communication_density(discussion_data: pd.DataFrame, participant: str, discussion_id: str) -> float: """ Calculate the communication density score for a specific participant in a discussion.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Communication density score """ # TODO: Implement communication density calculation logic # Example: Could be based on message frequency, interaction patterns, etc. pass - Assistant
Nice work — you’ve already got the participation measure implemented in line with the paper. Let’s continue by filling in the cosine similarity and semantic similarity parts so that other GCA measures (responsivity, cohesion, impact, etc.) can be built on top. Here’s the continuation of your script with working implementations: pythonCopyEditdef calc_cos(contribution_1: str, contribution_2: str) -> float: """ Calculate the cosine similarity between two contributions using TF-IDF (optionally reducible with SVD for LSA). """ vectorizer = TfidfVectorizer(stop_words="english") tfidf_matrix = vectorizer.fit_transform([contribution_1, contribution_2])
# Optional: reduce dimensionality with SVD (LSA) if tfidf_matrix.shape[1] > 100: # only if high-dimensional svd = TruncatedSVD(n_components=100, random_state=42) reduced = svd.fit_transform(tfidf_matrix) sim = cosine_similarity(reduced)[0, 1] else: sim = cosine_similarity(tfidf_matrix)[0, 1] return float(sim)def calc_semantic_similarity( discussion_data: pd.DataFrame, contribution_1: str, contribution_2: str ) -> float: """ Calculate the semantic similarity between two contributions following the GCA definition:
s_a,b(t, u) = p_a(t) * p_b(u) * cos(d_t, d_u) where p_a(t) is 1 if participant a contributed at time t, else 0. """ # Find metadata row_1 = discussion_data[discussion_data.text == contribution_1].iloc[0] row_2 = discussion_data[discussion_data.text == contribution_2].iloc[0] author_1, time_1 = row_1.author, row_1.contribution_id author_2, time_2 = row_2.author, row_2.contribution_id # Get participation sequence values p_a_t = calculate_participation(discussion_data, author_1)['participation_seq'][time_1 - 1] p_b_u = calculate_participation(discussion_data, author_2)['participation_seq'][time_2 - 1] # Cosine similarity of texts cos_sim = calc_cos(contribution_1, contribution_2) return p_a_t * p_b_u * cos_sim✅ Now you can do: pythonCopyEditsimilarity = calc_semantic_similarity(discussion_data, "I think personality...", "Some disorders...") print(similarity)
Next steps (to extend to other GCA metrics)
Responsivity → average semantic similarity of a participant’s contributions to others’ prior contributions (lagged).
Internal Cohesion → average similarity of a participant’s contributions to their own prior contributions.
Social Impact → average similarity of others’ contributions after a participant’s contribution.
Newness → project contribution vector onto prior contributions’ span (given vs new info).
Communication Density → ‖vector‖ / len(words).
👉 Do you want me to implement responsivity and internal cohesion next, so your script can already compute them in line with the equations from the paper?
- User
No. Im ready to implement this:
def calc_cross_cohesion_matrix(discussion_data: pd.DataFrame) -> tuple[list[list[float]], list[str]]: """ computes the cross-cohesion matrix for a given lag of the discussion. The element in row i and column j is the cross-cohesion value for authors i and j. For two authors a and b, the cross-cohesion value a lag Tao is defined as: c_cohesion_a,b(Tao) = (sim_a_b(1,Tao+1) + sim_a_b(2,Tao+2) + ... + sim_a_b(n-Tao, n))/norm(p_ab(Tao))
and norm(p_ab(Tao)) is defined as the sum: p_a(1)p_b(Tao+1) + p_a(2)p_b(Tao+2) + ... + p_a(n-Tao)p_b(n) :return: the cross-cohesion matrix and the list of contributors (authors) in the discussion """ - Assistant
Perfect — you’re at the heart of the GCA framework now 👍. Let’s implement calc_cross_cohesion_matrix based on the definition in the paper (Eq. 15–16 in the PDF).
Here’s a direct implementation: pythonCopyEditdef calc_cross_cohesion_matrix( discussion_data: pd.DataFrame, lag: int = 1 ) -> tuple[list[list[float]], list[str]]: """ Compute the cross-cohesion matrix for a given lag τ.
For two authors a and b, cross-cohesion is defined as: ξ_ab(τ) = ( Σ_{t=τ+1..n} s_ab(t-τ, t) ) / ||p_ab(τ)|| where: s_ab(t,u) = p_a(t) * p_b(u) * cos(d_t, d_u) ||p_ab(τ)|| = Σ_{t=τ+1..n} p_a(t-τ) * p_b(t) Args: discussion_data (pd.DataFrame): must contain 'author', 'text', 'contribution_id'. lag (int): the lag τ to consider (default=1, meaning immediate response). Returns: cross_matrix (list[list[float]]): k x k matrix of cross-cohesion values. authors (list[str]): the list of participants in consistent order. """ authors = discussion_data['author'].unique().tolist() n = discussion_data.shape[0] k = len(authors) # Initialize k x k matrix cross_matrix = [[0.0 for _ in range(k)] for _ in range(k)] for i, a in enumerate(authors): for j, b in enumerate(authors): numerator = 0.0 denominator = 0.0 for t in range(lag, n): # contributions indexed by position c_prev = discussion_data.iloc[t - lag] c_curr = discussion_data.iloc[t] # only consider contributions by a (at t-lag) and b (at t) if c_prev.author == a and c_curr.author == b: # p_a(t-lag) * p_b(t) is guaranteed to be 1 here sim = calc_cos(c_prev.text, c_curr.text) numerator += sim denominator += 1 # since p_a* p_b = 1 in this case # Assign cross-cohesion value if denominator > 0: cross_matrix[i][j] = numerator / denominator else: cross_matrix[i][j] = 0.0 return cross_matrix, authors✅ Example usage: pythonCopyEditmatrix, authors = calc_cross_cohesion_matrix(discussion_data, lag=1) print("Authors:", authors) print("Cross-cohesion matrix (lag=1):") for row in matrix: print(row)
This implementation matches the paper’s definition:
Iterates over contributions at lag τ.
Restricts pairs to those where a wrote at t–τ and b wrote at t.
Computes cosine similarity of texts.
Normalizes by the count of such pairs (denominator).
👉 Do you also want me to extend this so it can compute for multiple lags at once (returning {τ: matrix}), or just keep it per-lag as above?
- User
I'm struggeling with equation 17, what are the dimentions of R(w)?
Here is what I started:
def calc_responsivity(discussion_data: pd.DataFrame, window: int) -> dict[str, float]: """ Responsivity across a time window is defined: R(w) = (cross_cohesion_m(1) + cross_cohesion_m(2) + ... + cross_cohesion_m(w))/w :return: """
- Assistant
Got it — in the paper, equation (17) defines responsivity as the average cross-cohesion over a time window www. Let’s break this down carefully:
- Dimensions of R(w)R(w)R(w)
Each cross-cohesion matrix X(τ)X(\tau)X(τ) at lag τ\tauτ is k × k, where kkk = number of participants.
R(w)R(w)R(w) is simply the average of these matrices for τ=1,2,...,w\tau = 1, 2, ..., wτ=1,2,...,w:
R(w)=1w∑τ=1wX(τ)R(w) = \frac{1}{w} \sum_{\tau=1}^{w} X(\tau)R(w)=w1τ=1∑wX(τ)
So R(w)R(w)R(w) is also a k × k matrix.
- Meaning of elements in R(w)R(w)R(w)
Row iii, column jjj = responsivity of participant iii responding to participant jjj across all lags up to www.
Diagonal entries Rii(w)R_{ii}(w)Rii(w) = internal cohesion (how much someone responds to themselves).
Off-diagonals Rij(w)R_{ij}(w)Rij(w) = responsivity to others.
-
Implementation for your function You already have calc_cross_cohesion_matrix for each τ\tauτ. We now sum over lags and average: pythonCopyEditdef calc_responsivity(discussion_data: pd.DataFrame, window: int) -> tuple[list[list[float]], list[str]]: """ Responsivity across time window w: R(w) = (X(1) + X(2) + ... + X(w)) / w
where each X(τ) is the cross-cohesion matrix at lag τ. Returns: R_w: k x k responsivity matrix authors: list of participants """ authors = discussion_data['author'].unique().tolist() k = len(authors)
Initialize k x k zero matrix
R_w = [[0.0 for _ in range(k)] for _ in range(k)]
Sum over lags
for tau in range(1, window + 1): X_tau, _ = calc_cross_cohesion_matrix(discussion_data, lag=tau) for i in range(k): for j in range(k): R_w[i][j] += X_tau[i][j]
Average over w
for i in range(k): for j in range(k): R_w[i][j] /= window
return R_w, authors
-
Getting participant-level scores
Responsivity of participant aaa = average of row aaa excluding diagonal.
Social impact of participant aaa = average of column aaa excluding diagonal.
Internal cohesion = diagonal element Raa(w)R_{aa}(w)Raa(w).
I can show you helper functions for these if you want.
Do you want me to add those helper functions so you can compute responsivity, social impact, and internal cohesion directly from R(w)R(w)R(w)?
- User
Is this correct?
""" def calculate_overall_responsivity(discussion_data: pd.DataFrame, participant: str, window: int) -> float: """ Calculate overall responsivity score for a specific participant in a discussion. First, we need the responsivity matrix for the discussion. Then we take the mean of the row corresponding to the participant.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Overall responsivity score :param discussion_data: :param participant: :param window: """ authors = discussion_data['author'].unique().tolist() res_matrix = calc_responsivity(discussion_data, window) total_res = 0 participant_idx = authors.index(participant) for i in range(1, len(authors) + 1): if not authors[i] == participant: total_res += res_matrix[participant_idx][authors[i]] return total_res / (len(authors) - 1) if len(authors) > 1 else 0.0"""
- Assistant
You’re really close, but there are two small issues in your code:
calc_responsivity returns a tuple: (matrix, authors), so you need to unpack it correctly.
Inside the loop, authors[i] is a string, but res_matrix is indexed numerically, so you should iterate by index, not author name.
Here’s the fixed version: pythonCopyEditdef calculate_overall_responsivity( discussion_data: pd.DataFrame, participant: str, window: int ) -> float: """ Calculate overall responsivity score for a specific participant in a discussion. Overall responsivity = mean of row values (excluding self) from R(w) matrix. """ res_matrix, authors = calc_responsivity(discussion_data, window) participant_idx = authors.index(participant) k = len(authors)
total_res = 0.0 count = 0 for j in range(k): if j != participant_idx: # exclude self total_res += res_matrix[participant_idx][j] count += 1 return total_res / count if count > 0 else 0.0✅ Why this works
res_matrix[participant_idx][j] = responsivity of participant → author j
Looping over all authors except self
Mean over row values = overall responsivity score
If you like, I can give you similar helper functions for
social impact (column-wise mean)
internal cohesion (diagonal value).
Do you want me to write those too?
- User
And now?
def calculate_overall_responsivity(discussion_data: pd.DataFrame, participant: str, window: int) -> float: """ Calculate overall responsivity score for a specific participant in a discussion. First, we need the responsivity matrix for the discussion. Then we take the mean of the row corresponding to the participant.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Overall responsivity score :param discussion_data: :param participant: :param window: """ res_matrix, authors = calc_responsivity(discussion_data, window) total_res = 0 participant_idx = authors.index(participant) for i in range(len(authors)): if not i == participant_idx: # exclude self total_res += res_matrix[participant_idx][authors[i]] return total_res / (len(authors) - 1) if len(authors) > 1 else 0.0 - Assistant
You're really close again, but the issue is in this line: pythonCopyEdittotal_res += res_matrix[participant_idx][authors[i]]
Here, authors[i] is a string, but res_matrix expects a numeric column index, not a name. You should use i directly instead of authors[i]. Here’s the corrected version: pythonCopyEditdef calculate_overall_responsivity( discussion_data: pd.DataFrame, participant: str, window: int ) -> float: """ Calculate overall responsivity score for a specific participant in a discussion. Overall responsivity = mean of row values (excluding self) from R(w) matrix. """ res_matrix, authors = calc_responsivity(discussion_data, window) participant_idx = authors.index(participant) k = len(authors)
total_res = 0.0 for i in range(k): if i != participant_idx: # exclude self total_res += res_matrix[participant_idx][i] return total_res / (k - 1) if k > 1 else 0.0This now correctly:
Gets the row for the participant
Averages over all other participants
Excludes the diagonal entry (self-responsivity)
Do you want me to give you social impact and internal cohesion in the same style so the three measures work together?
- User
Is this correct?
def calculate_internal_cohesion(discussion_data: pd.DataFrame, participant: str, discussion_id: str, window: int) -> float: """ Calculate internal cohesion score for a specific participant in a discussion. First, calculate the cross-cohesion matrix for the discussion. Return Responsivity for the participant.
Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Internal cohesion score """ # Get the cross-cohesion matrix and authors cross_cohesion_matrix, authors = calc_cross_cohesion_matrix(discussion_data, lag=window) author_idx = authors.index(participant) return cross_cohesion_matrix[author_idx][author_idx] - Assistant
This almost works, but there’s a key difference between what you wrote and how internal cohesion is defined in the paper.
Internal cohesion = average self-responsivity over all lags up to www:
Raa(w)=1w∑τ=1wXaa(τ)R_{aa}(w) = \frac{1}{w} \sum_{\tau=1}^{w} X_{aa}(\tau)Raa(w)=w1τ=1∑wXaa(τ)
What you wrote only uses one lag = window rather than averaging across all lags.
So to match the paper, you should use calc_responsivity, not calc_cross_cohesion_matrix.
Here’s the correct version: pythonCopyEditdef calculate_internal_cohesion(discussion_data: pd.DataFrame, participant: str, window: int) -> float: """ Internal cohesion = diagonal value R_aa(w) from responsivity matrix R(w), averaged across all lags up to w. """ res_matrix, authors = calc_responsivity(discussion_data, window) author_idx = authors.index(participant)
# Diagonal value = self-responsivity return res_matrix[author_idx][author_idx]✅ Why this is correct
calc_responsivity already averages cross-cohesion matrices across lags.
The diagonal entry [author_idx][author_idx] = self-cohesion (internal cohesion).
If you want, I can also show you how to calculate social impact the same way — that’s just the column mean from R(w) instead of the row mean. Do you want me to add that?
- User
Does the following implementation match the paper?
""" def calculate_social_impact(discussion_data: pd.DataFrame, participant: str, window: int) -> float: """ Calculate social impact score for a specific participant in a discussion. By averaging over the colum vector of the responsivity matrix R(w) corresponding to the all other participants.
impact_a(w) = (R_1a(w) + R_2a(w) + ... + R_ka(w)) / (k-1) Args: discussion_data: DataFrame containing the discussion data participant: The participant ID/name discussion_id: The discussion ID Returns: float: Social impact score :param participant: :param discussion_data: :param window: """ # Get the cross-cohesion matrix and authors cross_cohesion_matrix, authors = calc_responsivity(discussion_data, window) # window=1 for social impact author_idx = authors.index(participant) # Calculate social impact as average of column corresponding to this participant total_impact = 0.0 num_participants = len(authors) for i in range(len(authors)): if i != author_idx: # exclude self total_impact += cross_cohesion_matrix[i][author_idx] return total_impact / (num_participants - 1) if num_participants > 0 else 0.0"""
- Assistant
Yes — your implementation now matches the paper’s equation for social impact. Here’s why:
The responsivity matrix R(w)R(w)R(w) is k×kk \times kk×k.
Ria(w)R_{ia}(w)Ria(w) = how much participant iii responds to participant aaa.
So the column average (excluding diagonal) gives:
impacta(w)=1k−1∑i≠aRia(w)\text{impact}a(w) = \frac{1}{k-1} \sum{i \neq a} R_{ia}(w)impacta(w)=k−11i=a∑Ria(w) Which is exactly what you implemented: pythonCopyEdittotal_impact += cross_cohesion_matrix[i][author_idx]
and pythonCopyEditreturn total_impact / (num_participants - 1)
Minor improvement To be consistent with the paper, check for num_participants > 1, because if k=1 there are no "others": pythonCopyEditreturn total_impact / (num_participants - 1) if num_participants > 1 else 0.0
If you want, I can give you a summary table showing: MetricFormula from paperYour implementation usesOverall responsivityRow mean excluding self✅ matchesInternal cohesionDiagonal entry Raa(w)R_{aa}(w)Raa(w)✅ matches (after fix)Social impactColumn mean excluding self✅ matches Do you want me to make this summary for all metrics?
- User
How is this?
""" def span_given(discussion_data: pd.DataFrame, time: int) -> set[list[float]]: """ Given a time t, return the set of embeddings of all contributions made until that time.
Args: time: The time index to filter contributions Returns: set: :param time: :param discussion_data: """ # Filter contributions up to the given time contributions = discussion_data[discussion_data['contribution_id'] < time] # compute the tf-idf embeddings for the contributions vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(contributions['text'].tolist()) if tfidf_matrix.shape[1] > 100: # only if high-dimensional svd = TruncatedSVD(n_components=100, random_state=SEED) tfidf_matrix = svd.fit_transform(tfidf_matrix) # Convert to DataFrame and add embeddings as a column contributions['embedding'] = tfidf_matrix.toarray().tolist() # Extract embeddings (assuming 'embedding' column exists) embeddings = contributions['embedding'].tolist() return set(embeddings) # Return unique embeddings as a set"""
- Assistant
This function will run, but it doesn’t really match the paper’s definition of the “given” space for newness. Here’s why:
In the paper, the given space GtG_tGt at time ttt is the subspace spanned by all previous contributions’ embeddings: Gt=span{d1,d2,...,dt−1}G_t = \text{span}{d_1, d_2, ..., d_{t-1}}Gt=span{d1,d2,...,dt−1} This is not a set of embeddings — it’s the vector space they span.
Using set(embeddings) doesn’t make sense here because:
Embeddings are vectors (lists of floats), not hashable objects for a Python set.
Even if you converted them, a set wouldn’t give you the span or subspace.
The paper uses this given space to compute the projection of the new contribution dtd_tdt into:
Given part = projection onto GtG_tGt
New part = projection onto orthogonal complement Gt⊥G_t^\perpGt⊥
How to fix it What you want to return is the matrix of all previous embeddings, not a set. That way, you can compute projections for newness. Here’s a corrected version: pythonCopyEditdef span_given(discussion_data: pd.DataFrame, time: int): """ Given a time t, return the TF-IDF (or LSA) embedding matrix of all contributions made before time t. This represents the 'given' space G_t in the paper. """ # Filter contributions before time t prev_contribs = discussion_data[discussion_data['contribution_id'] < time]['text'].tolist()
if len(prev_contribs) == 0: return None # no given space yet # TF-IDF + optional LSA vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(prev_contribs) if tfidf_matrix.shape[1] > 100: svd = TruncatedSVD(n_components=100, random_state=42) tfidf_matrix = svd.fit_transform(tfidf_matrix) else: tfidf_matrix = tfidf_matrix.toarray() return tfidf_matrix # matrix of size (#prev_contribs × dim)This way, you can use the matrix to compute projections:
given_part = projection(d_t, G_t)
new_part = projection(d_t, G_t⊥)
If you want, I can show you how to compute the newness measure from the paper using this G_t. Do you want me to add that?
- User
Is this better?
""" def span_given(discussion_data: pd.DataFrame, time: int) -> np.ndarray: """ Given a time t, return the set of embeddings of all contributions made until that time. This represents the 'given' space G_t in the paper. Args: time: The time index to filter contributions
Returns: list[list[float]]: Matrix of embeddings for contributions made until time t :param time: :param discussion_data: """ # Filter contributions up to the given time contributions = discussion_data[discussion_data['contribution_id'] < time]['text'].tolist() # compute the tf-idf embeddings for the contributions vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(contributions) if tfidf_matrix.shape[1] > 100: # only if high-dimensional svd = TruncatedSVD(n_components=100, random_state=SEED) tfidf_matrix = svd.fit_transform(tfidf_matrix) else: tfidf_matrix = tfidf_matrix.toarray() return tfidf_matrix # matrix of size (#prev_contribs × dim)"""
- Assistant
Yes — this is much better and now actually matches the paper’s definition of the given space GtG_tGt. Here’s why it works:
GtG_tGt = span of all contributions before time ttt
You correctly return the matrix of embeddings for those contributions, so you can later compute:
The projection of the current contribution onto GtG_tGt (given part)
The projection onto Gt⊥G_t^\perpGt⊥ (new part)
A few small improvements you might want:
Handle the case where there are no contributions before ttt.
Use a shared vectorizer instead of fitting a new one for every call — otherwise the TF-IDF space changes each time.
Here’s a slightly safer version: pythonCopyEditdef span_given(discussion_data: pd.DataFrame, time: int, vectorizer=None, svd=None) -> np.ndarray: """ Return the TF-IDF (or LSA) embedding matrix for all contributions before time t. This is the 'given' space G_t in the paper.
Args: discussion_data: DataFrame with columns ['contribution_id', 'text'] time: Time index (contribution_id) cutoff vectorizer: Optional pre-fit TfidfVectorizer svd: Optional pre-fit TruncatedSVD model for LSA Returns: np.ndarray: Matrix (#prev_contribs × dim) or None if no contributions """ # Select previous contributions prev_texts = discussion_data[discussion_data['contribution_id'] < time]['text'].tolist() if len(prev_texts) == 0: return None # Fit or transform TF-IDF if vectorizer is None: vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(prev_texts) else: tfidf_matrix = vectorizer.transform(prev_texts) # Optional LSA dimensionality reduction if svd is not None: tfidf_matrix = svd.transform(tfidf_matrix) else: if tfidf_matrix.shape[1] <= 100: tfidf_matrix = tfidf_matrix.toarray() return tfidf_matrixNext, you can use this GtG_tGt to compute newness: n(ct)=∥dt⊥∥∥dt⊥∥+∥dt∥∥n(c_t) = \frac{|d_t^\perp|}{|d_t^\perp| + |d_t^\parallel|}n(ct)=∥dt⊥∥+∥dt∥∥∥dt⊥∥ where dt⊥d_t^\perpdt⊥ is the projection onto Gt⊥G_t^\perpGt⊥. Do you want me to give you the newness computation code next?
- User
def span_given(discussion_data: pd.DataFrame, time: int, vectorizer=None) -> np.ndarray: """ Given a time t, return the set of embeddings of all contributions made until that time. This represents the 'given' space G_t in the paper. Args: time: The time index to filter contributions
Returns: list[list[float]]: Matrix of embeddings for contributions made until time t :param vectorizer: :param time: :param discussion_data: """ # Filter contributions up to the given time contributions = discussion_data[discussion_data['contribution_id'] < time]['text'].tolist() # compute the tf-idf embeddings for the contributions if vectorizer is None: vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(contributions) if tfidf_matrix.shape[1] > 100: # only if high-dimensional svd = TruncatedSVD(n_components=100, random_state=SEED) tfidf_matrix = svd.fit_transform(tfidf_matrix) else: tfidf_matrix = tfidf_matrix.toarray() return tfidf_matrix # matrix of size (#prev_contribs × dim) - Assistant
This version works in the sense that it will produce a matrix of embeddings before time ttt, but there are a few technical details to fix so it really aligns with the paper and avoids subtle issues:
- Correctness vs. the paper
In the paper, the given space GtG_tGt must be in the same embedding space for all times.
Right now you are calling vectorizer.fit_transform at each time ttt, so the TF-IDF basis changes every time — not good for consistency.
Instead, fit the TfidfVectorizer once on the entire discussion, then use transform() for all time windows.
- Edge cases
If there are no contributions before time ttt, you should return None or an empty array — otherwise you'll get an error.
- SVD
Same as TF-IDF: if you want dimensionality reduction (LSA), fit TruncatedSVD once on all contributions, then apply it to each window.
Here’s a better version that handles these issues: pythonCopyEditdef span_given(discussion_data: pd.DataFrame, time: int, vectorizer=None, svd=None) -> np.ndarray: """ Given a time t, return the TF-IDF or LSA embeddings of all contributions made until t
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.