computationalanalysissocial_roles.pdfPDFI implemented Python functions for the 6 GCA…
- User
computational_analysis_social_roles.pdfPDFI implemented Python functions for the 6 GCA aspects. Write a prompt for LLM as a judge to evaluate "Overall Responsivity".
The LLM will review a discussion, and for a given participant, its task is to determine the participant's overall responsivity on a given scale.
Include in the prompt:
- A role "You are a psychologist, specializing in..."
- Clear definition of what Responsivity aims to capture
- The min and max values
- Clear instructions
- Other prompting strategies that can be useful.
- Assistant
Here’s a well-structured prompt for the LLM to judge Overall Responsivity in a discussion, based on the Group Communication Analysis (GCA) framework in your document:
Role: You are a psychologist specializing in collaborative learning, group interaction dynamics, and computational discourse analysis. Your task is to evaluate the Overall Responsivity of a given participant within a group discussion. Definition of Overall Responsivity: Overall Responsivity measures the extent to which a participant's contributions are directly connected to, and build upon, the contributions of other group members. It reflects their ability to acknowledge, incorporate, and respond meaningfully to others’ ideas, rather than contributing in isolation. High responsivity indicates active engagement in the shared construction of knowledge; low responsivity suggests minimal acknowledgment or integration of peers’ perspectives. Scale: Rate the participant’s Overall Responsivity on a scale from 1 to 5:
1 = Very Low: Rarely or never responds meaningfully to others; contributions appear disconnected or isolated.
2 = Low: Occasionally responds to others, but most contributions show little integration with prior discussion.
3 = Moderate: Sometimes responds meaningfully and integrates peers’ ideas but with inconsistency.
4 = High: Frequently builds on others’ contributions, maintaining good coherence with prior discussion.
5 = Very High: Consistently engages with and elaborates on others’ ideas, demonstrating strong integration and shared understanding.
Instructions:
Carefully read the entire discussion transcript.
Focus on the target participant's contributions in relation to prior messages from other participants.
Consider both the frequency and quality of responses to peers:
Are contributions directly connected to previous ideas?
Do they elaborate, clarify, or build upon others’ points?
Do they help sustain a coherent and shared discussion?
Ignore off-topic chatter unless it reflects interactional responsiveness.
Assign a single Overall Responsivity score (1–5) for the participant.
Provide a brief explanation citing examples to justify the score.
Additional Guidance:
Use a holistic judgment, considering both linguistic cues (e.g., references, agreement, elaboration) and discourse flow.
If uncertain between two scores, choose the lower one unless clear evidence supports the higher.
Focus on interaction quality, not content accuracy or social tone.
If you like, I can also create similar judging prompts for Participation, Internal Cohesion, Social Impact, Newness, and Communication Density to keep the format consistent. Would you like me to do that?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.