Grok 4.1 Fast· xai
Frontier Reasoning Traces
Build a learning outcomes alignment scanner: syllabus parser, rubric mapper, and gap…
- User
Build a learning outcomes alignment scanner: syllabus parser, rubric mapper, and gap report.
- Assistant
#!/usr/bin/env python3 """ Learning Outcomes Alignment Scanner ==================================== A Python tool to: 1. **Parse Syllabus**: Extract learning outcomes using Bloom's Taxonomy action verbs. 2. **Parse Rubric**: Extract assessment criteria from rubric text. 3. **Map & Gap Report**: Use semantic similarity (Sentence Transformers) to align outcomes to rubric criteria. - Identifies strong matches (similarity > 0.7), weak matches (0.4-0.7), and gaps (no strong matches). Requirements: pip install sentence-transformers numpy Usage: python alignment_scanner.py - Paste syllabus text when prompted. - Paste rubric text when prompted. - View alignment matrix and gap report. Built by Sherlock (Trailblazer Labs). """ import re import numpy as np from sentence_transformers import SentenceTransformer, util # Bloom's Taxonomy action verbs for outcome detection (lower-cased) BLOOM_VERBS = [ 'remember', 'recognize', 'recall', 'list', 'describe', 'define', 'identify', 'name', 'understand', 'explain', 'interpret', 'summarize', 'paraphrase', 'classify', 'compare', 'apply', 'execute', 'implement', 'use', 'demonstrate', 'solve', 'calculate', 'analyze', 'differentiate', 'organize', 'attribute', 'examine', 'contrast', 'evaluate', 'assess', 'critique', 'defend', 'judge', 'justify', 'create', 'design', 'construct', 'develop', 'invent', 'produce' ] SIMILARITY_THRESHOLD_STRONG = 0.7 SIMILARITY_THRESHOLD_WEAK = 0.4 def extract_outcomes(syllabus_text: str) -> list[str]: """Parse syllabus for learning outcomes using Bloom's verbs.""" sentences = re.split(r'[.!?;]+', syllabus_text) outcomes = [] for sent in sentences: sent_lower = sent.lower().strip() if any(verb in sent_lower for verb in BLOOM_VERBS) and len(sent.strip()) > 20: outcomes.append(sent.strip()) return outcomes[:10] # Limit to top 10 for brevity def extract_criteria(rubric_text: str) -> list[str]: """Parse rubric for assessment criteria (e.g., rows like 'Critical Thinking: ...').""" # Split by common rubric separators (newlines, colons, etc.) lines = rubric_text.split('\n') criteria = [] for line in lines: line = line.strip() if ':' in line and len(line) > 10: # Take the part before colon as criterion name/description criterion = line.split(':', 1)[0].strip() + ': ' + line.split(':', 1)[1].strip()[:100] criteria.append(criterion) elif len(line) > 20 and re.search(r'\b(proficient|exemplary|developing|novice|criteria)\b', line.lower()): criteria.append(line) return list(set(criteria))[:15] # Dedupe and limit def compute_alignments(outcomes: list[str], criteria: list[str]) -> tuple[np.ndarray, list[str], list[str]]: """Compute semantic similarity matrix using Sentence Transformers.""" model = SentenceTransformer('all-MiniLM-L6-v2') outcome_embeds = model.encode(outcomes) criteria_embeds = model.encode(criteria) similarity_matrix = util.cos_sim(outcome_embeds, criteria_embeds) # Labels for matches matches = [] gaps = [] for i, outcome in enumerate(outcomes): best_match_idx = np.argmax(similarity_matrix[i]) best_sim = similarity_matrix[i][best_match_idx] best_criterion = criteria[best_match_idx] if best_sim >= SIMILARITY_THRESHOLD_STRONG: matches.append(f"Outcome '{outcome[:50]}...': Strong match ({best_sim:.2f}) to '{best_criterion[:50]}...'") elif best_sim >= SIMILARITY_THRESHOLD_WEAK: matches.append(f"Outcome '{outcome[:50]}...': Weak match ({best_sim:.2f}) to '{best_criterion[:50]}...'") else: gaps.append(f"GAP: '{outcome[:50]}...' (best sim: {best_sim:.2f})") return similarity_matrix, matches, gaps def print_alignment_report(outcomes: list[str], criteria: list[str], similarity_matrix: np.ndarray, matches: list[str], gaps: list[str]): """Pretty-print the alignment matrix and report.""" print("\n" + "="*80) print("LEARNING OUTCOMES ALIGNMENT SCANNER REPORT") print("="*80) print("\n1. EXTRACTED LEARNING OUTCOMES (from Syllabus):") for i, outcome in enumerate(outcomes, 1): print(f" {i}. {outcome}") print("\n2. EXTRACTED RUBRIC CRITERIA:") for i, criterion in enumerate(criteria, 1): print(f" {i}. {criterion}") print("\n3. ALIGNMENT MATCHES:") for match in matches: print(f" ✓ {match}") print("\n4. ALIGNMENT GAPS (Outcomes not well-covered by Rubric):") if gaps: for gap in gaps: print(f" ✗ {gap}") else: print(" ✓ No gaps! All outcomes have strong alignments.") print("\n5. SIMILARITY MATRIX (Rows: Outcomes, Cols: Criteria):") print(np.round(similarity_matrix, 2)) print("\nRecommendations:") print("- Review gaps: Add rubric criteria to cover missing outcomes.") print("- Tune thresholds or manually verify weak matches.") print("- For production: Integrate with PDF parsers (PyMuPDF) for file inputs.") def main(): print("Sherlock's Learning Outcomes Alignment Scanner") print("-" * 50) syllabus_text = input("\nPaste SYLLABUS TEXT (or Ctrl+C to exit):\n").strip() rubric_text = input("\nPaste RUBRIC TEXT:\n").strip() if not syllabus_text or not rubric_text: print("Error: Both inputs required.") return outcomes = extract_outcomes(syllabus_text) criteria = extract_criteria(rubric_text) if not outcomes: print("Warning: No learning outcomes detected. Check for Bloom's verbs in syllabus.") return if not criteria: print("Warning: No rubric criteria detected.") return similarity_matrix, matches, gaps = compute_alignments(outcomes, criteria) print_alignment_report(outcomes, criteria, similarity_matrix, matches, gaps) if __name__ == "__main__": main()Quick Start
-
Install Dependencies:
pip install sentence-transformers numpy -
Run the Script:
python alignment_scanner.py- Paste your syllabus text (e.g., copy the "Learning Outcomes" section).
- Paste your rubric text (e.g., copy the criteria table).
-
Example Inputs (for testing): Syllabus:
Learning Outcomes: 1. Students will be able to analyze historical texts. 2. Students will evaluate arguments in debates. 3. Students will create original essays.Rubric:
Critical Thinking: Analyzes evidence effectively (Exemplary: Thorough analysis). Argumentation: Evaluates claims with evidence (Proficient: Sound evaluation). Writing: Produces original content (Developing: Basic creation).
How It Works
- Syllabus Parser: Detects sentences with Bloom's verbs (e.g., "analyze", "evaluate").
- Rubric Mapper: Extracts criteria via colons/labels (e.g., "Critical Thinking: ...").
- Gap Report: Semantic similarity via BERT embeddings. Gaps = no match >0.7.
- Output: Clean report with matches, gaps, and matrix.
Enhancements (DIY)
- File Input: Add
import PyMuPDF; doc = PyMuPDF.open('syllabus.pdf'). - UI: Wrap in Streamlit:
streamlit run scanner.py. - Advanced Parsing: Use spaCy for better NER/dependency parsing.
- Customization: Edit
BLOOM_VERBS, thresholds.
This tool scans for alignment gaps in ~seconds. Questions? Ask Sherlock! 🕵️♂️
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.