Skip to content
TheChatArchiveThe Chat Archive
Grok 4.1 Fast· xaiHugging Face Frontier Reasoning Traces

Build a learning outcomes alignment scanner: syllabus parser, rubric mapper, and gap…

  1. User

    Build a learning outcomes alignment scanner: syllabus parser, rubric mapper, and gap report.

  2. Assistant
    #!/usr/bin/env python3
    """
    Learning Outcomes Alignment Scanner
    ====================================
    
    A Python tool to:
    1. **Parse Syllabus**: Extract learning outcomes using Bloom's Taxonomy action verbs.
    2. **Parse Rubric**: Extract assessment criteria from rubric text.
    3. **Map & Gap Report**: Use semantic similarity (Sentence Transformers) to align outcomes to rubric criteria.
       - Identifies strong matches (similarity > 0.7), weak matches (0.4-0.7), and gaps (no strong matches).
    
    Requirements:
    pip install sentence-transformers numpy
    
    Usage:
    python alignment_scanner.py
    - Paste syllabus text when prompted.
    - Paste rubric text when prompted.
    - View alignment matrix and gap report.
    
    Built by Sherlock (Trailblazer Labs).
    """
    
    import re
    import numpy as np
    from sentence_transformers import SentenceTransformer, util
    
    # Bloom's Taxonomy action verbs for outcome detection (lower-cased)
    BLOOM_VERBS = [
        'remember', 'recognize', 'recall', 'list', 'describe', 'define', 'identify', 'name',
        'understand', 'explain', 'interpret', 'summarize', 'paraphrase', 'classify', 'compare',
        'apply', 'execute', 'implement', 'use', 'demonstrate', 'solve', 'calculate',
        'analyze', 'differentiate', 'organize', 'attribute', 'examine', 'contrast',
        'evaluate', 'assess', 'critique', 'defend', 'judge', 'justify',
        'create', 'design', 'construct', 'develop', 'invent', 'produce'
    ]
    
    SIMILARITY_THRESHOLD_STRONG = 0.7
    SIMILARITY_THRESHOLD_WEAK = 0.4
    
    def extract_outcomes(syllabus_text: str) -> list[str]:
        """Parse syllabus for learning outcomes using Bloom's verbs."""
        sentences = re.split(r'[.!?;]+', syllabus_text)
        outcomes = []
        for sent in sentences:
            sent_lower = sent.lower().strip()
            if any(verb in sent_lower for verb in BLOOM_VERBS) and len(sent.strip()) > 20:
                outcomes.append(sent.strip())
        return outcomes[:10]  # Limit to top 10 for brevity
    
    def extract_criteria(rubric_text: str) -> list[str]:
        """Parse rubric for assessment criteria (e.g., rows like 'Critical Thinking: ...')."""
        # Split by common rubric separators (newlines, colons, etc.)
        lines = rubric_text.split('\n')
        criteria = []
        for line in lines:
            line = line.strip()
            if ':' in line and len(line) > 10:
                # Take the part before colon as criterion name/description
                criterion = line.split(':', 1)[0].strip() + ': ' + line.split(':', 1)[1].strip()[:100]
                criteria.append(criterion)
            elif len(line) > 20 and re.search(r'\b(proficient|exemplary|developing|novice|criteria)\b', line.lower()):
                criteria.append(line)
        return list(set(criteria))[:15]  # Dedupe and limit
    
    def compute_alignments(outcomes: list[str], criteria: list[str]) -> tuple[np.ndarray, list[str], list[str]]:
        """Compute semantic similarity matrix using Sentence Transformers."""
        model = SentenceTransformer('all-MiniLM-L6-v2')
        outcome_embeds = model.encode(outcomes)
        criteria_embeds = model.encode(criteria)
        
        similarity_matrix = util.cos_sim(outcome_embeds, criteria_embeds)
        
        # Labels for matches
        matches = []
        gaps = []
        for i, outcome in enumerate(outcomes):
            best_match_idx = np.argmax(similarity_matrix[i])
            best_sim = similarity_matrix[i][best_match_idx]
            best_criterion = criteria[best_match_idx]
            
            if best_sim >= SIMILARITY_THRESHOLD_STRONG:
                matches.append(f"Outcome '{outcome[:50]}...': Strong match ({best_sim:.2f}) to '{best_criterion[:50]}...'")
            elif best_sim >= SIMILARITY_THRESHOLD_WEAK:
                matches.append(f"Outcome '{outcome[:50]}...': Weak match ({best_sim:.2f}) to '{best_criterion[:50]}...'")
            else:
                gaps.append(f"GAP: '{outcome[:50]}...' (best sim: {best_sim:.2f})")
        
        return similarity_matrix, matches, gaps
    
    def print_alignment_report(outcomes: list[str], criteria: list[str], similarity_matrix: np.ndarray, matches: list[str], gaps: list[str]):
        """Pretty-print the alignment matrix and report."""
        print("\n" + "="*80)
        print("LEARNING OUTCOMES ALIGNMENT SCANNER REPORT")
        print("="*80)
        
        print("\n1. EXTRACTED LEARNING OUTCOMES (from Syllabus):")
        for i, outcome in enumerate(outcomes, 1):
            print(f"  {i}. {outcome}")
        
        print("\n2. EXTRACTED RUBRIC CRITERIA:")
        for i, criterion in enumerate(criteria, 1):
            print(f"  {i}. {criterion}")
        
        print("\n3. ALIGNMENT MATCHES:")
        for match in matches:
            print(f"  ✓ {match}")
        
        print("\n4. ALIGNMENT GAPS (Outcomes not well-covered by Rubric):")
        if gaps:
            for gap in gaps:
                print(f"  ✗ {gap}")
        else:
            print("  ✓ No gaps! All outcomes have strong alignments.")
        
        print("\n5. SIMILARITY MATRIX (Rows: Outcomes, Cols: Criteria):")
        print(np.round(similarity_matrix, 2))
        
        print("\nRecommendations:")
        print("- Review gaps: Add rubric criteria to cover missing outcomes.")
        print("- Tune thresholds or manually verify weak matches.")
        print("- For production: Integrate with PDF parsers (PyMuPDF) for file inputs.")
    
    def main():
        print("Sherlock's Learning Outcomes Alignment Scanner")
        print("-" * 50)
        
        syllabus_text = input("\nPaste SYLLABUS TEXT (or Ctrl+C to exit):\n").strip()
        rubric_text = input("\nPaste RUBRIC TEXT:\n").strip()
        
        if not syllabus_text or not rubric_text:
            print("Error: Both inputs required.")
            return
        
        outcomes = extract_outcomes(syllabus_text)
        criteria = extract_criteria(rubric_text)
        
        if not outcomes:
            print("Warning: No learning outcomes detected. Check for Bloom's verbs in syllabus.")
            return
        if not criteria:
            print("Warning: No rubric criteria detected.")
            return
        
        similarity_matrix, matches, gaps = compute_alignments(outcomes, criteria)
        
        print_alignment_report(outcomes, criteria, similarity_matrix, matches, gaps)
    
    if __name__ == "__main__":
        main()
    

    Quick Start

    1. Install Dependencies:

      pip install sentence-transformers numpy
      
    2. Run the Script:

      python alignment_scanner.py
      
      • Paste your syllabus text (e.g., copy the "Learning Outcomes" section).
      • Paste your rubric text (e.g., copy the criteria table).
    3. Example Inputs (for testing): Syllabus:

      Learning Outcomes:
      1. Students will be able to analyze historical texts.
      2. Students will evaluate arguments in debates.
      3. Students will create original essays.
      

      Rubric:

      Critical Thinking: Analyzes evidence effectively (Exemplary: Thorough analysis).
      Argumentation: Evaluates claims with evidence (Proficient: Sound evaluation).
      Writing: Produces original content (Developing: Basic creation).
      

    How It Works

    • Syllabus Parser: Detects sentences with Bloom's verbs (e.g., "analyze", "evaluate").
    • Rubric Mapper: Extracts criteria via colons/labels (e.g., "Critical Thinking: ...").
    • Gap Report: Semantic similarity via BERT embeddings. Gaps = no match >0.7.
    • Output: Clean report with matches, gaps, and matrix.

    Enhancements (DIY)

    • File Input: Add import PyMuPDF; doc = PyMuPDF.open('syllabus.pdf').
    • UI: Wrap in Streamlit: streamlit run scanner.py.
    • Advanced Parsing: Use spaCy for better NER/dependency parsing.
    • Customization: Edit BLOOM_VERBS, thresholds.

    This tool scans for alignment gaps in ~seconds. Questions? Ask Sherlock! 🕵️‍♂️

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.