Skip to content
TheChatArchiveThe Chat Archive
GLMGLM-5 / Pony Alpha· unknownHugging Face Frontier Reasoning Traces

Create a KPI glossary with lineage and contracts.

  1. User

    Create a KPI glossary with lineage and contracts.

  2. GLM Assistant

    1. Analyze the Request: * Topic: Create a KPI (Key Performance Indicator) Glossary. * Key Features: Must include "Lineage" and "Contracts". * Target Audience: Likely a data analyst, data engineer, business intelligence professional, or a manager looking to implement data governance or a semantic layer. * Goal: To provide a structured, understandable, and practical example of what such a glossary looks like.

    1. Deconstruct the Concepts:

      • KPI Glossary: A dictionary of business metrics.
      • Lineage: Where the data comes from (Source →\rightarrow Transformation →\rightarrow Final Metric). It shows the flow.
      • Contracts: The rules governing the data (SLAs, data types, freshness, ownership, definition). It guarantees behavior.
    2. Determine the Structure of the Output:

      • I need to provide a framework or a template, not just a definition.
      • It's best to present a few concrete examples (e.g., a Finance metric, a Sales metric, a Marketing metric) to illustrate the concepts.
      • I should define the metadata fields required for the glossary.
    3. Drafting the Content - Step-by-Step:

      • Introduction: Briefly explain why this is important (trust, consistency, debugging).

      • The Structure (The "Schema"): What fields go into this glossary?

        • Basic Info (Name, Description, Owner).
        • The "Contract" (Type, Granularity, Refresh Rate, SLA).
        • The "Lineage" (Source System, SQL/Logic, Dependencies).
        • Status/Health (Warning thresholds, Current status).
      • Example 1: Financial Metric (Monthly Recurring Revenue - MRR)

        • Why MRR? It's a classic SaaS metric with complex logic (add-ons, churn).
        • Contract: Needs to be money (decimal), monthly, ready by the 2nd day of the month.
        • Lineage: Stripe →\rightarrow Raw Table →\rightarrow Staging →\rightarrow Mart.
      • Example 2: Sales Metric (Win Rate)

        • Why Win Rate? Involves counting (won deals) vs. total deals.
        • Contract: Percentage, updated daily.
        • Lineage: Salesforce →\rightarrow CRM table.
      • Example 3: Marketing Metric (Customer Acquisition Cost - CAC)

        • Why CAC? Combines marketing spend (ads) and sales data.
        • Lineage: Needs to show a join between two systems (Marketing platform + CRM). This highlights lineage complexity.
    4. Refining the "Lineage" and "Contract" sections:

      • Lineage Visualization: Since I can't draw a real graph, I'll use text-based arrows or a "Path" structure.

        • Example: Source: Salesforce →\rightarrow Transformation: Fivetran →\rightarrow DW: Snowflake (stg_sales) →\rightarrow Logic: dbt model (fct_win_rate).
      • Contracts (Data Contracts): This is about expectations.

        • Expectation: "This number will never be negative." "It will be updated by 8 AM." "It must match the finance report."
    5. Structuring the Output for the User:

      • Use Markdown tables for readability.
      • Use code blocks for SQL/Logic examples (adds realism).
      • Add a section on "How to Implement" (Tools).
    6. Self-Correction/Review during drafting:

      • Initial thought: Just list definitions.
      • Correction: The prompt asks for lineage and contracts. I need to be specific about these. A standard business glossary isn't enough. It needs to look like a technical specification.
      • Emphasis: Make sure the "Contract" implies a two-way agreement (Data Team ↔\leftrightarrow Business Team).
    7. Final Polish:

      • Check for clarity.
      • Ensure the distinction between "Business Definition" and "Technical Lineage" is clear.
      • Add a YAML example, as that's how modern Data Catalogs (like dbt or DataHub) often represent this.
    8. (Self-Correction during output generation): I will structure this as a "Modern Data Stack" KPI Glossary, assuming the user wants a technical/business hybrid view.

      • Drafting the YAML block: This is crucial for "Contracts".
      • Drafting the Lineage section: Needs to visualize the flow.
    9. Final Output Generation: (This matches the provided good response). To create a robust KPI glossary that includes lineage and contracts, you must move beyond a simple dictionary of terms. You need a Metadata Structure that acts as a bridge between Business Intelligence (definitions) and Data Engineering (implementation).

    Here is a structural template and three concrete examples of how to document KPIs using Lineage (where data comes from) and Contracts (the rules data must follow).


    Part 1: The Metadata Framework

    When documenting a KPI, you should populate the following three pillars:

    1. The Business Definition (The "What")

    • Metric Name: Friendly name.
    • Description: Plain English explanation.
    • Owner: Who is accountable for this number (e.g., CFO, CMO).
    • Granularity: Level of detail (e.g., Per User, Per Day, Monthly).

    2. The Data Contract (The "Rules")

    A Data Contract is a guarantee that the data feeding the KPI will not change format unexpectedly. It defines the "Schema" and "SLA."

    • Input Schema: Expected column names and data types from the source.
    • Freshness SLA: How stale the data is allowed to be (e.g., "Updated by 8 AM UTC").
    • Quality Checks: Hard constraints (e.g., Total Sales cannot be negative).
    • Consumer: Who/What relies on this (e.g., Looker Dashboard, CEO Report).

    3. The Lineage (The "How")

    This traces the journey from the raw source to the final metric.

    • Source System: Where the raw data lives (e.g., Salesforce, PostgreSQL).
    • Transformation Layer: The logic applied (SQL/Python scripts).
    • Serving Layer: Where the KPI is visualized.

    Part 2: The Glossary Examples

    Here are three examples of how to document common KPIs using this framework.

    KPI 1: Monthly Recurring Revenue (MRR)

    Financial Health

    AttributeDetail
    DefinitionThe predictable revenue generated by your business from all active subscriptions in a specific month.
    OwnerVP of Finance
    GranularityPer Month, segmented by Plan Type

    📜 The Data Contract

    • Input Columns:
      • user_id (UUID)
      • subscription_start_date (Date)
      • monthly_price (Decimal(10,2))
    • Constraint Rules:
      • monthly_price must be ≥0\ge 0.
      • Exclude records where status = 'churned'.
    • SLA: Data must be aggregated and available in the fct_billing table by the 1st of the month at 06:00 AM.

    🔀 The Lineage

    1. Source: Stripe API / Salesforce CRM
    2. Staging: stg_payments (Raw extract)
    3. Transform: int_mrr_calculation (SQL: Sum monthly price where active)
    4. Mart: fct_finance_overview
    5. Consumer: Executive Finance Dashboard

    KPI 2: Net Promoter Score (NPS)

    Customer Sentiment

    AttributeDetail
    DefinitionA measure of customer loyalty ranging from -100 to +100, calculated based on a single question: "How likely are you to recommend us?"
    OwnerHead of Customer Success
    GranularityWeekly Aggregate

    📜 The Data Contract

    • Input Columns:
      • response_id (Int)
      • score (Int: Range 0-10)
      • response_timestamp (Timestamp)
    • Constraint Rules:
      • score must be an integer between 0 and 10.
      • If score is null, exclude from calculation.
    • Calculation Logic: %Promoters(9−10)−%Detractors(0−6)\% \text{Promoters} (9-10) - \% \text{Detractors} (0-6).

    🔀 The Lineage

    1. Source: Delighted API / Typeform
    2. Staging: stg_nps_responses
    3. Transform: nps_score_calculation (Python Pandas/SQL: Bucket scores into groups)
    4. Mart: kpi_customer_health
    5. Consumer: CS Team Slide Deck

    KPI 3: Conversion Rate (Landing Page)

    Marketing Performance

    AttributeDetail
    DefinitionThe percentage of visitors to a specific landing page who completed a desired action (clicked "Buy").
    OwnerGrowth Marketing Manager
    GranularityDaily, by Traffic Source

    📜 The Data Contract

    • Input Columns:
      • session_id (String)
      • page_url (String)
      • event_type (String: 'view', 'click')
      • utm_source (String)
    • Constraint Rules:
      • Denominator: Unique session_id with event_type='view'.
      • Numerator: Unique session_id with event_type='click'.
      • Filtering: Exclude internal IP addresses.
    • SLA: Real-time streaming (max latency 5 minutes).

    🔀 The Lineage

    1. Source: Google Analytics 4 / Snowplow
    2. Staging: raw_web_events
    3. Transform: fct_conversion_funnel (SQL: Count distinct sessions)
    4. Mart: marketing_performance_model
    5. Consumer: Real-time CMO Dashboard

    Part 3: Implementation Strategy (How to Build It)

    To make this usable, do not store it in a static PDF or Excel sheet. You should implement this using a Data Catalog or your Transformation Layer (like dbt).

    Option A: Using YAML (Code-based)

    If you use a tool like dbt (data build tool), you can define the contract and lineage directly in your code. This ensures that if the data breaks the contract, the KPI pipeline fails automatically.

    # models/marts/finance/fct_mrr.yml
    version: 2
    
    models:
      - name: fct_mrr
        description: "Financial KPIs regarding Monthly Recurring Revenue."
        meta:
          owner: "VP of Finance"
          
        columns:
          - name: mrr_amount
            description: "The calculated MRR value."
            tests:
              - not_null
              - dbt_utils.accepted_range:
                  min_value: 0
                  inclusive: true
                  
          - name: date_month
            description: "The month of the record."
            tests:
              - not_null
    

    Option B: Using a Data Catalog (UI-based)

    Tools like DataHub, Atlan, or Monte Carlo allow you to visualize the Lineage graph.

    1. Ingest your data sources.
    2. Write the "Definition" in the description field.
    3. Add "Tests" (Contracts) to the specific columns.
    4. The tool automatically generates the visual lineage graph (Lineage →\rightarrow Transform →\rightarrow Consumer).

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.