Create a data quality SLA: contracts, monitors, and remediation.
- User
Create a data quality SLA: contracts, monitors, and remediation.
Assistant
The user wants a Data Quality Service Level Agreement (SLA). The request specifies three key components:
- Contracts: The agreed-upon standards (dimensions, thresholds, ownership).
- Monitors: The technical implementation of checking those standards (how we measure).
- Remediation: The process for fixing issues when they arise (incident response).
Target Audience/Context: Likely a Data Engineering or Data Governance team looking to establish a formal framework to ensure data reliability for downstream consumers (analysts, data scientists, applications).
Structure of the Response: I should create a comprehensive template or framework that can be adapted. A good structure would be:
- Introduction/Philosophy: Briefly explain why this matters.
- Part 1: Data Quality Contracts (The "What"):
- Dimensions (Accuracy, Completeness, Timeliness, etc.).
- Severity Levels (Critical, High, Medium, Low).
- The actual contract table (SLA definitions).
- Part 2: Monitoring Strategy (The "How"):
- Tools (dbt tests, Great Expectations, Monte Carlo, etc.).
- Frequency (Real-time vs. Batch).
- Alerting channels.
- Part 3: Remediation Process (The "Action"):
- Incident Triage workflow.
- Root Cause Analysis (RCA).
- Roles & Responsibilities (RACI).
- Example Scenario: A concrete example helps ground the abstract concepts.
Drafting Content - Section by Section:
-
Introduction: Keep it professional. "This document defines the agreement between Data Producers and Data Consumers."
-
Part 1: Contracts:
- I need to define specific metrics. "Freshness" is a big one. "Nullness/Completeness." "Uniqueness."
- I need to define tiers of data (Gold/Silver/Bronze) because not all data deserves the same SLA.
- Drafting the SLA Matrix:
- Tier 1 (Critical): Uptime 99.9%, Freshness < 15 mins, Accuracy 100%.
- Tier 2 (Standard): Freshness < 24 hrs, Accuracy 99.5%.
-
Part 2: Monitors:
- How do we implement this? We need a concept of "inhibitors" (stop the job) vs "warnings" (send an alert).
- Tools: Mention SQL checks, schema evolution checks, volume checks (row count).
-
Part 3: Remediation:
- This needs a workflow.
-
- Detection -> 2. Notification -> 3. Assessment -> 4. Resolution -> 5. Post-Mortem.
- Crucial part: What happens if SLA is breached? (e.g., Disable dashboard, page on-call engineer).
-
Refining the Approach:
- Make it actionable. Instead of just theory, provide a table format.
- Include a "Data Product Tiering" section, as SLAs are useless if applied equally to a scratch table and a financial report.
-
Self-Correction during drafting:
- Initial thought: Just list metrics like "Accuracy."
- Correction: "Accuracy" is hard to measure without a ground truth. Better to use "Validity" (regex match) or "Referential Integrity" (foreign keys). I will include these more technical terms.
-
Formatting: Use Markdown tables for the SLA definitions. Use bolding for key terms.
Final Polish: Ensure the distinction between "Contracts" (the promise), "Monitors" (the check), and "Remediation" (the fix) is clear, as requested.
Let's write the response following this structure. I'll create a "Data Quality SLA Framework" template.
- Title: Data Quality SLA Framework.
- Preamble: Who is this for?
- Section 1: Data Contracts: Define Tiers (Tier 0 to Tier 2). Define Metrics (Freshness, Completeness, Consistency).
- Section 2: Monitoring & Observability: Implementation details (tests, anomaly detection).
- Section 3: Incident Remediation: Escalation matrix, SLA breach consequences.
(Self-Correction): Make sure to mention who is responsible. The "Data Producer" vs "Data Consumer" dynamic is central to an SLA.
Okay, proceeding to generate the output based on these thoughts. A Data Quality Service Level Agreement (SLA) is a formal commitment between Data Engineering (the Provider) and Business/Analytics Teams (the Consumer). It defines what constitutes "good data," how it is measured, and what happens when the data fails.
Below is a comprehensive framework for building a Data Quality SLA.
1. Data Contracts (The "What")
The contract defines the rules of the road. Before monitoring, you must define the Data Product Tiers and the Dimensions of Quality.
A. Data Tiering
Not all data is equal. You cannot apply the same rigor to a raw scraping dataset as you do to financial payroll data.
Tier Description Max Allowable Downtime Freshness Requirement Accuracy Target Tier 1 (Mission Critical) Revenue, Security, Financial Reporting < 15 mins Real-time / Near Real-time 99.99% Tier 2 (Business Critical) Dashboards, KPIs, Daily Ops < 4 hours Daily (by 6 AM) 99.5% Tier 3 (Operational) Logs, Internal Testing, Sandbox < 24 hours Weekly / Ad-hoc 95% B. Quality Dimensions (The Metrics)
The contract must specify which metrics are being measured.
- Completeness: Are there nulls? (e.g.,
user_idcannot be null). - Uniqueness: Are there duplicates? (e.g., Transaction ID must be unique).
- Timeliness: Is the data fresh? (e.g.,
updated_atis within the last 24 hours). - Validity: Does it follow a format? (e.g., Email matches regex,
ageis positive). - Integrity: Do foreign keys match? (e.g., Every
order_idinpaymentsexists inorders).
2. Monitors (The "How")
This is the technical implementation of the contract. This usually lives within your Data Quality (DQ) tool (e.g., Monte Carlo, dbt tests, Great Expectations, Soda).
A. Monitoring Types
- Schema Change Monitoring:
- Rule: Alert immediately if a column is removed or data type changes without a PR merge.
- Volume/Anomaly Monitoring:
- Rule: Alert if row count drops by >20% compared to the same time last week (potential data loss).
- Field Health:
- Rule: Alert if the "Null %" of a critical field exceeds 1%.
B. The Monitoring Matrix
Map your monitors to your Tiers to determine severity.
Monitor Type Tier 1 Response Tier 2 Response Tier 3 Response Freshness Lag (Data late by >30m) Page On-Call (Immediate) Slack Alert (#data-alerts) Email Alert (Daily Digest) Schema Change (Breaking change) Halt Pipeline + Page On-Call Log Warning + Slack Alert Log Warning Null % Spike (Nulls > 5%) Slack Alert (High Priority) Log Warning Ignore (Monitor Only)
3. Remediation (The "Action")
When a monitor triggers, what happens? Remediation involves three phases: Acknowledgment, Triage, and Resolution.
A. Incident Severity Levels
Define these clearly so stakeholders know how urgent the issue is.
- SEV-1 (Critical): Data is wrong, and it is directly affecting revenue or customer-facing features.
- Example: Customer balances are displaying as $0.
- Action: All hands on deck. Page the Data Lead.
- SEV-2 (High): Key dashboards are stale or broken, but internal operations can continue.
- Example: The "Daily Sales" dashboard didn't refresh.
- Action: Investigate within 1 hour. Notify stakeholders of delay.
- SEV-3 (Low): Data quality issue in a non-critical field or historical backfill.
- Action: Ticket created for the next sprint.
B. Remediation Workflow (The SOP)
- Detection: Automated monitor triggers alert.
- Triage (The "Stop the Bleeding" phase):
- Is it the data source or the pipeline? Check the raw source.
- Action: If Tier 1 data is corrupt, quarantine the data. Do not let bad data flow into BI tools. Fail the pipeline job.
- Communication:
- Template: "Data Issue Detected: [Table Name]. Impact: [Dashboards]. ETA for Fix: [Time]. Status: Investigating."
- Resolution:
- Fix the code or wait for the upstream provider to fix the source.
- Perform a backfill (re-run historical data) if necessary.
- Post-Mortem (The "Never Again" phase):
- For any SEV-1, a document is required within 48 hours answering:
- Why did the monitor catch it (or not)?
- How do we prevent this specific edge case next time?
- For any SEV-1, a document is required within 48 hours answering:
Summary Checklist
If you are implementing this today, start here:
- Identify your top 3 Tier 1 tables. (Don't try to SLA everything at once).
- Define 3 "Golden Rules" for those tables (e.g., Primary Key unique, Freshness < 1hr, No nulls).
- Set up an Alert that pings a dedicated Slack channel when those rules break.
- Write a "Who to Call" list for each Tier 1 dataset.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.