Draft a market data governance program: vendor contracts, entitlements, lineage, quality…
- User
Draft a market data governance program: vendor contracts, entitlements, lineage, quality SLAs, and audits.
Assistant
The user wants a draft of a "Market Data Governance Program." The request specifically lists five key components to include:
- Vendor Contracts
- Entitlements (Access Control)
- Lineage (Data Flow/Provenance)
- Quality SLAs (Service Level Agreements)
- Audits
Target Audience/Context: This is likely for a financial institution (bank, hedge fund, asset manager) or a large corporation dealing with significant external data streams (market data like stock prices, FX rates, economic indicators). The tone needs to be professional, structured, and actionable—something a Chief Data Officer (CDO) or Head of Market Data could use as a starting framework.
Structure of the Output: I need to organize this as a formal program document.
- Executive Summary/Introduction: Purpose and scope.
- Policy Framework: The governing principles.
- Component 1: Vendor Contract Management: Focus on legal terms, usage rights, and costs.
- Component 2: Entitlements & Access Management: Focus on "least privilege," inventory, and license types (user-based, usage-based).
- Component 3: Data Lineage & Cataloging: Focus on "source of truth," flow mapping, and technical metadata.
- Component 4: Data Quality & SLAs: Focus on accuracy, timeliness, and completeness metrics.
- Component 5: Audit & Compliance: Focus on internal checks and vendor audits.
- Roles & Responsibilities: Who does what.
- Implementation Roadmap: High-level steps to get this running.
Drafting Content - Specifics & Key Terms:
- Vendor Contracts: Mention "redistribution rights," "non-display usage," "termination clauses," and "audit rights" (vendors love to audit banks).
- Entitlements: Mention "Gold copies" vs. "Silver copies," Real-time vs. Delayed, Internal vs. External usage. The concept of "Entitlement Matrix."
- Lineage: Upstream (vendor) to Downstream (reporting/trading). Need to track transformations.
- Quality SLAs: Latency (ms), Completeness (fill rates), Accuracy (outliers). What happens when SLAs break? (Credits/Refunds).
- Audits: Vendor audits (software scans), Internal usage audits, Cost allocation audits.
Refining the Draft:
- Introduction: Keep it brief. Define "Market Data" (Reference data, pricing, etc.).
- Section 1 (Contracts): Emphasize the lifecycle—negotiation to renewal. Key clause: Indemnification.
- Section 2 (Entitlements): The "Inventory" is crucial here. You can't govern what you don't know.
- Section 3 (Lineage): Connect this to impact analysis. If a feed breaks, who is affected?
- Section 4 (SLAs): Make it quantitative. 99.9% uptime.
- Section 5 (Audits): Distinguish between "Financial Audit" (costs) and "Compliance Audit" (license breaches).
Self-Correction/Review during drafting: Initial thought: Just list bullet points. Correction: The prompt asks for a "Program." This implies a structured framework, not just a checklist. I should use headings, policy statements, and procedural steps.
Focus Check: Ensure I address the specific financial context. For example, in lineage, mention "Trading Desks" or "Risk Models." In contracts, mention "Exchanges" (CME, ICE, etc.) as they are the ultimate upstream licensors.
Final Polish: Add a section on "Governance Roles" (Data Stewards) and a "Technology Enablers" section (because you can't do this with spreadsheets effectively—need tools like Bloomberg AIM, BeyonD, or Collibra).
Let's structure the response now.
- Program Charter: High-level goals.
- Module 1: Vendor & Contract Lifecycle: Managing the paper.
- Module 2: Entitlement & Access Control: Managing the licenses.
- Module 3: Lineage & Impact Analysis: Managing the flow.
- Module 4: Quality & SLAs: Managing the integrity.
- Module 5: Audit & Compliance: Managing the verification.
- Roles & RACI: Who is responsible.
- Technology Stack: Tools needed.
This looks like a solid, comprehensive plan. This is a comprehensive framework for a Market Data Governance Program. This document is designed for a Chief Data Officer (CDO), Head of Market Data, or Compliance Officer to implement controls that reduce cost, mitigate licensing risk, and ensure data integrity.
Market Data Governance Program Framework
1. Executive Summary
Objective: To establish a centralized framework for managing the lifecycle of market data—acquiring, distributing, and consuming—in compliance with vendor contracts and internal quality standards. Scope: Real-time market feeds (e.g., Refinitiv, Bloomberg), reference data (e.g., Russell, S&P), and alternative data sets.
2. Pillar I: Vendor Contract Management
Goal: Ensure legal agreements match business usage and prevent "auto-renewal" budget shocks or accidental non-compliance.
Policies
- Centralized Repository: All data vendor contracts must be stored in a centralized contract management system (CMS) accessible by Legal, Procurement, and Data Governance.
- Pre-Purchase Assessment: No data may be purchased without a "New Data Request" form signed by a Budget Holder and a Data Steward confirming the intended use case (e.g., internal trading vs. client reporting).
Procedures
- Clause Standardization: Negotiate standard clauses for:
- Audit Rights: Limiting vendor audit rights to 30 days' notice and restricting scope to relevant data sets.
- Termination for Convenience: Ensuring exit clauses allow dropping data within 30-90 days.
- Usage Rights Inventory: Maintain a mapping of Permitted Use Cases per vendor (e.g., "Internal Use Only," "Derived Data," "Redistribution to Clients").
Key Performance Indicator (KPI)
- % of contracts reviewed 90 days prior to renewal.
- % of data spend with fully executed Master Service Agreements (MSAs).
3. Pillar II: Entitlements & Access Control
Goal: Enforce "Least Privilege" access to ensure users only see data the firm has paid for and licenses permit.
Policies
- Zero Standing Access: No user receives market data access by default; it must be requested and approved.
- Role-Based Access Control (RBAC): Access is granted based on job function (e.g., "Equity Trader" gets Level 2 quotes; "HR" gets none).
- License Segmentation: Distinction between "Professional" and "Non-Professional" (NP) users, as NPs often pay lower fees but have restricted data usage.
Procedures
- Access Request Workflow:
- User requests access via Service Portal.
- Manager approves (Cost center check).
- Data Steward approves (License compliance check).
- Provisioning Team executes.
- Quarterly Recertification: Managers must actively re-certify their team’s access every quarter. Non-response triggers automatic revocation.
- Leaver Process: HR triggers immediate disabling of accounts upon resignation.
KPI
- % of users with orphaned accounts (no manager recertification).
- Average time to revoke access upon termination (Target: < 4 hours).
4. Pillar III: Data Lineage & Cataloging
Goal: Track data from source to consumption to answer "Where did this price come from?" and "Who is using this feed?"
Policies
- Golden Source Designation: Official sources (e.g., "The Primary Bloomberg Feed") are designated as Golden Sources for specific attributes (e.g., "US Equity Closing Prices").
- Downstream Tracking: All downstream systems (databases, spreadsheets, APIs) must register as consumers in the Data Catalog.
Procedures
- Source Registration: Ingest metadata from feed handlers into the Data Catalog (e.g., Apache Atlas, Collibra).
- Transformation Logging: ETL/ELT jobs must log transformations (e.g., "Currency Conversion applied").
- Impact Analysis: Before sunsetting a data feed, run an "Impact Analysis" report to identify all downstream dashboards and models that rely on it.
KPI
- % of critical data elements with complete end-to-end lineage.
- Time taken to generate an "Impact Report" for a data source change (Target: < 1 hour).
5. Pillar IV: Quality & Service Level Agreements (SLAs)
Goal: Ensure data is accurate, timely, and complete, and that vendors pay penalties when they fail.
Data Quality Dimensions
- Timeliness: (e.g., Tick data must arrive within 50ms of exchange publication).
- Completeness: (e.g., 99.99% of tickers in the reference file must have a closing price).
- Accuracy: (e.g., Data must match the exchange official close).
SLA Framework
- Internal SLAs: Agreements between the Data Team and Trading Desks (e.g., "Helpdesk will respond to data spikes within 15 minutes").
- External SLAs: Contractual metrics with vendors.
- Example: If the feed uptime is < 99.9%, the vendor provides a 10% service credit.
Procedures
- Automated Monitoring: Implement data quality firewalls (e.g., Great Expectations, Deequ) to flag "Stale Data" (prices not changing) or "Spikes" (price moves > 3 standard deviations).
- Downtime Credits: A quarterly review of vendor outages to claim service credits.
KPI
- Data Quality Score (DQS) per feed.
- Total value of vendor credits claimed.
6. Pillar V: Audits & Compliance
Goal: Prove compliance to internal auditors and external vendors to avoid multi-million dollar fines.
Policies
- Audit Trail Retention: Logs of who accessed what data must be retained for a minimum of 7 years (or per regulatory requirement).
- No Shadow IT: Data cannot be procured via corporate credit cards or free trials without Governance oversight.
Procedures
- Internal Audits:
- Annual review of user lists vs. authorized personnel.
- Check for "Data Leakage" (e.g., Internal data appearing on public websites).
- Vendor Audit Simulation:
- Conduct a "Mock Audit" annually. Act as if a vendor (like Morgan Stanley or Bloomberg) is auditing you.
- Validate that entitlement reports match the contract counts.
- Incident Response:
- If a breach is found (e.g., 50 users sharing 10 licenses), immediately execute a remediation plan (purchase back-licenses or remove access) and document it.
KPI
- Number of high-risk audit findings open > 30 days.
- Result of annual "Mock Audit" (Pass/Fail).
7. Implementation Roadmap
Phase 1: Foundation (Months 1-3)
- Action: Consolidate all vendor contracts into one drive.
- Action: Define the "Golden Source" for the top 5 critical data sets (FX, Rates, Equities).
- Action: Implement quarterly user access recertification.
Phase 2: Control & Automation (Months 4-9)
- Action: Deploy a Data Catalog tool to begin mapping lineage.
- Action: Automate data quality checks (latency/stale checks) on feed handlers.
- Action: Negotiate SLA rebate clauses into upcoming contract renewals.
Phase 3: Optimization (Months 10-12)
- Action: Conduct first internal Mock Audit.
- Action: Use lineage data to identify duplicate data sources and cut costs.
8. Roles & Responsibilities (RACI)
Activity Data Steward Data Engineer Legal/Procurement Business Owner Contract Negotiation Consulted Informed Accountable Responsible User Entitlements Accountable Responsible Informed Consulted Data Quality Fixes Consulted Accountable Informed Responsible Audit Response Responsible Consulted Accountable Informed
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.