Build a model risk management (SR 11‑7) program: inventory, tiering, validation,…
- User
Build a model risk management (SR 11‑7) program: inventory, tiering, validation, outcomes analysis, and change control.
- Assistant
Model Risk Management (MRM) Program Compliant with SR 11-7
This document outlines a comprehensive Model Risk Management (MRM) Program aligned with the Federal Reserve's SR 11-7 Guidance on Model Risk Management. The program establishes robust policies, processes, and controls to identify, measure, monitor, and mitigate model risk across the organization. It emphasizes three lines of defense: Model Owners (1st line), Model Risk Management (MRM) Team (2nd line), and Internal Audit (3rd line).
Key principles:
- Board/Senior Management Oversight: Approve the program, review reports quarterly.
- MRM Committee: Chaired by CRO or equivalent; meets monthly to review inventory, tiers, validations, outcomes, and changes.
- Tools: Centralized repository (e.g., SharePoint, Collibra, or custom database) for inventory and documentation.
- Training: Annual for all model stakeholders.
- Reporting: Quarterly MRM dashboard to senior management; annual report to Board.
The program covers the requested components: Inventory, Tiering, Validation, Outcomes Analysis, and Change Control.
1. Model Inventory
Maintain a living, centralized Model Inventory that catalogs all models in use, development, or retirement. Updated monthly or upon material changes.
Inventory Fields (Minimum)
Field Description/Example Model ID Unique alphanumeric (e.g., MOD-2024-001). Model Name Descriptive name (e.g., "PD Credit Risk Model"). Model Type Statistical, machine learning, algorithmic, vendor, etc. Owner/Business Unit Primary owner and unit (e.g., Risk Analytics Team). Users/Stakeholders Departments using outputs (e.g., Credit Approval). Purpose/Use Key uses (e.g., capital calculation, pricing). Materiality Yes/No (impacts >$X million or regulatory filings). Tier High/Medium/Low (see Tiering section). Development Date Initial build date. Status Development/Production/Retired. Vendor If third-party (e.g., Moody's). Data Sources Inputs (e.g., internal loan data). Next Validation Due date. Last Outcomes Review Date of last monitoring. Process:
- Model Owners submit new models via intake form.
- MRM Team reviews/approves additions within 5 business days.
- Automated alerts for overdue reviews.
2. Model Tiering
Classify models by risk level (High, Medium, Low) based on Impact (business/financial/regulatory) and Complexity (methodology/data). Reviewed annually or upon changes.
Tiering Matrix
Impact → Complexity ↓ Low Complexity (e.g., simple rules-based) Medium Complexity (e.g., logistic regression) High Complexity (e.g., neural networks, ensembles) High Impact (e.g., >$50M decisions, regulatory capital) Medium High High Medium Impact (e.g., $10-50M, internal limits) Low Medium High Low Impact (e.g., < $10M, non-material reports) Low Low Medium Criteria:
- Impact: Potential loss, decision stakes, regulatory scrutiny (e.g., CCAR models = High).
- Complexity: Uncertainty in assumptions, data quality, sensitivity to inputs.
- Escalation: Any model with AI/ML or vendor black-box = Minimum Medium.
Process:
- Model Owner proposes tier; MRM Team approves.
- High-tier models require pre-implementation approval.
3. Validation
Independent Model Validation ensures models are conceptually sound, implemented correctly, and perform appropriately. Performed by qualified validators independent of developers/owners.
Validation Types and Cadence
Tier Initial Validation Periodic Validation Targeted Validation High Pre-implementation Annual Upon material change/use case expansion Medium Pre-implementation Biennial Upon moderate change Low Pre-production Every 3 years Upon significant change Validation Report Contents:
- Conceptual soundness review.
- Data quality assessment.
- Implementation testing (code walkthroughs).
- Outcomes testing (backtesting).
- Benchmarking against alternatives.
- Recommendations (e.g., recalibrate).
Process:
- MRM Team assigns validator (internal expert or external consultant).
- Timeline: 60 days for High-tier.
- Outcomes: Approved / Approved with Conditions / Rejected.
- Remediation tracked in inventory.
4. Outcomes Analysis
Ongoing monitoring to detect performance degradation. Includes backtesting, benchmarking, and stress testing. Reviewed quarterly for High-tier, semi-annually for others.
Key Activities
Activity Description Thresholds for Escalation Performance Monitoring Track key metrics (e.g., AUC, RMSE) vs. benchmarks. >10% drift → Alert. Backtesting Compare predictions vs. actuals (e.g., default rates). Binomial tests; p-value <0.05 → Material weakness. Benchmarking Compare to challenger models or industry standards. Relative error >15% → Review. Sensitivity/Stress Testing Test under adverse scenarios (e.g., recession data). Output variance >20% → Recalibrate. Stability Analysis Monitor input data drift (e.g., KS test). Data shift >0.1 → Retrain. Process:
- Automated dashboards (e.g., Tableau) with alerts.
- MRM Committee reviews exceptions; escalates to senior management.
- Document in "Outcomes Log" linked to inventory.
5. Change Control
Formal process for model changes to prevent unauthorized modifications and ensure re-validation.
Change Categories
Category Examples Approval Required Re-Validation Material New use case, major recalibration, code rewrite. MRM Committee Full Moderate Minor parameter tweak, data source update. MRM Team Targeted Administrative Documentation update, owner change. Model Owner None Change Control Workflow:
- Request: Submitter logs change in repository (fields: rationale, impact assessment, risk).
- Review: Model Owner assesses tier impact (1-3 days).
- Approval: Per table above (documented vote).
- Implementation: In controlled environment; test pre/post.
- Post-Change: Update inventory, schedule validation/outcomes review.
- Audit Trail: Version control (e.g., Git for code); retain 7 years.
Prohibitions: No changes without approval; emergency changes require after-action review within 48 hours.
Implementation Roadmap
Phase Timeline Milestones 1: Setup Month 1 Deploy inventory tool; train staff; baseline inventory. 2: Assess Months 2-3 Tier all models; complete initial validations for High-tier. 3: Operate Month 4+ Launch monitoring dashboards; first MRM Committee meeting. 4: Enhance Ongoing Annual audits; incorporate lessons learned. Metrics and Reporting
- KPIs: % models validated on time (>95%), overdue outcomes (0%), change approval rate.
- Dashboard: Real-time view of inventory status, tier distribution, open issues.
- Escalation: Material issues to Board within 30 days.
This program is scalable and adaptable. For customization (e.g., thresholds), consult legal/regulatory teams. Contact MRM Team Lead for questions.
Approved by: [Senior Management Signature Placeholder]
Effective Date: [Insert Date]
Version: 1.0
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.