Build a fair lending compliance for ML: ECOA/Reg B, explainability, adverse action, and…
- User
Build a fair lending compliance for ML: ECOA/Reg B, explainability, adverse action, and monitoring.
- Assistant
Fair Lending Compliance Framework for Machine Learning Models
Executive Summary
This framework establishes a comprehensive compliance program for machine learning (ML) models used in lending decisions, ensuring adherence to the Equal Credit Opportunity Act (ECOA) and Regulation B (Reg B, 12 CFR Part 1002). It addresses key pillars: non-discrimination, explainability, adverse action notices, and ongoing monitoring. The program mitigates risks of disparate treatment (intentional discrimination) and disparate impact (unintentional effects on protected classes), while promoting transparency and accountability.
Applicability: All ML models involved in credit underwriting, pricing, account management, or servicing (e.g., approval/denial, interest rates, limits).
Regulatory Foundation:
- ECOA/Reg B: Prohibits discrimination on prohibited bases (race, color, religion, national origin, sex, marital status, age (provided able to contract), receipt of public assistance, good faith exercise of ECOA rights).
- CFPB Guidance: U.S. 2019 Advisory on AI/ML in fair lending (focus on testing, documentation, monitoring).
- NCUA/FFIEC: Similar expectations for credit unions and supervised institutions.
Implementation Timeline: Phase 1 (Development/Validation: 3 months), Phase 2 (Deployment: 1 month), Phase 3 (Monitoring: Ongoing).
1. Governance and Responsibilities
Role Responsibilities Fair Lending Committee (FLC) Oversight; approves models; reviews audits. Chaired by Chief Compliance Officer (CCO). Meets quarterly. Data Science Team Builds/fairness-tests models; documents explainability. Compliance Team Validates models; generates adverse action notices; monitors performance. Legal/External Audit Annual independent review; regulatory reporting. Business Units Provides feedback; ensures operational integration. - Training: Annual for all involved (e.g., ECOA basics, bias detection).
- Documentation: Model cards for every ML model (inputs, outputs, fairness metrics, risks).
2. Model Development and Validation (ECOA/Reg B Compliance)
Prevent discrimination at the source through rigorous fairness integration.
2.1 Data Management
- Prohibited Practices:
- No direct use of prohibited bases.
- No proxy variables (e.g., ZIP code as race proxy → geocode to census block groups only if justified and tested).
- Data Sourcing:
Step Controls Collection Anonymize sensitive attributes; augment with synthetic data if imbalanced. Cleaning Remove/test for correlations with protected classes (e.g., Pearson > 0.7 flags proxy). Labeling Audit for labeling bias (e.g., human raters' demographics).
2.2 Bias Detection and Mitigation
- Fairness Metrics (Threshold: <5% disparity):
Metric Definition Target Demographic Parity P(approve|protected) ≈ P(approve|unprotected) Approval rates equal. Equal Opportunity True Positive Rate equal across groups. Qualified applicants approved equally. Equalized Odds True/False Positive Rates equal. Low disparate impact. Disparate Impact Ratio Approval minority / approval majority. ≥0.80 (80% rule). - Techniques:
- Pre-processing: Reweight/resample data (e.g., SMOTE for minorities).
- In-processing: Fairness constraints in training (e.g., AIF360 library).
- Post-processing: Threshold adjustment for approvals.
- Validation:
- Holdout test sets stratified by protected classes (proxy via available data like sex/marital status; infer others ethically if needed).
- Statistical tests: Fisher's exact test for disparities (p<0.05 flags issue).
- Document overrides (e.g., "Proxy risk mitigated via reweighting").
Approval Gate: Model cannot deploy without FLC sign-off on fairness report.
3. Explainability and Interpretability
ML models must produce human-understandable rationales for decisions (Reg B §1002.9 → clear, specific reasons).
-
Requirements:
- Global: Model-agnostic explanations (e.g., feature importance via permutation).
- Local: Instance-level (e.g., why this applicant denied).
-
Techniques (Prioritize interpretable models like GLMs; fallback to XAI):
Method Use Case Example Output SHAP/LIME Black-box "Credit score contributed -25% to denial (below threshold)." Partial Dependence Plots Global Plot showing income's non-linear impact. Counterfactuals Local "Approval if income >55k)." Rule Lists Rule extraction "IF debt/income >40% AND score <650 → Deny." -
Model Card Template:
Model: CreditRiskML_v2 Intended Use: Underwriting Fairness: DI Ratio=0.85 (sex); Metrics Table Explainability: SHAP integrated; Top 4 reasons extracted. Risks: Potential proxy in occupation → Mitigated. -
Testing: Blind audits (Compliance reviews 100 explanations quarterly).
4. Adverse Action Notices (Reg B §1002.9)
Automated generation for denials, unfavorable terms, or counters.
-
Content Requirements:
Element ML Implementation Principal Reasons (Top 4-5) Top SHAP values mapped to Reg B codes (e.g., "Debt obligations too high"). Score/Threshold "Your ML risk score: 620/850 (threshold: 650)." Clear Language No jargon; consumer-tested. Protected Class Disclosure Include ECOA rights notice. -
Process:
- ML decision → Extract top reasons.
- Map to standardized list (Appendix C Reg B: e.g., "Number of credit inquiries").
- Generate PDF/email within 30 days.
- Log for audits.
-
Edge Cases: If ML fully automated, include "Automated decision; contact for review."
Example Notice Snippet:
Adverse Action Notice We regret to inform you your application was denied. Principal Reasons: 1. Credit history (35% impact) 2. Debt-to-income ratio (25% impact) 3. Income level (20% impact) ML Score: 580 (Cutoff: 620)5. Ongoing Monitoring and Reporting
Detect drift/bias post-deployment.
5.1 Monitoring Dashboard (Weekly Automated)
Metric Frequency Alert Threshold Fairness Drift Weekly DI Ratio <0.80 Model Performance Daily AUC <0.75; Calibration drift >10% Adverse Action Trends Monthly Disparity in notices by group >10% Proxy Monitoring Quarterly Feature correlations re-tested - Tools: MLflow/Pachyderm for logging; AIF360 for fairness.
- Audits:
- Internal: Quarterly (sample 500 decisions).
- External: Annual (third-party fairness audit).
- Regulatory: Retain 3 years of logs.
5.2 Response Protocol
Issue Action Bias Detected Pause model; retrain; notify FLC. Explainability Failure Fallback to legacy model. High Complaints Root-cause analysis; CFPB report if systemic. 6. Risk Assessment and Continuous Improvement
- Initial Risk Score: High for complex ML → Full framework.
- KPIs: 100% compliance rate; <2% disparity.
- Updates: Review annually or post-regulation (e.g., CFPB AI rules).
This framework ensures defensible, ethical ML lending. For implementation support (code/templates), provide specifics on your tech stack (e.g., Python/TensorFlow). Questions?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.