Skip to content
TheChatArchiveThe Chat Archive
Grok 4.1 Fast· xaiHugging Face Frontier Reasoning Traces

Draft a medical device SaMD/AI compliance checklist and PMS plan.

  1. User

    Draft a medical device SaMD/AI compliance checklist and PMS plan.

  2. Assistant

    SaMD/AI Medical Device Compliance Checklist and Post-Market Surveillance (PMS) Plan

    Draft prepared by Sherlock, AI by Trailblazer Labs
    Date: [Insert Current Date]
    Scope: This checklist and PMS plan are designed for Software as a Medical Device (SaMD) incorporating AI/ML elements. They align with key regulations including EU MDR 2017/745 (Annex I, II, III), FDA SaMD Framework and AI/ML SaMD Action Plan, ISO 13485, IEC 62304, IEC 62366-1, ISO 14971, IMDRF SaMD guidance, and emerging AI-specific guidances (e.g., MDCG 2023-7, FDA Good Machine Learning Practice). Customize for jurisdiction, risk class (e.g., Class I-III or I-IV per MDR), and SaMD intended use. Not legal advice—consult regulatory experts.


    1. SaMD/AI Compliance Checklist

    Use this checklist during pre-market development, submission (e.g., EU Technical Documentation, FDA 510(k)/De Novo/PMA), and audits. Mark as Compliant (Y/N/NA), with evidence references.

    1.1 Regulatory Classification and General Requirements

    ItemDescriptionY/N/NAEvidence
    1.1.1SaMD defined per IMDRF/FDA (standalone software with medical purpose; no hardware integration unless specified).
    1.1.2Risk classification completed (e.g., FDA I/II/III; EU Class I/IIa/IIb/III; IMDRF a/b/c/d based on significance of state, info, clinical condition).
    1.1.3AI/ML classification addressed (locked vs. adaptive/continuous learning models).
    1.1.4GSPR/QSR conformity declared (EU Annex I; FDA QSR 21 CFR 820).
    1.1.5UDI assigned and implemented (Unique Device Identification).

    1.2 Quality Management System (QMS)

    ItemDescriptionY/N/NAEvidence
    1.2.1ISO 13485:2016 QMS certified/implemented (design controls, CAPA, complaints, etc.).
    1.2.2Software lifecycle processes per IEC 62304 (Class A/B/C based on risk).
    1.2.3Change control for AI model updates (versioning, retraining protocols).

    1.3 Technical Documentation (EU Annex II/III; FDA Design History File)

    ItemDescriptionY/N/NAEvidence
    1.3.1Detailed description of SaMD/AI architecture (algorithms, data inputs/outputs, training/validation datasets).
    1.3.2Bill of materials/software architecture diagram.
    1.3.3Cybersecurity design (encryption, access controls, vulnerability assessments per FDA cybersecurity guidance).
    1.3.4Performance specs, algorithms, and limitations documented.

    1.4 Risk Management (ISO 14971:2019)

    ItemDescriptionY/N/NAEvidence
    1.4.1Risk Management File (RMF) with hazard analysis, FMEA/FTA for SaMD/AI-specific risks (e.g., bias, overfitting, adversarial attacks).
    1.4.2AI risks mitigated: data bias, dataset diversity, explainability (e.g., SHAP/LIME), robustness testing.
    1.4.3Residual risks acceptable and disclosed in IFU.
    1.4.4Post-market risk integration (benefit-risk analysis).

    1.5 Clinical Evaluation (MEDDEV 2.7/1 Rev 4; FDA Clinical Guidance)

    ItemDescriptionY/N/NAEvidence
    1.5.1Clinical Evaluation Plan (CEP) and Report (CER) with literature review, clinical investigations, or equivalence.
    1.5.2AI performance metrics: sensitivity/specificity, AUC-ROC, calibration plots; stratified by subgroups (bias analysis).
    1.5.3Real-World Evidence (RWE) plan if applicable for adaptive AI.
    1.5.4Scientific validity, analytical/clinical validation demonstrated.

    1.6 Verification, Validation, and Usability

    ItemDescriptionY/N/NAEvidence
    1.6.1Software V&V per IEC 62304 (unit/integration/system testing).
    1.6.2AI-specific testing: dataset splits (train/valid/test), cross-validation, external validation sets.
    1.6.3Usability engineering per IEC 62366-1 (formative/summative testing with users).
    1.6.4Simulated use, stress testing, interoperability testing.

    1.7 AI/ML-Specific Considerations (FDA GMLP; MDCG AI Guidance)

    ItemDescriptionY/N/NAEvidence
    1.7.1Dataset governance: source, size, diversity, preprocessing documented; bias mitigation (e.g., reweighting, augmentation).
    1.7.2Model transparency/explainability (e.g., feature importance).
    1.7.3Algorithm transparency (pre-determined vs. ML; training process).
    1.7.4Change management for model updates (Predetermined Change Control Plan for adaptive AI).
    1.7.5Ethical AI: fairness, accountability documented.

    1.8 Labeling, Instructions for Use (IFU), and Promotion

    ItemDescriptionY/N/NAEvidence
    1.8.1IFU includes intended use, limitations, warnings (e.g., AI confidence scores, operator training).
    1.8.2Symbols per ISO 15223; multilingual as required.
    1.8.3Promotional claims substantiated (no off-label).

    1.9 Pre-Market Submission Readiness

    ItemDescriptionY/N/NAEvidence
    1.9.1Notified Body/competent authority engagement (if Class IIa+).
    1.9.2Clinical trials registered (if required, e.g., ClinicalTrials.gov).
    1.9.3PMS/Vigilance plan in place (see Section 2).

    Completion Notes: Total items compliant: /. Gaps actioned by [date]. Auditor sign-off: ________________.


    2. Post-Market Surveillance (PMS) Plan

    Document Reference: [e.g., PMS-001 v1.0]
    Device: [SaMD/AI Name, Model, UDI] | Risk Class: [e.g., IIb] | PMS Responsible: [QRA/Regulatory Lead]
    Review Frequency: Annual (or post-update); integrate with PSUR (EU) / PMSR (FDA).
    Objective: Proactively monitor safety, performance, and effectiveness post-market to detect signals, mitigate risks, and support lifecycle changes (MDR Art. 83-86, Annex III; FDA 21 CFR 803/822). For AI: Monitor drift, bias emergence, real-world performance.

    2.1 PMS Objectives and Scope

    • Collect/analyze data on foreseeable risks, usability, clinical performance/benefits.
    • Detect new/ emerging risks (esp. AI: dataset shift, adversarial inputs).
    • Support PSUR/PMSR, CAPA, field safety notices, recalls.
    • Metrics: Complaint rates, adverse events (MAUDE/EUDAMED), performance KPIs (e.g., AUC drift <5%).

    2.2 Data Sources and Collection Methods

    Data SourceCollection MethodFrequencyResponsibleAnalysis Tool
    User complaints/helpdeskAutomated ticketing (e.g., Zendesk), hotlineReal-timeCustomer SupportTrend analysis (Excel/Tableau)
    Adverse events/vigilanceEUDAMED/FDA MedWatch/MDR reportingReal-timePharmacovigilanceSignal detection software (e.g., Oracle Argus)
    Registry/RWE studiesEHR integration, patient registriesQuarterlyClinical/HEORStatistical software (R/Python)
    Literature/scientificPubMed/Google Scholar alertsMonthlyMedical AffairsReview database
    User analytics/telemetryUsage logs, error rates, AI confidence scoresContinuousData AnalyticsML monitoring (e.g., Evidently AI)
    Audits/capabilitiesSupplier audits, cybersecurity scansAnnualQMSAudit reports
    PSUR inputsSales/volume data, market feedbackAnnualRegulatoryPMS database

    2.3 Signal Detection and Risk Assessment

    • Thresholds: Complaint rate >0.1%; performance degradation >10%; new risk probability >1/10,000.
    • Methods: Statistical process control (SPC), Bayesian anomaly detection for AI drift; FMEA refresh annually.
    • Escalation: Signals → Risk Review Board (within 7 days) → CAPA if confirmed.
    • AI-Specific: Monitor input data drift (e.g., KS-test), model retraining triggers.

    2.4 Reporting and Communication

    ReportContentFrequencyRecipients
    PMS ReportData summary, signals, actionsAnnualNB/RA/Management
    PSUR (EU Class IIa+)Benefit-risk update, clinical dataEvery 2 years (post-CE mark)NB/EUDAMED
    Field Safety NoticeConfirmed issuesAs neededUsers/RA
    Trend ReportAI performance KPIsQuarterlyInternal

    2.5 Resource Allocation and Training

    • Team: QRA (lead), Data Scientist (AI), Clinical Expert.
    • Budget: [e.g., $XXK/year for tools/studies].
    • Training: Annual PMS/AI ethics for staff; SOPs: PMS-001, AI-MON-001.

    2.6 Plan Review and Updates

    • Review annually or post-model update/change.
    • Metrics for PMS effectiveness: Signal detection rate >95%, time-to-resolution <30 days.
    • Approval: ________________ Date: __________ Next Review: [Date +1yr].

    Annexes: [PMS SOPs, Data Collection Forms, Signal Log Template].


    This draft provides a robust starting point. For implementation, integrate into your QMS and validate with a mock audit. Questions? Provide specifics for tailoring (e.g., jurisdiction, AI type).

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.