Article 9 — Risk Management System

Legal Provision

Risk management system: establishes the requirement to have in place a risk management system, prior and during the deployment of the AI system.

Scope

Applies to providers of the high-risk AI system.

Requirement type

Duty: when a high-risk AI system is placed on the market or put into service and continuously throughout its lifecycle.

Actions to demonstrate compliance

A. Binary (yes/no): if these elements are present, there is evidence towards demonstrating compliance.

• Documented risk-management system.
• Testing, prior to the placement in the market, of the high-risk AI system for risk management purposes.

B. Non-binary (require a contextual assessment): building on the binary actions, a contextual assessment of these points is required to further verify compliance with each specific legal basis.

• Identification and analysis of known and foreseeable risks (including misuse).
• Evaluation of identified risks.
• Acceptable residual risk.
• Metrics and probabilistic thresholds appropriate for the intended purpose of the high-risk AI system.
• Possible impact on minors or vulnerable groups.
• Proof of feedback integration from post-market insights into the risk management cycle.

Monitoring requirement

• Verification that the provider is monitoring and feeding information back into the risk-management system.
• Among others:
  • a. Existence of a documented post-market monitoring plan.
  • b. Verifiable mechanisms for collecting real-world data.
  • c. Documented processes for analysing post-market data.
  • d. Incident and malfunction tracking (e.g., registers, mitigation measures with timestamps).
  • e. Feedback loop into the risk-management system.
  • f. Records of updates to the AI system or safeguards (e.g., dates of updates, records describing new training data, retraining logs…).
  • g. Evidence of internal or external audits conducted to verify ongoing compliance with the risk management obligations.

Consequences of non-compliance

• Administrative fines up to €15 000 000 or 3% of total worldwide annual turnover.
• Penalties and enforcement measures adopted at the Member State level.