Source: European Parliament
The Artificial Intelligence Act (‘AI Act’)[1] introduces proportionate rules for AI systems in the EU, following a risk-based approach.
Certain AI practices that pose an unacceptable risk are prohibited[2]. AI systems classified as ‘high-risk’ include, among others, AI systems intended to be used to evaluate the eligibility of natural persons for essential public assistance benefits and services as well as to grant, reduce, revoke, or reclaim such benefits and services[3].
These systems will have to comply with a set of requirements[4] and be assessed for their conformity prior to their placement on the market and use.
In addition, deployers of high-risk AI systems will also have obligations, such as to implement the human oversight, monitor for risks and inform affected persons about the use. Deployers who are public authorities or provide public services must also carry out a fundamental rights impact assessment.
Market surveillance authorities under the AI Act are empowered to investigate AI systems for their compliance with the AI Act, including to request corrective measures and withdrawal of the AI system from the market.
Natural and legal persons can also submit complaints that will enable rapid intervention. In addition, the AI Act strengthens the role of authorities supervising existing EU legislation on fundamental rights, such as equality bodies and data protection authorities, empowering them to effectively investigate and address potential violations, including cases of discrimination.
The Commission will cover social rating systems and discriminatory profiling practices in the guidelines on the application of the AI Act[5].
- [1] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance), PE/24/2024/REV/1, OJ L, 2024/1689, 12.7.2024.
- [2] This includes AI practices that produce a social score that leads to detrimental or unfavourable treatment of certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generated or collected or that is unjustified or disproportionate to their social behaviour or its gravity. As per Article 5(8) of the AI Act, this Article shall not affect the prohibitions that apply where an AI practice infringes other EU law.
- [3] Annex III point 5(a) of the AI Act.
- [4] E.g. risk management system, data governance to ensure data quality and avoid discrimination, transparency, documentation, human oversight, accuracy and robustness.
- [5] E.g. for prohibited practices under Article 5 of the AI Act and for high-risk AI systems under Article 6 of the AI Act.