On December 17, 2024, the European Data Protection Board (EDPB) adopted an opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models (opinion), shedding light on significant privacy concerns in the development and deployment of Artificial Intelligence (AI) models.
This opinion determines how the EU General Data Protection Regulation (GDPR) privacy principles are specifically applied to the unique context of AI models. The opinion ties these principles to the use of practical, technological measures, including implemented appropriate and effective privacy-preserving techniques such as differential privacy. Furthermore, the opinion references the forthcoming EU Artificial Intelligence Act (AI Act) to underscore complementary objectives. While this linkage is not extensively detailed, the opinion’s principles naturally complement the AI Act’s emphasis on accountability, risk-based governance, and transparency, creating a cohesive framework that navigates the balance between privacy and innovation.
Key Points
The opinion highlights three key issues organizations using and/or developing AI must do:
- determine the circumstances under which AI models can be considered anonymous;
- evaluate the feasibility of using legitimate interest as a legal basis; and
- consider the consequences of using personal data that was processed unlawfully.
Anonymity of AI Models
The EDPB considers that AI models trained on personal data are not anonymous by default and should be assessed on a case-by-case basis. It is important to note that this conclusion applies not just to the underlying training data but also to the AI model itself and its outputs, as these may inadvertently reveal personal information or patterns derived from the data. For an AI model to be considered anonymous both “(i) the likelihood of direct (including probabilistic) extraction of personal data regarding individuals whose personal data were used to train the model; as well as (ii) the likelihood of obtaining, intentionally or not, such personal data from queries, should be insignificant for any data subject”. In other words, controllers are required to prove, through robust measures, that data subjects cannot be identified from these models. This applies both to the data used during the training phase and the outputs generated by the AI model. For instance, organizations must ensure that training data is thoroughly anonymized to prevent re-identification and that outputs do not unintentionally disclose sensitive information.
This entails adopting various approaches and technical measures, such as:
- implementing data filtering processes to remove irrelevant personal data;
- implementing appropriate and effective privacy-preserving techniques such as differential privacy;
- employing pseudonymization to replace identifiable elements with pseudonyms; and
- utilizing federated learning to train models across decentralized datasets without transferring personal data.
These measures go beyond compliance and aim to build trust in AI models by prioritizing robust data security and protecting privacy.
Legitimate Interest as a Legal Basis
The opinion reviews the ability to use legitimate interest as a basis for processing personal data in the context of training algorithms for AI models. While recognising this option, the opinion emphasizes the need to follow the standard three-step Legitimate Interest Analysis (LIA) under GDPR, which evaluates “(i) the pursuit of a legitimate interest by the controller or by a third party; (ii) the processing is necessary to pursue the legitimate interest; and (iii) the legitimate interest is not overridden by the interests or fundamental rights and freedoms of the data subjects”.
The EDPB highlights that, under the first criterion, a legitimate interest must meet three key principles (as outlined in Section 68 of the opinion): (a) the interest is lawful; (b) the interest is clearly and precisely articulated; and (c) the interest is real and present, not speculative. Examples provided include improving threat detection in an information system, enhancing operational security, and optimizing resource allocation in AI systems. These are illustrative examples, and other lawful purposes, decided on a case-by-case basis, may also qualify as legitimate interests under the GDPR, depending on the specific circumstances of processing.
The second criterion involves a “necessity test” demonstrating that the processing is essential to achieve the stated purpose. In other words, this criterion requires an analysis of the alternatives available for reaching the same legitimate interest. For example, the amount of personal data used in the AI model should be evaluated to determine if less intrusive alternatives could achieve the same goal of the legitimate interest just as effectively.
Finaly, the third criterion involves a “balancing test” between the data subjects’ fundamental rights and the interests of the the controller. The EDPB states that the more specifically an interest is defined in relation to the intended purpose of the processing, the easier it becomes to clearly understand the benefits and risks that need to be considered in the balancing test. In addition, the EDPB suggests elaborating the data subjects’ existing rights to ensure said balance, such as, allowing the data subjects to exercise their right to erasure even when the specific grounds listed in Article 17(1) GDPR do not apply. Given the availability of various and flexible technical alternatives and solutions, it seems companies can find the silver lining by ensuring compliance through implementing privacy-enhancing technologies into their products and models to facilitate the required balance, even in cases where the processing may initially seem intrusive.
The Consequences of Using Personal Data That Was Processed Unlawfully
While fines under GDPR can reach up to €20 million or 4% of annual global income, a critical risk lies in the corrective actions imposed. These can include erasing part of the dataset that was processed unlawfully or, where this is not possible, depending on the facts at hand, having regard to the proportionality of the measure, ordering the erasure of the whole dataset used to develop the AI model and/or the AI model itself. As we have seen such threats fulfilled in the well-known Clearview AI case (see for example the Australian decision here), the company was ordered to delete its algorithmic templates created through the use of unlawful personal data.
A Broader Perspective: The Intersection with the EU AI Act
The EDPB’s guidance aligns seamlessly with the EU AI Act, creating a comprehensive framework for managing high-risk AI systems. Together, these frameworks emphasize transparency, accountability, and harmonized governance. The EDPB’s call for Data Protection Authorities (DPAs) to oversee high-risk AI systems ensures that GDPR principles are upheld, even as the AI landscape evolves.
Technologies like Federated Learning and Differential Privacy are not only solutions for GDPR compliance but also align with the AI Act’s emphasis on minimizing risks while fostering innovation. These interconnected frameworks highlight the EU’s forward-thinking approach to AI governance, balancing technological progress with fundamental rights protection.
Practical Takeaways
Organizations developing AI systems should take proactive steps to align with these evolving standards. In light of the opinion and its application within the broader context of the EU AI Act, organizations shall verify the lawfulness of the personal data used by them for AI development purposes; consider, inter alia, implementing privacy-preserving techniques like differential privacy or pseudonymisation; documenting compliance efforts through detailed records; and staying ahead of regulatory developments under applicable regulations.
To avoid these consequences, organizations must ensure their use of AI models aligns with said opinion and upcoming applicable regulations. We at APM will be happy to assist you by conducting thorough data protection impact assessments, implementing necessary changes, and guiding you in order to ensure compliance in your future products as well. Contact us for expert guidance on adapting to these evolving regulations.
APM Privacy, Technology and Regulation Team.
This document is intended to provide only a general background regarding this matter. It should not be regarded as setting out binding legal advice but rather as a practical overview based on our understanding.
