Take-Aways (AI)
- Report identifies data protection risks for LLMs along the entire lifecycle: training, application, feedback and agent systems.
- Main problems: uncontrolled training data, memorization of personal content, leakage and lack of transparency in decision-making processes.
- Recommended measures: Data minimization, privacy by design, verified anonymization and technical protection mechanisms (e.g. differential privacy, filters).
- Governance recommendations: Access roles, output controls, separate feedback processing, regular risk reviews and transparent documentation.
In March 2025, as part of the “Support Pool of Experts” program published a 100-page report on data protection risks when using large language models (LLMs) and on security measures (“AI Privacy Risks & Mitigations – Large Language Models (LLMs)”). The program serves to support the supervisory authorities in complex cases, with the aim of contributing to the application of the GDPR without being binding.
The report analyzes after a detailed Introduction also to technical aspects and Agentic AI the data protection challenges associated with the development, operation and integration of LLMs in AI systems.
Risks
The risk analysis is based on the life cycle of an LLM system:
1. risks in the training phase:
- Uncontrolled data sources: LLMs are often trained with large data sets, which can originate from web scraping, among other things. They may unintentionally contain personal or sensitive information.
- Memorization of content: The model can “memorize” personal content and reproduce it later.
- Lack of anonymization: Even if data is pre-processed, anonymization is not always sufficient.
- Legal uncertainty: The legal status of training data (e.g. consent, purpose limitation) may be unclear.
2. risks in the application of the model:
- Leakage: Outputs can refer directly or indirectly to real people or training content.
- Reconstructability: Certain inputs can cause the model to partially reproduce training patterns or source texts.
- RAGs: If data sources (e.g. a RAG) are used for prompts and the response, sensitive content may be disclosed.
3. risks due to feedback:
- Unclear data flows: Feedback from users (e.g. corrections and ratings) can be saved and processed again, including without consent/inadmissibly.
- Profile development: Continuous interactions with a system can store user-related behavior patterns.
- Lack of earmarking: Confirmation data is often not processed separately and is mixed with other data records.
4. systemic risks in agent architectures:
- Autonomy: LLMs in agent-based systems may be able to make independent decisions with potential effects on those affected.
- Lack of transparency: The decision-making process is technically complex and may not be comprehensible to outsiders.
- Risk potentiation: The more external tools and data sources are integrated, the more difficult it is to control data flows.
Measures
The document mentions risk mitigation measures such as
1. design and conception:
- Data minimization: Avoidance of unnecessary personal data sources in training, in particular no use of raw data from non-validated open sources.
- Privacy by Design: Integration of data protection principles in architecture and system planning, e.g. through modular data separation or restricted context windows.
- Technical protective measures: Installation of differential privacy or similar.
2. training and data preparation:
- Data selection: Use of verified, preferably anonymized or pseudonymized training data; preferably synthetic or aggregated sources.
- Filter: Use of automated detection and exclusion mechanisms for sensitive content (e.g. personal identifiers, health data, financial information).
- Anonymity: Validation of whether training data meets the criteria of irreversibility in the sense of the Opinion 28/2024 correspond.
3. operation and application:
- Access and role concepts: Implementation of differentiated access restrictions for admins, developers and end users.
- Output control: Use of output filters, prompt firewalls and toxicity-based blacklists to prevent the output of harmful or identifying content.
- Token level protection: Use of semantic token filtering to detect and mask sensitive content before delivery to the user.
- RAG protection: For Retrieval Augmented Generation (RAG): Access restrictions, logging of queries, validation of the data used.
4. feedback and improvement:
- Separation: Technical separation between operational use and training data.
- Consent Management: Transparent design of feedback, labeling of voluntary feedback, documented consent.
- Deletion processes: Implementation of reversible feedback storage with the option of revocation and deletion.
5. monitoring, residual risk analysis and governance:
- Risk review: Regular technical and organizational reviews, especially in the event of system updates, architecture changes or model replacements.
- Residual risk classification: Systematic assessment of accepted residual risks, based on comprehensible severity and probability of occurrence.
- Documentation and transparency: Maintenance of a risk register, logging of data protection-relevant decisions, disclosure to supervisory authorities.