Things you should know about the EU AI Act and data management

Why are the EU AI Act and data management critical for businesses? Set to become the world’s first comprehensive regulation governing artificial intelligence (AI), the EU AI Act (“the Act”), proposed by the European Commission, aims to ensure ethical, secure, and responsible AI practices while safeguarding individuals’ rights. As AI rapidly revolutionizes industries and economies and integrates into daily life, robust governance is crucial. In this article we help you gain an understanding of the EU AI Act and the importance of data management, provide recommendations for preparation, and finally share strategic business opportunities not to be missed.

Understanding the EU AI Act: a strategic imperative

Effective AI governance is essential to mitigate risks, protect individual rights, and promote public trust in AI systems. It encompasses various dimensions, including transparency, accountability, fairness, and security. The Act categorizes AI systems into four risk levels. Each category is subject to different regulatory requirements, with the most stringent measures applied to high-risk AI systems.

Practical scenario: Consider a company developing an AI recruitment tool that faces allegations of bias due to poor data practices – e.g., it was trained on past hiring data that favored a particular demographic, like candidates from certain universities. Because of this, the system automatically filters out resumes of candidates from certain backgrounds, even though they are highly qualified, resulting in a shrinking talent pool, legal and reputation risks (e.g., discrimination lawsuits and bad press), and lack of innovation (e.g., a homogenous team leads to weaker problem-solving). These serious issues could have been avoided with strong governance.

To share a real-life example, take the case of the National Health Service (NHS), in the UK, where an AI system shared patient data without consent back in 2015, resulting in legal and ethical concerns, and eventually legal action.

The EU AI Act lays out the following foundational pillars, to help organizations using AI avoid situations like the above:

• Prohibition of unacceptable AI practices: AI systems that pose a clear threat to safety, livelihoods, and rights are banned. This includes AI systems that manipulate human behavior or exploit vulnerabilities.
• Regulation of high-risk AI systems: High-risk AI systems, such as those used in critical infrastructure, education, employment, and law enforcement, must comply with strict requirements related to data quality, transparency, and human oversight.
• Transparency obligations: AI systems interacting with humans, generating deepfakes, or used for biometric identification must disclose their nature to users.
• Governance and enforcement: The Act establishes national supervisory authorities and a European Artificial Intelligence Board to ensure compliance and enforcement.

Businesses developing or deploying high-risk AI must maintain detailed technical documentation, ensure traceability of data used for AI training and testing, and conduct conformity assessments before deployment.

The cornerstone of compliance: the unavoidable importance of data management

At the cornerstone of AI compliance is the reliance on data, making effective data management indispensable for compliance with the EU AI Act. In what ways is data essential in AI management?

1. Data quality and integrity: High-quality, accurate, and unbiased data is essential for training reliable AI models.
2. Data privacy and security: Compliance with data protection regulations, such as the General Data Protection Regulation (GDPR), is paramount. Organizations must adopt measures to protect personal data and ensure data security.
3. Data transparency and accountability: Transparent data practices and clear documentation are critical for accountability. Organizations should maintain detailed records of data sources, processing methods, and AI model performance.
4. Ethical data use: Ethical considerations should guide data collection, processing, and usage. Organizations must avoid biases, discrimination, and unfair practices in their AI systems

Practical scenario: A company discovers inconsistencies in its data governance during an audit, delaying a new AI product launch – e.g., imagine an e-commerce company developing an AI-powered pricing system realizing that the AI system was trained on scraped competitor pricing data (violating antitrust regulations), and customer purchase history data without obtaining explicit user consent. Early investment in governance could have prevented these unnecessary roadblocks.

To look at a real-life example, EY recently assisted a public services organization in defining a robust operating model for its AI program, enhancing data quality and accountability in order to streamline product development, reduce compliance risks, and accelerate time-to-market for innovative AI solutions.

Preparing for today and for the future: key actions for businesses

As the EU AI Act paves the way for a regulated AI landscape, it’s crucial for European businesses to prepare for compliance. Proactive steps will not only help avoid penalties but also enable organizations to leverage AI ethically and responsibly, ensuring a competitive edge in the future.