Advancing responsible AI by tackling social bias in language models

Photographic portrait of Yara Elias
Yara Elias

EY Canada Senior Manager, AI Risk, Risk Consulting

3 minute read 04 Apr. 2025
Related topics
AI
Trust

EY's research tackles social bias in AI, offering insights and methodologies for ethical language models and responsible innovation.

Co-authored by:

In brief

  • EY, in collaboration with the Vector Institute and Queen’s University, conducted pivotal research to address social bias in developing actionable strategies and methods for the ethical use of Gen AI in various industries.
  • Key findings from two studies highlight innovative approaches for social bias mitigation, including the BiasKG framework and effective unlearning techniques, positioning organizations to enhance transparency and trust in their AI systems.

In today’s ever more rapidly advancing technological landscape, staying ahead in AI development is essential for organizations aiming to maintain a competitive edge.

Recognizing this necessity, EY team worked with the Vector Institute, Queen’s University and Industry Participants on a collaborative research project, Uncovering Bias in LLMs. This initiative combined leading -class knowledge in AI and machine learning with advanced AI engineering capabilities to transform innovative research into practical industry solutions and promote adoption across sectors. Through this collaboration, EY team contributed to advancements in bias mitigation techniques and developed valuable tools and insights to support the ethical and effective use of AI in real-world scenarios.

The widespread adoption of large language models (LLMs) such as GPT-4 and LLaMA-2 in business operations has created an urgent need to address potential social biases in these systems.

In response, two pivotal studies were conducted through a collaborative research initiative. EY team worked with the Vector Institute, researchers and other key organizations to develop critical insights and actionable strategies for mitigating bias while preserving system performance. These studies shed light on how LLMs can produce biased outputs and proposed advanced strategies to address the issue.

The first study, BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models, reveals LLMs’ susceptibility to manipulation and social bias. Using a specialized technique, BiasKG, researchers exposed how biases become particularly pronounced when models are given biased knowledge in context.

BiasKG is a novel knowledge graph with social biases to introduce the knowledge to the model via retrieval-augmented generation (RAG). The effect is enhanced when the model is asked to provide explanations, linking fairness to transparency and ethical AI. BiasKG identified these biases and demonstrated their utility as a framework for testing and evaluating the robustness of AI systems against societal bias.

How EY can help

  • AI Insights

    Explore our global insights that look at how you can build confidence in AI, drive exponential value throughout your organization and deliver positive human impact. Learn more.

    Read more

This work emphasizes the importance of early-detection tools for businesses to safeguard trust and enhance the ethical use of AI. This research paper was accepted at the TrustNLP workshop during the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).

 

The second study, Mitigating Social Biases in Language Models through Unlearning, explored state-of-the-art methods for reducing bias in LLMs without the computational cost of retraining. Among the approaches tested, negation via task vector (TV) emerged as the most effective one. It reduced bias in some models by up to 40% while preserving performance and providing the flexibility to adapt to specific needs.

 

Direct preference optimization (DPO) proved effective but was more computationally intensive, while partitioned contrastive gradient unlearning (PCGU) demonstrated potential but required refinement to ensure coherence and consistent results.

 

These findings underscore the trade-offs between various techniques and highlight TV’s scalability and adaptability as a standout option for organizations seeking to balance fairness and operational efficiency. This work was accepted at the TrustNLP workshop at the 2024 NAACL conference and the 2024 Empirical Methods in Natural Language Processing (EMNLP) conference as part of the Industry Track.

 

Together, these studies provide organizations with a robust understanding of AI bias and offer practical tools to address it. By implementing the insights and techniques developed through this collaborative research, businesses can make their AI systems ethical, transparent and aligned with their long-term strategic goals. These advancements position forward-thinking companies as leaders in responsible AI innovation.

 

This research represents a collaborative effort by seasoned AI professionals. Contributors to BiasKG include Chu Fei Luo and Faiza Khan Khattak of the Vector Institute, Ahmad Ghawanmeh of EY, and Xiaodan Zhu of Queen’s University .

 

The Machine Unlearning study was authored by Omkar Dige and Faiza Khan Khattak of the Vector Institute, Diljot Singh, Tsz Fung Yau and Mohammad Bolandraftar of Scotiabank, Qixuan Zhang from EY, and Xiaodan Zhu of Queen’s University.

 

The views reflected in this article are the views of the author and do not necessarily reflect the views of the global EY organization or its member firms.

Summary

EY, in partnership with the Vector Institute and Queen’s University, has conducted essential research to combat social bias in large language models (LLMs). This initiative produced actionable strategies and methods for ethical use of Generative AI. Key findings from two studies introduced the Bias knowledge Graph (BiasKG) framework and effective unlearning techniques, empowering organizations to enhance transparency and trust in their AI systems while promoting responsible innovation across various industries.

About this article

Photographic portrait of Yara Elias
Yara Elias
EY Canada Senior Manager, AI Risk, Risk Consulting
Mathematician with a passion for responsible AI and a penchant for dancing
Related topics
AI
Trust

EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.