Assessing and mitigating unfairness in credit models

EY and Microsoft have worked together to explore how the Microsoft Fairlearn toolkit can help financial institutions embed fairness in their lending practices.

The importance of fairness in financial modeling

Financial services organizations play a central role in the financial wellbeing of individuals, communities and businesses. Every day, these organizations make decisions that impact people’s lives, such as approving loan applications, foreclosing on mortgages, or paying out life insurance claims.

In recent years, financial services organizations have begun to use artificial intelligence (AI) and machine learning (ML) in their decision-making processes. With the ability to ingest and analyse vast amounts of data, both structured and unstructured, AI algorithms have the potential to help financial services organizations better meet their customers’ needs.

As AI plays an increasingly prominent role in the financial services industry, organizations must use it responsibly and plan for unintended consequences. When an organization deploys an AI system, there’s the potential to inadvertently deny people services, amplify existing gender or racial biases, or violate laws such as the Canadian Human Rights Act.

To address these kinds of concerns, fairness must be explicitly prioritized throughout the AI development and deployment lifecycle. Transparency and fairness are the most important aspects of a sound and trustworthy AI/ML model.

Here we explore how the Microsoft Fairlearn toolkit can be used to assess and improve the fairness of a ML model for loan adjudication.

How is fairness defined?

Fairness metric quantify the extent to which a model satisfies a given definition of fairness. Fairlearn covers several standard definitions of fairness for binary classification, as well as definitions that are appropriate for regression. These definitions either require parity in model performance (e.g., accuracy rate, error rate, precision, recall) or parity in selection rate (e.g., loan approval rate) between different groups defined in terms of a sensitive feature such as “gender” or “age.” We note that the sensitive feature need not be used as an input to the model, it is only required when evaluating fairness metrics.

The following three definitions are widely used in classification settings where being classified as “positive” results into an allocation of an opportunity or resource (e.g., a loan) and having a positive label in a dataset means that the corresponding individual is “qualified”:

• Demographic parity (independence): Individuals within each group should be classified as positive at equal rates. Equivalently, the selection rates across all groups should be equal.
• Equal opportunity (separation): The qualified individuals in each group should be classified as positive at equal rates. Equivalently, the true-positive rates across all groups should be equal.
• Equalized odds (sufficiency): The qualified individuals within each group should be classified as positive at equal rates; the unqualified individuals within each group should also be classified as positive at equal rates. Equivalently, the true-positive rates across all groups should be equal and the false-positive rates across all groups should be equal.

Fairlearn

Fairlearn is an open-source Python toolkit for assessing and improving the fairness of AI systems. The design of Fairlearn reflects the understanding that there is no single definition of fairness and that prioritizing fairness in AI often means making trade-offs based on competing priorities. Fairlearn enables data scientists and developers to select an appropriate fairness definition, to navigate trade-offs between fairness and model performance, and to select an unfairness mitigation algorithm that fits their needs.

Fairlearn focuses on fairness-related harms that affect groups of people, such as those defined in terms of race, gender, age, or disability status. Fairlearn supports a wide range of fairness definitions for assessing a model’s effects on groups of people, covering both classification and regression tasks. These fairness definitions can be evaluated using an interactive visualization dashboard, which also helps with navigating trade-offs between fairness and model performance. Besides the assessment component, Fairlearn also provides a range of unfairness mitigation algorithms that are appropriate for a wide range of contexts.

Fairlearn offers an interactive visualization dashboard that can help users assess which groups of people might be negatively impacted by a model and compare multiple models in terms of their fairness and performance.

When setting up the dashboard for fairness assessment, the user selects two items:

• The sensitive feature (e.g., gender or age) that will be used to assess the fairness of one or multiple models
• The performance metric (e.g., accuracy rate) that will be used to assess model performance

These selections are then used to generate visualizations of a model’s impacts on groups defined in terms of the sensitive feature (e.g., accuracy rates for “female” and “male” as defined in terms of the gender feature). The dashboard also allows the user to compare the fairness and performance of multiple models, enabling them to navigate trade-offs and find a model that fits their needs.

Unfairness mitigation algorithms

Conclusion

Although we focused on the use of AI in a loan adjudication scenario, AI systems are increasingly used throughout the entire credit lifecycle. After an applicant is approved for a loan, AI systems are often used to determine the loan amount and the interest rate that are offered to the applicant. During the remaining life of a credit product, a financial services organization can also use AI systems for account management based on the account behaviour, credit bureau information, and other customer information. AI systems can support pre-approval of credit-card limit increases or assist with a collection strategy when a customer exhibits difficulty with repaying.

As financial services organizations use AI systems more extensively, prioritizing fairness in AI is fundamental to their success.

Although this collaboration between Microsoft and EY focuses on the use of a software toolkit, we emphasize that fairness in AI is a sociotechnical challenge, so no software toolkit will “solve” fairness in all AI systems. That is not to say that software toolkits cannot play a role in developing fairer AI systems — simply that they need to be precise and targeted, embedded within a holistic risk management framework that considers AI systems’ broader societal context, and supplemented with additional resources and processes.