As technology advances, will accountability be a casualty?

The potential of automated decision-making remains largely untapped in the financial sector – our risk survey reveals most institutions use machines to make only low- or medium-level decisions. But as their adoption of automation accelerates, the complex workings behind machine-generated decisions may present problems for financial institutions.

The EU’s General Data Protection Regulation (GDPR) grants individuals the “right to an explanation” when an automated decision has had a major impact on them, or to ask for a human to make the decision. But the algorithms and artificial intelligence behind automation are increasingly sophisticated, making it difficult to understand and articulate just how decisions are made.

Building “explainability,” accountability – and trust – into the AI systems they deploy will be critical as financial institutions expand their use. As outlined in a recent EY report, How do you teach AI the value of trust?, this can be done by taking a holistic approach to these systems that considers not just business and technological implications, but also their broader ethical, social, environmental and regulatory impacts across their life cycle (from design to implementation). In this way, firms will understand how the system functions and evolves, as well as clearly define lines of accountability.

Institutions doing this best are deploying robust policies and standards specific to AI development, using validation tools, conducting regular inventories and commissioning independent audits to make certain all AI algorithms are properly governed and perform as intended.

As the financial ecosystem expands, senior managers are urgently reviewing relationships with third-party providers. Key priorities include:

• Creating service contracts with vendors that include clearly defined obligations
• Checking that third parties have appropriate risk controls and governance in place
• Considering requiring vendors to allow audit firms to objectively validate their compliance with risk-control obligations (for example, via SSAE 16 audits and SOC1 reports)

But even as they increase internal efforts, financial institutions surveyed told us that they would like external guidance. Clarity from regulators around the expectations related to third-party risk management of new technologies would help firm up the accountability framework.

For their part, regulators acknowledge the need to update regulatory requirements applying to regulated outsourcing institutions. In its recent report on innovation in the financial sector,⁵ the US Treasury made a number of recommendations, including “… setting clear and appropriately tailored expectations for chain outsourcing,” while the European Banking Authority (EBA) recommendations on outsourcing to the cloud took effect on July 1, 2018.⁶

One of the biggest accountability challenges for regulators around third-party vendors is the connectivity between partners, institutions, sectors and geographies. Understanding and testing how a technology failure or breach at one third-party vendor could impact the wider financial ecosystem is a regulatory priority.

In our perspective, As technology races ahead, are utilities the upgrade you need?, we take a more detailed look at the issues arising from the use of shared services.

But even holding financial institutions accountable for the actions of third-party providers won’t be enough to defend against the growing threat of systemic risk. Consider how quickly the use of cloud-based services have become embedded in the financial services infrastructure – holding a senior manager accountable for any failure does nothing to mitigate systemic risk or financial losses.

Regulators are exploring possible responses. This includes considering whether the scale of operations outsourced to the cloud and/or onward via chain outsourcing requires the zone of accountability to be extended to include infrastructure providers. Some market observers are asking whether regulators should require key infrastructure providers to at least disclose their business continuity plans and maintain a prescribed level of operational capital, as is the case for firms inside the regulatory perimeter.

But extending the regulatory perimeter won’t get senior managers off the hook. Financial institutions will still be obligated to both know and understand those processes and associated risks directly under their control but carried out by a cloud-based provider or a decision-making algorithm.

We see four key areas where new tools can help senior managers enhance their firm’s accountability:

1. Build new frameworks for new technologies

As technology changes the nature of risk and accountability, risk frameworks should expand to include the identification, monitoring and management of potential adverse outcomes of machine-generated decisions. Clarifying and documenting accountability around these, as well as approaches to investigating adverse events and communicating the lessons learned from them, are key elements. Technology is moving lightning fast – and errors can occur, and spread, just as quickly. It’s vital that response mechanisms can keep the pace.

2. Embed accountability in risk control improvements in the 3LoD model 

Our 2018 EY/IIF global bank risk management survey shows that most banks are undergoing an accelerated transformation driven by a technological revolution and highlights several key areas that also make a crucial contribution to the accountability obligation:

• Embedding balanced risk-taking and risk discipline into businesses
• A digital transformation of risk management; enabling risk management through automation, machine learning and artificial intelligence
• The 3LoD model; developing its operation and roles

3. Document third-party processes

As third parties play a bigger role in the operations of financial institutions, documenting responsibilities and implementing contingency planning will become critical tenets of the new accountability mandate. This is not just good practice but essential – the latest European Banking Authority guidelines recommend these records be available to the regulator.

Documenting core processes from end to end, especially when they cross institutional boundaries, helps contain systemic risk by enabling regulators to define accountability for specific process components and show clearly where and when the handoffs between institutions occur. For institutions, this mapping is undoubtedly an onerous task but one that brings benefits in the long term. Senior leaders are discovering the value of closely monitoring the process risks for which they or their firms are accountable and determining how information needs to be shared with other players in the process chain, as well as with regulators.

Many institutions fail to consider the need for change or exit strategies in relationships with third-party vendors. The importance of these can’t be underestimated when outsourcing models are evolving fast and complexities around the use of technologies, the cloud and data lakes are growing.

4. Apply technology to improve accountability

Advancing technologies including the cloud have huge potential to deliver an even greater level of accountability than has been embedded in systems and processes up until now. For example, a recent market study by the UK’s FCA found many direct-to-consumer (D2C) investment platforms currently lack effective best-execution monitoring and may even be noncompliant with basic investor protections. Integrating and enhancing monitoring capabilities could strengthen the integrity of the platform and help management demonstrate greater oversight of the product and how it reinforces positive outcomes for customers.

Cases like this highlight how individual institutions and the sector as a whole should consider how the tangible cost of technology development may well be outweighed by the less tangible benefit of more demonstrable product accountability together with the avoidance of future fines for rule breaches.

Transformation and governance are interconnected

Technological transformation will impact both a financial institution’s operating model and its governance – the two are interconnected. It makes sense then that however a firm progresses its adoption of digital processes to better manage risk, embedding accountability measures that keep pace with change across the business and third-party providers is critical.