Gen AI Keyboard

How evolution of AI in testing can become a revolution

Over the next five years, AI in software testing will morph from an enabler to a transformative, critical component of delivery.

In brief 

  • AI has the potential to fundamentally change software testing. 
  • At present, AI is helping to accelerate and improve existing processes, enabling testers to do more and do it quicker.
  • Before AI can intrinsically change software testing, there are roadblocks to overcome.

Rapid digital transformation continues to weave technology deep into the fabric of our day-to-day lives. As innovation escalates, so does our reliance on software: We already use it to monitor everything from the quality of our drinking water and reliability of our power grids, to the health of our hearts. Validating the quality and testing of that software has never been more important. Unfortunately, while the world has undergone an explosion of digital innovation, such as cloud-based core banking platforms, AI-enabled assistants and event-driven architectures, software testing has lagged.

How EY can help

That stagnation is not for lack of trying. In fact, there have been several attempts to reimagine testing and automation: Behavior-driven development methodologies and related testing frameworks aimed to rethink the entire development process, from requirements through testing; DevOps processes aimed to fully automate and streamline the entire build and deployment process; and agile and scrum aimed to embed testing into delivery by breaking down silos between roles and shift toward a product-quality mindset. They all fundamentally shift how testing is thought about and succeeded in quickly spreading adoption; however, in most large organizations, these were only partially adopted.
 

While application and infrastructure technology, such as cloud-native technologies and low-code and no-code platforms, have moved at a breakneck pace, testing has been the weakest link, holding back products from getting to end users quickly. If testing can’t keep up with development and infrastructure deployment, the whole production cycle slows down. Herein lies the opportunity to better integrate artificial intelligence (AI) in software testing. First, we’ll examine the current state of testing, followed by roadblocks of AI implementation and future opportunities for AI in testing. Finally, we’ll feature what organizations can do now to boost testing capabilities.

Fingers on keyboard
1

Chapter

The current state of AI in software testing

Technology in testing has stagnated and fallen behind new development and architecture practices.

Currently in software testing, AI is applied mostly in limited, isolated ways that don’t take full advantage of its potential. Companies have confined AI to low-stakes use cases to perform isolated tasks, such as self-healing object locators or free-text test design using natural language processing (NLP), because they are easy, low-hanging fruit that companies can quickly implement. While using AI in this way may help to expedite parts of the process, it falls short of potential benefits that it could bring.

To illustrate this progressive spectrum, we’ve detailed the levels of AI in testing and the point at which it brings about a fundamental change in how testing is done.

Select each image below to learn more:

From Level 1 to Level 2, AI adoption in testing incrementally increases the productivity and possibilities of what testers can do, but the fundamental processes, methodologies and activities are no different. In these levels, AI processes assist humans in accomplishing the same tasks but faster or more efficiently.

Level 3 is the threshold to reimagining how testing is done. Entire parts of the testing process are changed or replaced, and the foundational aspects can be reimagined for better and faster outcomes, thanks to AI. Bots and AI testing workforces become crews working with humans in the testing process. The goal is not to make every process entirely autonomous — some processes can’t or shouldn’t be automated. But when the right processes are automated, human resources are freed up to work on high-value testing tasks, promoting faster development and speed to market.

Women purple background
2

Chapter

Roadblocks to AI testing solutions

Trust, nuanced knowledge, and engineering and support have been barriers to boosting AI in testing.

With so much hype and potential, what’s preventing testing professionals from adopting AI to improve testing capabilities? Aside from just understanding where to start, there are many barriers to adopting AI in more impactful ways. Some significant roadblocks include:

Technology alone cannot fundamentally shift how testing is done. How you use the technology ultimately enables the desired change. Changes in processes, approaches, methodologies and the culture are changes technology cannot make. But the continued acceleration of digital transformation suggests that the greatest risk lies in the decision not to adopt AI tools.

Man in server room
3

Chapter

Future opportunities

As maturity improves, the role of AI becomes more elevated, autonomous, integrated and impactful.

As AI gains ground in testing, software quality will improve incrementally, with testers using the technology in isolated, targeted ways to improve parts of the process. Over the next few years, AI will cross into transformation, fundamentally changing the way software testing is done. Instead of merely supplementing human efforts, AI will work autonomously, freeing testers to make more high-value contributions. Virtually every area of testing will see significant, rapid evolution.

Here are some of the key opportunities we see in the near term for AI to have a significant impact in testing:  

  • Proactively identifying gaps in requirements and acceptance criteria before development or testing even starts 
  • Analyzing and quantifying test coverage against requirements or stories 
  • Performing change impact analysis to isolate:
    • What’s changing
    • How it’s changing
    • The most efficient way to test it
  • Reconciling regression test suites and optimizing coverage and variations 
  • Generating automated tests from manual ones 
  • Autonomously identifying or generating test data to use for testing 
  • Completing early testing cycles using an AI-driven virtual testing workforce to:
    • Take a first pass at test design and execution
    • Report back results or blockers to a human for further action

Using the maturity levels of tech and AI in testing from chapter 1, below we show the time frame for how we believe most organizations will continue to evolve over the next several years toward more transformative use cases for AI in testing.

This transformation will touch every stage of testing. Select each image below to learn more:

AI will help humans with testing tasks they overlook today, fundamentally changing testing in the future.

Women using phone
4

Chapter

Four things you can do now

Organizations can prepare now for powerful advancements in the future.

We’ve explored the current state of AI in testing and some of the roadblocks that keep software testers from fully optimizing AI. Abundant, groundbreaking opportunities await. But what does this mean for testing right now?

AI isn’t yet a mainstream part of enterprise testing, but if you wait too long, you’ll fall behind. Here’s what you can do now, select each image below to learn more:

 

Tip

One of the reasons that AI is such a hot topic is the low barrier to entry. You don’t need to be a data scientist or build models from scratch anymore — and you shouldn’t. Avoid the technical debt and use well-adopted and scalable open-source or cloud technology to accelerate and build momentum.

 

Summary

AI shows incredible potential for the future of software testing. While its current applications are limited, over the next several years, AI can fundamentally change testing and, ultimately, the speed of product releases. To effectively adopt AI in testing, organizations should prepare their data and processes now to not get left behind as the tech evolves in the future.

Related articles

Driving confidence in AI: FS perspective on Responsible AI

In this webcast, panelists will explore and define how financial services institutions can take a balanced risk management approach in adopting GenAI.

13 Dec 2023 | 16:30 your local time

Five priorities for harnessing the power of GenAI in banking

For banks with the right strategy, talent and technology, GenAI can transform operations and help reimagine future business models. Learn more.

28 Nov 2023 Jan Bellens + 1

    EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.