Most AI solutions combine elements of rule-based and nondeterministic logic. For example, robotic process automation (RPA) solutions have been implemented by several organizations to automate business processes that can be defined as fixed sequences or decision trees. To expand the capability of RPA to cover decisions that involve judgment, organizations are combining RPA, cognitive computing and machine learning technologies into intelligent automation (IA) — solutions that learn through training. That training yields powerful capabilities, but the process of training AI solutions is fraught with costly traps.
Chief among these is the tendency to underestimate the time and effort required to train — and periodically retrain — AI solutions. Much of the development work that has traditionally been done by IT is now shifting to people across the business; specifically, to subject-matter experts, who may have limited capacity to spare for these new responsibilities. For example, in a recent project with an EY client, almost 40% of the client organization’s time was spent by trainers asking questions and rating the answers of a cognitive system. That is important work, but it can represent a significant cost to the business.
Three questions can help organizations design an efficient training process for AI solutions.
1. How much expertise is required?
Identifying the level of business expertise that will be required for AI solution “training” enables the team to build the appropriate scope of effort and schedule right into the plan, thus helping to ensure that expertise will be available when required. This mapping effort can also flag potential gaps, especially in the time available from the appropriate domain experts. The organization can then assess whether to engage consultants to train AI solutions fully or partially, a tactic that can help reduce the demands on staff and improve the performance of the solution by accessing expertise that is not yet available inside the organization.
Another key consideration is ensuring that AI training teams can tap into the software engineering skills necessary for successful AI coding. Most AI projects will eventually encounter software performance issues, largely because the domain experts are not likely to also have the software engineering expertise to accurately code the algorithms and business rules applied within the AI solution. Ensuring that the solution training process includes qualified software engineers will reduce the need for costly fixes down the road.
2. Are there more efficient ways to train the solution?
Some AI solutions can be trained in “batches,” using preexisting data. This requires a data set that includes the correct output given a predefined set of inputs; for example, a simple table of inputs and the corresponding decision of the expert. Another example is a “training query set,” which is a list of queries that represent common questions and answers. The AI solution can use this query set as a starting point for its learning. Another route to limiting overall training effort is to purchase a pre-trained solution. Increasingly, AI software vendors offer products that have been pre-trained using industry and domain knowledge.
For example, a risk management and regulatory compliance firm used pre-trained solutions to identify and address common regulatory citations. In another case, a large software firm used this approach to build domains for its AI suite and associated applications. In addition to the reduction in time and cost that pre-training solutions can deliver, some even include the ability to access additional industry data that can enhance organizational data for richer insights.
3. What is the optimal training effort to generate the best impact?
How much is too much? Organizations will want to identify the optimal amount of training that equips the AI solution to generate the best results. After all, overtraining not only wastes effort but also risks “overfitting” — that is, developing a solution that is too finely tuned to the cases in the training data set, losing valuable generality when applied to broader data. A good approach is to first define a benchmark to measure the performance of the solution.
For example, a client was able to compare the customer satisfaction ratings for interactions with a chatbot against those with call center staff after each training cycle, which yielded important insights that helped to shape the chatbot training. Another client used a customer control group to measure the sales uplift attributable to a next-best-offer solution. One important yet often overlooked consideration is the time required to periodically retrain AI solutions. As the environment in which the solution operates evolves, its performance may suffer. Monitoring and periodic retraining can ensure continued relevance. For example, the algorithms that predict customer behavior may need to be retrained as customers become conditioned by repeated messages. Similarly, training query sets should be regularly updated to reflect new information.