Analytics and intuition: Why health care big data needs both

Technology may be advancing, but it’s still human instinct that counts when digging for the insights that can improve performance.

Across every industry, and in all walks of life, we are generating more and more data. This data serves as a potentially rich source of insight that can inform decisions and actions. But only if it is analyzed correctly.

Many organizations have reached a point where their ability to generate data exceeds their ability to consume and analyze that information. They have built capacity for analytics production, but not insight.

Leaders acknowledge this disconnect. In an EY survey of senior executives, 81% of respondents agreed that data should be at the heart of decision-making. But only 31% said they had significantly restructured their operations to incorporate analytics.

Combining analytics and intuition: learning from health care

It is arguable that the sector where data can provide the greatest impact – both in terms of business performance and the benefit to humankind – is health care. Here, the ability to recognize and act upon the right data can genuinely make the difference between life and death.

In recent years, the digitization of medical records by state public health bodies and research by medical institutions and the pharmaceutical industry has begun to pave the way for the widespread use of analytics in health care.

Now, more medical information is being generated and gathered today than ever before: electronic records, health insurance claims, real-time monitoring by connected mobile devices, plus ever more creative ways to gather and combine data. It’s a trend set to continue.

With aging populations putting increasing pressure on health services, and health care costs rising worldwide, working out how to make health care more efficient is a pressing concern.

But analytics – enabling the generation of insights into the way people live their lives – also presents a great opportunity to inject new, creative and effective thinking into health care.

When analytics is not enough

The power of analytics lies in its capacity to find patterns among vast swathes of data quickly. But it’s not always accurate at interpreting those patterns and making the right diagnosis. The binary code of analytical algorithms still lacks the intuitive capability to guess at the motivations driving certain human behaviors.

Perhaps the best-known example of a failure in algorithmic health care analytics is Google Flu Trends. When it first appeared, its ability to spot flu outbreaks weeks ahead of traditional methods was hailed as a breakthrough.

But for the 2013 flu season, the tool’s predictions were widely off the mark. Why? The algorithm didn’t distinguish between someone who had symptoms and someone merely asking about them. With such a vast reservoir of data to draw upon, the volume of misleading or false data became so great as to render the findings almost meaningless.

The idea behind Google Flu Trends was a good one, but its correlation-based insights were too simplistic. Its binary thinking lacked the creative latitude to consider alternative human behaviors that were actually driving the data-set. Today, experiences like this are helping to improve medical analytics.

What it cannot usually do is relate those patterns to real-world scenarios and contexts – the sheer amount of contextual and experiential data this would require is still far beyond even the most advanced Artificial Intelligence algorithms. Although artificial intelligence may be improving, algorithms can only take us so far. Strings of code can help us identify trends in data but still aren’t smart enough to derive true insight on their own.

This is why intuition and creativity remain firmly within the realms of humans – combining quantitative data with qualitative insight that can lead to answers that algorithms could never reach.

Analytics are at their best when they complement these very human traits to help draw more timely conclusions from those patterns, ultimately improving patient outcomes by helping humans reach the right conclusions faster.