AI in Healthcare: How about a second opinion?

AI is poised to make a considerable impact on the healthcare field. As one example, generative AI may enable 24/7 healthcare education and wellness coaching, using mobile texting and a wide variety of sensors for fitness tracking and remote patient monitoring. AI may also be used to assist humans in analysis, diagnosis, care plans and prescriptions.

The potential benefits of AI in healthcare included improvements in quality (more data-driven, effective and convenient care) and scalability (more accessible, affordable and distributed care). These are both important to address challenges with healthcare equity and underserved communities.

But there are many concerns about the accuracy of AI in healthcare applications. These include:

• Biases and inequalities in the learning models that enable AI responses;

•  Confabulation, the generation of plausible-sounding but potentially inaccurate information, a characteristic of generative AI language models when they try to respond based on limited or incomplete knowledge; 

• Concerns about lack of quality data, including clinical data in training models;

•  Lack of transparency and explainability (the black box problem).

These concerns may limit the adoption of AI-based solutions in a wide range of areas where they would potentially be valuable.

Frequently, the suggested solution to these concerns is to provide oversight by human healthcare professionals, or prohibit AI applications entirely, particularly for scenarios in which a medical diagnosis is made or intervention prescribed.

However, even in human-only healthcare systems and processes, mistakes can be made. Misdiagnoses and errors in prescriptions and treatment plans are hard to track accurately but are a significant factor in healthcare risks. And it should be noted that here are human equivalents of the same concerns listed for AI platforms. Opinions provided by human experts can sometimes be characterized by:

• Biases and inequalities;

• Responses based on incomplete information or limited time to collect or process information;

• Insufficient clinical data;

• Incomplete or outdated training;

• Difficulties with transparency and explainability.

It should be noted that many of the recent studies of LLM accuracy and appropriateness compare LLM responses to those of a panel of human clinicians. But are clinician responses in this scenario representative of responses that real patients get "in the wild", across a broad range of real-world care delivery environments and ecosystems? In these scenarios, clinicians may face difficult time constraints, get incomplete or contradictory information from patients and systems, and have varying levels of training.

It's possible that statistics on second opinions are relevant. In real-world human-expert healthcare scenarios, the “oversight” function to address these concerns is often provided through second opinions, responses solicited from one or more experts to verify the advice or view of the expert previously consulted. In healthcare applications, a second opinion may include an different diagnosis or treatment plan. The Mayo Clinic, which provides a large number of second opinions or diagnosis confirmations, did a study in 2017 reporting that almost 88% of their consultations result in a new or refined diagnosis. In only 12% of the cases is the original diagnosis confirmed as complete and correct. Although there are a number of mitigating factors, at the least this seems to suggest a higher percentage of variability in clinician responses in real-world scenarios.

We absolutely should be putting AI capabilities under a microscope before they are deployed more widely. Perhaps we can develop a parallel effort to address issues with human-only systems. Would AI would be useful in that effort? Would a "second opinion model" be one approach?

In real-world scenarios, arranging for a second opinion can be difficult for healthcare consumers. Sometimes the first expert is offended and/or unwilling to share information. In a tightly controlled hierarchy of supervision, e.g., within a siloed healthcare provider system, it may be hard to get permission or be reimbursed. In underserved communities, it may be even more challenging.

Getting one informed opinion is hard; getting a second opinion may be unaffordable and virtually impossible. In the current healthcare ecosystem, second opinions are a luxury.

Both AI-only and human-only systems of experts have challenges with complete accuracy. Both AI and human experts have strengths and weaknesses when it comes to analyzing data and rendering opinions. What if we could leverage networks of both types using some form of second-opinion structure to lower risks, enhance quality and reduce errors, while improving efficiency and lowering costs.

HEALTH & SECOND

Let’s imagine a fictional system/service providing communities with an integrated human + AI network that facilitates second opinions, connecting across both types of knowledge experts. We’ll call it Health & Second.

There are a number of services facilitating human second opinions, often through telehealth, and various schemes for human oversight and top-down regulation of AI applications. Health & Second takes a somewhat different bottom-up approach, using networks of both types of experts to check for the possibility of alternate diagnoses and opinions. It supports four possible variations in matchups between first and second opinions.

The system itself is fairly straightforward. Health & Second, working with existing EMR or blockchain-enabled personal health data platforms, archives or facilitates on-demand access to sufficient data on the patient to inform the opinions, while adhering to patient data security rules. This may include:

• Previous opinions

• Recent testing, diagnoses, care plans or medications

• Social determinants of health (SDoH) or other relevant personal, social, or care-related data

• Recent RPM or wearables data

• EMR archives

• Any advanced data which has been used for first opinions, e.g. genetic and genomic data.

The system allows the patient to approve access to any, all, or a subset of the data above, for each second opinion requested.

At the same time, Health & Second maintains a directory of available second opinion experts. These are tagged and categorized according to relevant categories, including:

• Human or AI;

• Healthcare specialty;

• Background and training;

• Other characteristics if relevant, e.g., AIs which have been trained with different models or requirements;

• Ratings or metrics based on other second opinions rendered;

• Availability, including any special procedures required to utilize the experts based on their platform, group or organization, as well as the client’s payer, provider, etc.

•  Cost, reimbursement status, etc.

Patients are able to set search filters on the database of experts if desired, so they are presented with a limited subset of the whole network. They can then select one or more experts and initiate the request for a second opinion.

Health & Second automates the dialogue with the expert, including any group or provider-specific procedures and requirements. It negotiates data sharing, subject to the access rules approved by the patient, and may enable the expert to ask questions or request more data and tests.

The system facilitates billing between user and expert, or between user and an existing payment or reimbursement process that covers the cost of the expert. It records the entire process and archives second opinions for later access by the user, or for sharing with other experts.

Although a version of these capabilities could certainly be built into the systems of a large healthcare provider or big-tech virtual health platform, there may be value in an independent service separate from existing provider hierarchies and proprietary ecosystems.

A possibly interesting note: the second opinion model may give us a structure for leveraging a variety of AI platforms, with different training and parameters, to check each other's responses even before a recommendation is made.

EXAMPLES

• A patient at a small rural hospital has been found to have a small sarcoma, an uncommon cancer category of soft tissues. The general pathologist, who is caring but overworked, identifies the subtype as not being aggressive and schedules a follow up in several months. The patient registers with Health & Second, a service that connects users to second opinions, including both human and AI experts. The patient applies for an inexpensive AI review of the data. The AI expert flags the sarcoma as likely to be a more aggressive subtype. The patient reports this to the general pathologist, who consults a specialist at a large cancer center; the patient is scheduled for a surgery.

• A senior living alone wants to exercise regularly, using a fitness wearable, and is beginning to work with an AI-enabled coaching app. The app is recommending an exercise plan including much more intensive workouts, and his daughter is getting a little worried. She registers him with Health & Second, asking for a second opinion from a human fitness specialist. The specialist notes several health issues that haven’t been entered into the senior’s health profile for the AI app. Once corrected, the AI-generated exercise plan is gentler and more conservative.

BENEFITS

So what's the value proposition (for individuals and communities) in a service like Health & Second? Integrating AI and human expert networks into a convenient and affordable independent system for providing second opinions has a number of critical benefits:

• Providing potentially more accurate information, diagnoses and recommendations;

• Allowing AI to back up human healthcare professionals, and human professionals to verify AI-enabled healthcare services;

• Encouraging the adoption of AI as a one potential source of healthcare expertise, particularly for underserved communities;

• Giving patients and caregivers an alternative in scenarios where their provider system is limited, siloed, biased or slow to respond;

• Providing an extra measure of peace of mind to individuals and community members who must navigate a complicated healthcare system with many resource constraints, in order to address a wide range of complex health conditions.

CAVEATS

There are some enormous caveats in this scenario. One of the biggest has to do the privacy and security of data, and a level of interoperability and ease of access that just doesn't exist yet.

At the same time, we may not want to trust the current generation of AI platforms, even with the level of oversight that we imagine being provided by the platform, before their accuracy has been vetted to standards comparable with the best human clinicians.

Still, we should not think about the problems we have to address (bias, misdiagnoses, transparency, etc.) as being confined to AI alone. We are experiencing a critical shortage of healthcare professionals, and this will only increase the pressure on human knowledge systems, as we struggle to improve and extend care to underserved communities around the world.