Digital Health Adviser. AI in Healthcare realist. Anesthesiologist and Intensive Care Physician.
The important milestone of medical AI development was highlighted by Eric Topol, MD in his Ground Truth publication "When Medical A.I. is Lifesaving. Counterbalanced by some disappointments". At the end of April, research group from Taiwan published a paper describing results of a multisite randomized controlled trial evaluating the ability of an AI-enabled electrocardiogram analysis tool to identify hospitalized patients with a high risk of mortality. The study was appeared in Nature Medicine and included 39 physicians and 15,965 patients. Electrocardiograms of patients in interventional arm were analyzed with AI and physicians received alert with report and warning messages, flagging patients predicted to be at high risk of mortality. The control arm included patients whose ECGs were analyzed by conventional method (by a physician with no AI involvement). The implementation of the AI-ECG alert was associated with a significant reduction (17%) of all-cause mortality within 90 days and saving 7 lives per 100 in some groups. As Dr. Topol pointed out, the results are comparable to most successful medical treatments, such as statins. Notably, it was the first-ever RCT to estimate the effect of AI-based intervention on patients' mortality. However, the results of four trials of real world scenarios of Generative AI use, as described in the same Ground Truth publication, were not so promising. Large Language Models (including Chat GPT 4) were unsuccessful in answering oncology questions and assisting in medical coding. They also failed to reduce clinicians’ time spent writing reply messages to patients (though they did generate more empathetic replies) and posed a risk of severe harm in 7% of generated messages which is definitely not acceptable. Surely, as Dr. Topol stated “Progress in medical A.I. won’t occur in a straight line”. It has been repeatedly confirmed that AI is far better than humans at analyzing repeated patterns not generated by humans, such as MRI or ECG. However, when it comes to working with more abstract entities, such as diagnoses or clinical messages, its effectiveness is much more modest. In any case, I strongly encourage you to refer to the original Ground Truth publication for more details - https://lnkd.in/dXaEAq6m
Healthcare Data Scientist | Researcher | Passionate about pediatrics and parenting
10moAndrej, thank you for this summary. Of course, thanks to Dr. Topol for his promotion of AI in medicine. Yes, based on the available knowledge, at the moment AI is already doing quite well in tasks where there are more or less clear evaluation criteria. And further on it will be observed more and more clearly, as I think. But in those circumstances that will require a non-trivial approach, there will probably be problems in applying AI. Although it's worth recognizing that it's not easy for humans to solve such problems either. Also, at this point it is obvious that AI requires very careful preparation of the prompt, which is not easy in itself. And requires separate training. It doesn't seem to be very convenient. Obviously, the more context AI has, the better its answers will be. The results of AI implementation in the healthcare show that AI models need to be further trained in specific application contexts, including the peculiarities of a particular physician's work.