Many physicians are concerned about the rising quality of artificial intelligence recommendations for patient diagnosis and treatment. A new study led by Prof. Dan Zeltzer from Tel Aviv University compared AI recommendations with those of physicians at Cedars-Sinai Medical Center in Los Angeles.
Study Methodology
The research examined 461 online clinic visits during the summer of 2024, focusing on adult patients with common symptoms like respiratory, dental, urinary, and vaginal issues. An AI algorithm, trained on millions of medical records, provided initial assessments and recommendations, which were then evaluated by physicians during video consultations.
Key Findings
A panel of four experienced doctors rated both AI and physician recommendations on a four-point scale. The results showed:
- AI recommendations were rated ‘optimal’ in 77% of cases, compared to 67% for physicians
- AI recommendations were rated ‘potentially harmful’ in 2.8% of cases, versus 4.6% for physicians
- In 68% of cases, AI and physician recommendations received the same score
- AI scored higher than physicians in 21% of cases, while physicians scored higher in 11%
Advantages of AI
The study highlighted several advantages of AI over human physicians:
- Stricter adherence to medical guidelines: AI was less likely to prescribe unnecessary antibiotics
- Comprehensive analysis of medical records: AI identified relevant.
- Better symptom identification: AI more accurately detected symptoms indicating serious conditions
Physician Advantages
While AI showed strengths, physicians maintained advantages in:
- Flexibility in assessment: Doctors could consider the patient’s overall condition beyond just the data
- Clinical judgment: Physicians could recognize when AI recommendations were unnecessary or excessive
Future Implications
Prof. Zeltzer concluded that while AI can provide more accurate recommendations than doctors in many cases, its primary role will be supporting physicians rather than replacing them. The study tested AI in a real-world setting, making its findings particularly meaningful for assessing AI’s readiness to assist doctors in practice. Zeltzer predicts that algorithms will increasingly assist in medical decisions, helping doctors make faster and more accurate choices with fewer errors.
The integration of AI in healthcare is expected to enhance care workflows, particularly in systems short on clinicians. While there are concerns about AI’s impact on medical jobs, experts believe that AI will primarily serve as a tool to support healthcare professionals rather than replace them.
Challenges and Future Directions
The study noted limitations, including uncertainty about how physicians used AI recommendations. Future research will need to explore the optimal integration of human expertise and AI in medicine, as well as address concerns about safety, trust, and regulation in AI adoption.
