Over the past year, studies have highlighted how artificial intelligence models in healthcare sometimes can increase bias. Now, a new study warns that clinicians could follow AI down the wrong path.
When reviewing data for respiratory failure, a common problem for older adults, clinicians were 11% less accurate in their diagnoses when using a biased AI model than if they didn’t use AI at all, the study found.
Disturbingly, this fact was true even when given explanations about how the AI came to a diagnostic conclusion.
For instance, the study results showed that even when clinicians were given information that showed that AI was making spurious connections between older adults and the likelihood of pneumonia, they didn’t spot the issue and adjust.
Although not all senior living and care settings have on-site clinicians, caregivers still are tasked with reviewing medical data and carrying out treatment options. Thus, the concern of being fooled, instead of aided, by AI-driven recommendations is a high-risk proposition.
The problem stems from AI models that are trained on biased data sets, which are skewed to lack racial or ethnic sensitivity. Because the AI does not “know” that it is being trained on biased healthcare data, those tools mimic and exacerbate these problems, frequently offering poor predictions depending on one’s background.
The study builds off of previous work on such biases with the ominous finding that offering transparent explanations for how the AI’s conclusions were generated did not help clinicians identify when the AI was compromised.
The study was not entirely negative toward using AI: The results reaffirmed that when accurate AI-enabled diagnostic tools were used as aids, clinicians improved their accuracy by 4%.
Overall, older adults are showing increasing awareness, and possibly even support, for using AI tools as part of their healthcare, a recent report found.
New tools also are being developed to help vet AI programs and identify which models have proven to be accurate and effective, as the McKnight’s Tech Daily has reported.
