Day: April 26, 2024

When it Comes to Healthcare, AI Still Needs Human Supervision

Photo by Tara Winstead on Pexels

State-of-the-art artificial intelligence systems known as large language models (LLMs) are poor medical coders, according to researchers at the Icahn School of Medicine at Mount Sinai. Their study, published in NEJM AI, emphasises the necessity for refinement and validation of these technologies before considering clinical implementation.

The study extracted a list of more than 27 000 unique diagnosis and procedure codes from 12 months of routine care in the Mount Sinai Health System, while excluding identifiable patient data. Using the description for each code, the researchers prompted models from OpenAI, Google, and Meta to output the most accurate medical codes. The generated codes were compared with the original codes and errors were analysed for any patterns.

The investigators reported that all of the studied large language models, including GPT-4, GPT-3.5, Gemini-pro, and Llama-2-70b, showed limited accuracy (below 50%) in reproducing the original medical codes, highlighting a significant gap in their usefulness for medical coding. GPT-4 demonstrated the best performance, with the highest exact match rates for ICD-9-CM (45.9%), ICD-10-CM (33.9%), and CPT codes (49.8%).

GPT-4 also produced the highest proportion of incorrectly generated codes that still conveyed the correct meaning. For example, when given the ICD-9-CM description “nodular prostate without urinary obstruction,” GPT-4 generated a code for “nodular prostate,” showcasing its comparatively nuanced understanding of medical terminology. However, even considering these technically correct codes, an unacceptably large number of errors remained.

The next best-performing model, GPT-3.5, had the greatest tendency toward being vague. It had the highest proportion of incorrectly generated codes that were accurate but more general in nature compared to the precise codes. In this case, when provided with the ICD-9-CM description “unspecified adverse effect of anesthesia,” GPT-3.5 generated a code for “other specified adverse effects, not elsewhere classified.”

“Our findings underscore the critical need for rigorous evaluation and refinement before deploying AI technologies in sensitive operational areas like medical coding,” says study corresponding author Ali Soroush, MD, MS, Assistant Professor of Data-Driven and Digital Medicine (D3M), and Medicine (Gastroenterology), at Icahn Mount Sinai. “While AI holds great potential, it must be approached with caution and ongoing development to ensure its reliability and efficacy in health care.”

Source: The Mount Sinai Hospital / Mount Sinai School of Medicine

Acid-lowering Meds Linked to Greater Risk of Migraines

Photo by Usman Yousaf on Unsplash

People who take acid-reducing drugs may have a higher risk of migraine and other severe headache than people who do not take these medications, a new study has shown. The acid-reducing drugs include proton pump inhibitors such as omeprazole and esomeprazole, histamine H2-receptor antagonists, or H2 blockers, such as cimetidine and famotidine, and antacid supplements.

The study, study published in Neurology®Clinical Practice, an official journal of the American Academy of Neurology, does not prove causation; only an association.

In acid reflux, stomach acid flows into the oesophagus, usually after a meal or when lying down, causing heartburn and ulcers. People with frequent acid reflux may develop gastroesophageal reflux disease, or GORD, which can lead to cancer of the oesophagus.

“Given the wide usage of acid-reducing drugs and these potential implications with migraine, these results warrant further investigation,” said study author Margaret Slavin, PhD, RDN, of the University of Maryland in College Park. “These drugs are often considered to be overprescribed, and new research has shown other risks tied to long-term use of proton pump inhibitors, such as an increased risk of dementia.”

For the study, researchers looked at data on 11,818 people who provided information on use of acid-reducing drugs and whether they had migraine or severe headache in the past three months.

A total of 25% of participants taking proton pump inhibitors had migraine or severe headache, compared to 19% of those who were not taking the drugs. A total of 25% of those taking H2 blockers had severe headache, compared to 20% of those who were not taking those drugs. And 22% of those taking antacid supplements had severe headache, compared to 20% of those not taking antacids.

When researchers adjusted for other factors that could affect the risk of migraine, such as age, sex and use of caffeine and alcohol, they found that people taking proton pump inhibitors were 70% more likely to have migraine than people not taking proton pump inhibitors. Those taking H2 blockers were 40% more likely and those taking antacid supplements were 30% more likely.

“It’s important to note that many people do need acid-reducing medications to manage acid reflux or other conditions, and people with migraine or severe headache who are taking these drugs or supplements should talk with their doctors about whether they should continue,” Slavin said.

Slavin noted that the study looked only at prescription drugs. Some of the drugs became available for over-the-counter use at non-prescription strength during the study period, but use of these over-the-counter drugs was not included in this study.

Other studies have shown that people with gastrointestinal conditions may be more likely to have migraine, but Slavin said that relationship is not likely to fully explain the tie between acid-reducing drugs and migraine found in the study.

A limitation of the study is that a small number of people were taking the drugs, especially the H2 blockers.

Source: American Academy of Neurology

In Anorexia Nervosa, What Triggers Wilful Starvation?

Photo from Freepik.

A new study led by investigators at Beth Israel Deaconess Medical Center (BIDMC) suggests that female mice that are prone to anxiety may prefer and actively seek out a starvation-like state in response to repeated exposure to stress. The findings, published in the journal Neuron, may provide a useful experimental model for investigating the neural mechanisms underlying anorexia nervosa – particularly its onset.

“While anorexia nervosa has been documented for over 300 years, its underlying causes remain unknown,” said first author Hakan Kucukdereli, PhD, of the division of Endocrinology, Diabetes and Metabolism in the Department of Medicine at BIDMC.

“Current animal models fail to capture a key hallmark of the disorder – wilful starvation. Thus, there has been the pressing need for a pre-clinical mouse model that captures the intentional seeking of a starvation state.”

In healthy individuals, the state of hunger (or caloric deficit) is a mildly uncomfortable state that drives food-seeking behavior. In the lab, Kucukdereli, senior author Mark L. Andermann, and colleagues knew that precise stimulation of a few thousand neurons known as AgRP neurons will cause even a well-fed mouse to seek out another meal.

They also knew that actual food restriction – which activates these AgRP neurons – and the artificial starvation state caused by stimulating these neurons can tamp down anxiety, thereby promoting food-seeking. (Imagine a hungry mouse in your kitchen that needs to be bold enough to hunt for food, even when your cat is around.)

Based on prior associations between stress, anxiety, and anorexia nervosa, Andermann and colleagues hypothesised that exposure to high levels of stress may actually trigger individuals to wilfully seek starvation as a means of reducing anxiety. The scientists trained 15 male and 17 female mice to run through a virtual reality corridor where they could choose to stop in one room associated with stimulation of their AgRP neurons or a second room associated with no stimulation.

In the absences of stress, male mice avoided AgRP stimulation; however, only a minority of female mice exhibited a strong aversion to it. Subsequent to repeated stress, however, many of these same mice behaved very differently. When the researchers exposed the mice to a five-minute period of unpredictable tail shocks, the males became, on average, less averse to AgRP stimulation. Meanwhile, female mice tended to preferAgRP stimulation following stress.

“Strikingly, a subset of females, but not males, began to vigorously seek this starvation-like state following stress,” said Andermann, who is also a professor of Medicine and Neurobiology at Harvard Medical School. “Surprisingly, individuals’ baseline levels of anxiety-like behavior measured weeks before the experiment could predict which females will develop a preference for this starvation-like state.”

Using machine learning to analyse the animals’ facial expressions, the researchers found that, after exposure to stress, female mice with strong preference for AgRP stimulation also showed facial expressions that directly correlated with their behaviour, potentially reflecting relief associated with a reduction in anxiety.

“Future research can link these moment-to-moment changes in facial expressions with ongoing activity of many neurons in brain regions that track physiological states or that process negative emotions,” Kucukdereli said. “Our approach lays the groundwork for future work that will identify the neural circuits that underlie the voluntary maintenance of long-term starvation in individuals with anorexia nervosa.”

Source: Beth Israel Deaconess Medical Center

Trial with Warning Letters on Antipsychotic’s Risks Safely Cut Prescribing

Photo by Towfiqu Barbhuiya on Unsplash

Warning letters from Medicare can safely cut prescribing of a powerful but risky antipsychotic, according to a new study published in JAMA Network Open. Researchers used data from Medicare, the U.S. public health insurance system for over 65s, to study the effects of the letters on hundreds of thousands of older adults with dementia. The study, by Columbia University Mailman School of Public Health, found a significant and lasting reduction in prescribing but no signs of adverse effects on patient health.

“Our study shows that low-cost letter interventions can safely reduce antipsychotic prescribing to patients with dementia,” said Adam Sacarny, PhD, assistant professor of Health Policy and Management at Columbia Mailman School. The work was conducted with researchers at the London School of Economics, Harvard T.H. Chan School of Public Health and Johns Hopkins University.

The researchers evaluated a large trial in which Medicare sent warning letters to high prescribers of quetiapine, the most popular antipsychotic in the U.S. Antipsychotics are frequently prescribed to people with dementia, but can cause numerous harms in this group. Researchers therefore studied the hundreds of thousands of older adults with dementia who were treated by the prescribers in the trial. Most previous studies on reducing prescribing in dementia care consisted of small trials or observational analyses, with limited evidence from large-scale randomised studies.

The results were striking. “People with dementia living in nursing homes and in the community were prescribed less and we did not detect negative health impacts for these groups,” said Michelle Harnisch, research student at the London School of Economics and first author of the study.

The findings are important because antipsychotics, such as quetiapine, are often used in dementia care to address behavioural symptoms. About 1 in 7 nursing home residents receives an antipsychotic every quarter. However, the drugs have a number of well-known risks. These include weight gain, cognitive decline, falls, and death. In turn, physician specialty societies, government regulators, and policymakers have aimed to reduce prescribing of these medications to people with dementia.

To test whether the warning letters reduced prescribing safely, the researchers used administrative data from Medicare to link the 5055 physicians in the original trial to the Medicare records of their patients with dementia. They ultimately analysed 84 881 patients in nursing homes and 261,288 patients living in the community.

The intervention reduced quetiapine use among nursing home patients by 7% and community-dwelling patients by 15%. The researchers did not find adverse effects across numerous health outcomes, including cognitive function, behavioural symptoms, depression, or metabolic outcomes like diabetes. There were signs of improved mental health outcomes, and the risk of death for patients living in the community fell slightly.

This research follows up on the original evaluation of the warning letters. In that study, members of the same research team also showed that the letters reduced prescribing. However, they focused on a considerably smaller sample of patients and studied a limited set of health outcomes. In contrast, the new research evaluates a number of key health indicators for dementia care and substantially expands the patient sample with a focus on dementia.

“These results show that this intervention and others like it could be leveraged to make prescribing safer and improve dementia care” noted Sacarny. “Similar interventions could also be adapted to other contexts to promote high-quality care.”

Source: Columbia University’s Mailman School of Public Health

How Spinal Cords can ‘Learn’ without Brain Involvement

In this study, spinal cords that associated limb position with an unpleasant experience learned to reposition the limb after only 10 minutes, and retained a memory the next day. Spinal cords that received random unpleasantness did not learn. Credit: RIKEN

Researchers in Japan have discovered the neural circuitry in the spinal cord that allows brain-independent motor learning. This study by Aya Takeoka at the RIKEN Center for Brain Science and colleagues found two critical groups of spinal cord neurons, one necessary for new adaptive learning, and another for recalling adaptations once they have been learned. The findings, published in Science, could help scientists develop ways to assist motor recovery after spinal cord injury.

It has been long been known that motor output from the spinal cord can be adjusted through practice even without a brain. This has been shown most dramatically in headless insects, whose legs can still be trained to avoid external cues. Until now, no one has figured out exactly how this is possible, and without this understanding, the phenomenon is not much more than a quirky fact. As Takeoka explains, “Gaining insights into the underlying mechanism is essential if we want to understand the foundations of movement automaticity in healthy people and use this knowledge to improve recovery after spinal cord injury.”

Before jumping into the neural circuitry, the researchers first developed an experimental setup that allowed them to study mouse spinal cord adaptation, both learning and recall, without input from the brain. Each test had an experimental mouse and a control mouse whose hindlegs dangled freely. If the experimental mouse’s hindleg drooped down too much it was electrically stimulated, emulating something a mouse would want to avoid. The control mouse received the same stimulation at the same time, but not linked to its own hindleg position.

After just 10 minutes, they observed motor learning only in the experimental mice; their legs remained high up, avoiding any electrical stimulation. This result showed that the spinal cord can associate an unpleasant feeling with leg position and adapt its motor output so that the leg avoids the unpleasant feeling, all without any need for a brain. Twenty-four hours later, they repeated the 10-minute test but reversed the experimental and control mice. The original experimental mice still kept their legs up, indicating that the spinal cord retained a memory of the past experience, which interfered with new learning.

Having thus established both immediate learning, as well as memory, in the spinal cord, the team then set out to examine the neural circuitry that makes both possible. They used six types of transgenic mice, each with a different set of spinal neurons disabled, and tested them for motor learning and learning reversal. They found that mice hindlimbs did not adapt to avoid the electrical shocks after neurons toward the top of the spinal cord were disabled, particularly those that express the gene Ptf1a.

When they examined the mice during learning reversal, they found that silencing the Ptf1a-expressing neurons had no effect. Instead, a group of neurons in the ventral part of the spinal cord that express the En1 gene was critical. When these neurons were silenced the day after learning avoidance, the spinal cords acted as if they had never learned anything. The researchers also assessed memory recall on the second day by repeating the initial learning conditions. They found that in wildtype mice, hindlimbs stabilised to reach the avoidance position faster than they did on the first day, indicating recall. Exciting the En1 neurons during recall increased this speed by 80%, indicating enhanced motor recall.

“Not only do these results challenge the prevailing notion that motor learning and memory are solely confined to brain circuits,” says Takeoka, “but we showed that we could manipulate spinal cord motor recall, which has implications for therapies designed to improve recovery after spinal cord damage.”

Source: RIKEN