How does AI-assisted mammography stack up?
Volume 8, Issue 13 | September 13, 2023
In This Issue
Draft FDA updates to 510(k) guidance
Questions about cancer-screening longevity study
Proposed update to Alzheimer’s clinical criteria
Sepsis Awareness Month is clearly needed
New and Noteworthy
The clearer, more transparent, more modern FDA 510(k)
Last week, the FDA released three draft guidance documents aimed at providing additional clarity and transparency for its 510(k) approval process for medical devices (including in-vitro diagnostics), as well as continuing to modernize that process. The 510(k) regulatory route is the process used by manufacturers to show that their new product is “substantially similar” to an existing equivalent product (predicate device) that’s already on the market. The drafts are focused on the following three topics:
Selection of predicate devices (Cheat Sheet: Establishes a “valid predicate device” standard, focused on devices that have the same intended use as the proposed product and have no safety issues)
Use of clinical data to support a 510(k) application (Cheat Sheet: Clarifies when clinical data needs to be submitted)
Evidentiary expectations for implanted devices (Cheat Sheet: Increases transparency for manufacturers)
The FDA will hold a webinar on the topic on October 26 and will accept comments until December 6, 2023.
Does cancer screening increase longevity? The answer is still maybe.
In our last issue, we covered a meta-analysis published by JAMA Internal Medicine which (as we and many others reported it) concluded that six common cancer screening tests did essentially nothing to increase longevity on a population basis. (One of the six did increase longevity, according to their analysis, but only by about three months.) A STAT First Opinion piece published last week by Peter B. Bach, MD, called that conclusion into question.
Bach’s main beef: The authors didn’t dig deeply enough into the mathematical details.
First off, this study simply cannot conclude that cancer screening doesn’t increase longevity - because that statement is a negative, which can’t be proven. Instead, what the study can tell you is that it can’t prove that cancer screening does increase longevity. (See the difference there? Go ahead, read it again. It didn’t make sense to us at first, either.)
Second, the analysis, he says, lacked the statistical power to make the claims it did. Or at least, it may have lacked that power. Bach admits that meta-analyses, by their very nature, “don’t have a specific power,” but still, the authors should have had more skepticism about the numbers that resulted from combining studies.
Bach also takes issue with the omission of certain studies, including meta-analyses, which might have changed the researchers’ results - especially in regards to lung cancer. And he notes that their use of 100 person-years instead of the standard 100,000 person-years as their death-rate denominator requires rounding that could mask small differences between groups.
Commentary: Finding answers to big questions like the one the JAMA Internal Medicine study asked is really tough, people. But if nothing else, consider this: This paper did not show that cancer screening is a slam dunk. Screening has costs, and on a population level, those costs are significant. While this current debate centers on population-level impact, we wonder how screening would fare in more targeted populations.
September is Sepsis Awareness Month - and more awareness is definitely needed
Following up on our story on the challenge of diagnosing sepsis early enough to treat and save lives, a recent CDC study makes it clear that the problem is wider and deeper than just diagnosis. In looking at hospitals across the US, the CDC found that 27% of hospitals have no sepsis program at all (55% in small hospitals). Even when a hospital has a program, more than half do not give the program leaders dedicated time to focus on sepsis-related issues and processes. You can read the full CDC Hospital Sepsis Program Report for all the details.
Food for Thought
For mammography, radiologist + AI ≥ radiologist + radiologist
More evidence that AI could safely help radiologists interpret mammograms appeared in a study of more than 55,000 women in Sweden, published in Lancet Digital Health last week. The study’s goal: to make sure that AI-assisted mammography interpretation was no worse at diagnosing breast cancer than the standard double reading by two radiologists. The study also looked at single reading by AI alone, as well as triple reading: AI plus two radiologists.
The results: AI-assisted mammography wasn’t any worse than the standard approach - in fact, it was better. The standard double reading by two radiologists diagnosed 250 cancers (cancer diagnosis rate 0.4%), and AI alone essentially matched that, at 246 (0.4%). Both AI plus one radiologist and AI plus two radiologists were statistically superior to the standard (261 for AI plus one, 269 for AI plus two; both came out to a cancer diagnosis rate of 0.5%).
Another fascinating detail: Mammograms interpreted by AI alone resulted in 47% fewer women being recalled for additional diagnostic imaging. This, the authors note, would present “a major reduction in unnecessary worry for women involved,” but also presents a string of questions about “medical responsibility, public acceptability, radiologist training, and certification of AI systems.”
Clinical criteria for Alzheimer’s changing to follow the science
At the recent Alzheimer's Association International Conference, the National Institutes on Aging and the Alzheimer’s Association proposed changes (draft and exhibits) to the clinical criteria for Alzheimer’s disease (AD). The updates reflect increasing understanding of the underlying drivers that distinguish the disease from other cognitive disorders.
Current criteria specify the presence of three late effects of AD, together known as AT(N), as seen in cerebrospinal fluid or via imaging:
A for amyloid plaques
T for Tau tangles
N for neurodegeneration (aka brain shrinkage or atrophy)
The “big news” in the new proposal is the demotion of these crude AT(N) diagnostics to focus on A and T (early presumed causes) and incorporate biomarkers that have been validated in plasma.
Commentary: This might appear to be a small step, but it is far more - the dawn of an evidence-based set of diagnostics that will fuel discovery and truly effective treatment development. Physician associations have a perceived (sometimes real) reputation as being slow to make changes in recommendations. We are pleased to see relatively fast progress here.
As has been the case in many disease states - from diabetes’ urine tests decades ahead of insulin to breast cancer’s HER2 tests before Herceptin to HIV viral-load testing before AZT for AIDS - the diagnostic breakthrough often precedes the therapeutic breakthrough. We hope this is also the beginning of a time when diagnostics are less taken for granted.
Quick Hits
SARS-CoV-2 evolves three times faster in white-tailed deer than it does in humans, according to a study in Nature Communications. However, current vaccines are still protective against the strains found in deer thus far.
EUA Update
The FDA issued one new 510(k) premarket notification, no new EUAs, ten amendments to existing EUAs, and four revocations in August. Data is available at TestingCommons.com
510(k) Premarket Notifications (1): Cepheid Xpert Xpress CoV-2/Flu/RSV plus
New EUAs: 0
COVID Molecular: (0)
COVID Antigen: (0)
Amendments to Existing EUA’s (10):
COVID Molecular: 4
COVID Collection Kits: 0
COVID Antigen: 3
Respiratory multiplex (COVID and Flu): 3
Revocations (4): Spectron Hymon SARS-CoV-2 Test Kit | Exact Science Laboratories SARS-CoV-2 (N gene detection) Test | Exact Science Laboratories COVID-Flu Multiplex Assay | Xtrava Health SPERA COVID-19 Ag Test |