Visage Breast Density AI Classifier – From Conception to Routine Use

Mammographic breast density is an important feature that relates to an individual’s risk for cancer, and patients with higher density can have obscured visualization on mammograms of potential cancer. Breast density plays a vital role in driving the patient’s clinical journey. Accordingly, the FDA updated its regulations to require reporting breast density (1). However, the density assessment is largely subjective and there is the potential for interobserver variability. Visage Imaging, Inc. partnered with Yale New Haven Hospital in 2019 to develop an artificial intelligence-based solution to address these limitations. 

Visage 7 Enterprise Imaging Platform | Visage Breast Density

The development of the breast density AI classifier (Visage Imaging, Inc., San Diego, CA) was based on a Yale dataset of about 33,000 mammography studies with a total of about 164,000 images. The goal of the AI was to show greater agreement in density classification with a panel, a consensus group of radiologists, than the individual radiologist, thereby providing an objective and more consistent assessment.

The initial results were promising (94.4% AI vs. 89.3% individual radiologist agreement for binary classification [dense vs. non-dense]), and so the AI was validated at NYU Langone Health, where the AI performed well, too (97.4% AI agreement). In no case did the AI algorithm disagree with the consensus reading by more than one density category (2). This speaks to not only the performance of the AI algorithm, but also its generalizability to other hospitals. Interestingly, although the AI algorithm was trained on density classifications made by individual radiologists, the trained algorithm performed more like that of a consensus group of radiologists. So, in this case, and challenging the notion that humans are the reference standard for performance, the AI algorithm, in a way, became “smarter” than the individual radiologist.

US FDA clearance was obtained shortly after that, in January 2021, just nine months after the initial submission. The accelerated development, testing, validation, regulatory clearance, productization, and deployment were enabled by the Visage AI Accelerator (3). The AI algorithm was productized and deployed at Yale to assess the breast density on every mammogram acquired since April 2021. Notably, the AI inference runs in the background, leveraging the Visage render servers' GPUs right after image acquisition completion (study close)/import into PACS. That way, the inference results are available immediately at the time of image viewing rather than waiting for an inference to run. At Yale, the breast density results are automatically displayed upon opening the study for viewing. Importantly, the radiologist can select a different density classification than what the AI predicts. This information is recorded for future analysis.

A post-deployment analysis at Yale was conducted from September 23, 2021, through July 11, 2022 (291 days) to assess the agreement of the AI algorithm with the individual radiologist. 93,650 screening mammograms had density assessment recorded from 164 readers. The agreement of the radiologists with the AI algorithm was very high at 99.35% (4). There was consistently high performance reported for FDA clearance and deployment in the real world, indicating that the Visage Breast Density AI algorithm is generalizable and robust.

Given recent US federal legislation regarding population-based breast density notification, methods are needed for more standardized density classification, and this tool is a solution. Furthermore, the Visage Breast Density tool has the potential to play a role in guiding supplemental screening and optimizing workflow.

References:

1. FDA U. FDA Updates Mammography Regulations to Require Reporting of Breast Density Information and Enhance Facility Oversight. In, 2023.
2. Lewin J, Schoenherr S, Seebass M, Lin M, Philpotts L, Etesami M, Butler R, Durand M, Heller S, Heacock L, Moy L, Tocino I, Westerhoff M. PACS-integrated machine learning breast density classifier: clinical validation. Clinical Imaging. 2023;101200-205.
3. Lin M. What's Needed to Bridge the Gap Between US FDA Clearance and Real-world Use of AI Algorithms. Academic Radiology. 2021.
4. Philpotts L, Szabla P, Lin M. Artificial Intelligence Platform for Determining Breast Density in Digital Breast Tomosynthesis Mammography: How Does It Perform? In:American Roentgen Ray Society, 2023.

The Essence of Visage [Video]