Novel concept for systematic testing of AI models for MRI acquisition shifts with simulated data

Christiane Posselt; Edith Gramotke; Abhijeet Parida; Mehmet Yiğitsoy; Stefanie Remmele

doi:10.1117/12.2653883

3 April 2023 Novel concept for systematic testing of AI models for MRI acquisition shifts with simulated data

Christiane Posselt, Edith Gramotke, Abhijeet Parida, Mehmet Yiğitsoy, Stefanie Remmele

Proceedings Volume 12467, Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment; 124671B (2023) https://doi.org/10.1117/12.2653883
Event: SPIE Medical Imaging, 2023, San Diego, California, United States

Conference Poster

Abstract

Recently published guidelines recommend the validation of models for medical image processing against variations of image acquisition parameters. This work presents a novel concept for systematic tests that investigate a model’s performance to MRI acquisition shifts in brain scans of multiple sclerosis patients. Contrast changes of T2-weighted FLAIR images related to changes of two scan parameters (TE, TI) are simulated using artificial data. These images were created from 114 brain MRI, normal tissue segmentation masks and expert MS lesion masks (NAMIC, OpenMS, MSSEG2, ISBI15, LC08). Contrast changes were simulated by using the FLAIR signal equation. The experiments evaluate the F1 scores of models trained on images based on different uniform and non-uniform acquisition parameters when tested on typical acquisition parameter shifts. The acquisition parameter variations of the experiments are based on guidelines for quality assurance and the analysis of protocols and images from different institutions and open datasets. The dependence of the F1 score on both parameters could be approximated by linear functions with R2 scores of 0.94 to 0.98. Thus, linear functions can be used to model the dependence of the F1 score on the influencing factors, thereby allowing for the derivation of a "save" range of image acquisition parameters to meet a desired performance metric. The segmentation model was more sensitive to changes in TE compared to TI. The maximum lesion F1 loss when applying the models to out-of-distribution data ranged from 0.04 to 0.36 and was significant in all the experiments, even when the model was trained on data representing scans of different contrast (p<0.01). This underlines the need for testing segmentation models against acquisition shifts.

Citation Download Citation

Christiane Posselt, Edith Gramotke, Abhijeet Parida, Mehmet Yiğitsoy, and Stefanie Remmele "Novel concept for systematic testing of AI models for MRI acquisition shifts with simulated data", Proc. SPIE 12467, Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment, 124671B (3 April 2023); https://doi.org/10.1117/12.2653883

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available