KEYWORDS: Data modeling, Data fusion, Diseases and disorders, Image fusion, Chest imaging, Performance modeling, Modeling, Machine learning, COVID 19, Education and training
PurposeOur study investigates whether graph-based fusion of imaging data with non-imaging electronic health records (EHR) data can improve the prediction of the disease trajectories for patients with coronavirus disease 2019 (COVID-19) beyond the prediction performance of only imaging or non-imaging EHR data.ApproachWe present a fusion framework for fine-grained clinical outcome prediction [discharge, intensive care unit (ICU) admission, or death] that fuses imaging and non-imaging information using a similarity-based graph structure. Node features are represented by image embedding, and edges are encoded with clinical or demographic similarity.ResultsExperiments on data collected from the Emory Healthcare Network indicate that our fusion modeling scheme performs consistently better than predictive models developed using only imaging or non-imaging features, with area under the receiver operating characteristics curve of 0.76, 0.90, and 0.75 for discharge from hospital, mortality, and ICU admission, respectively. External validation was performed on data collected from the Mayo Clinic. Our scheme highlights known biases in the model prediction, such as bias against patients with alcohol abuse history and bias based on insurance status.ConclusionsOur study signifies the importance of the fusion of multiple data modalities for the accurate prediction of clinical trajectories. The proposed graph structure can model relationships between patients based on non-imaging EHR data, and graph convolutional networks can fuse this relationship information with imaging data to effectively predict future disease trajectory more effectively than models employing only imaging or non-imaging data. Our graph-based fusion modeling frameworks can be easily extended to other prediction tasks to efficiently combine imaging data with non-imaging clinical data.
Quantitative image features that can be computed from medical images are proving to be valuable biomarkers of underlying cancer biology that can be used for assessing treatment response and predicting clinical outcomes. However, validation and eventual clinical implementation of these tools is challenging due to the absence of shared software algorithms, architectures, and the tools required for computing, comparing, evaluating, and disseminating predictive models. Similarly, researchers need to have programming expertise in order to complete these tasks. The quantitative image feature pipeline (QIFP) is an open-source, web-based, graphical user interface (GUI) of configurable quantitative image-processing pipelines for both planar (two-dimensional) and volumetric (three-dimensional) medical images. This allows researchers and clinicians a GUI-driven approach to process and analyze images, without having to write any software code. The QIFP allows users to upload a repository of linked imaging, segmentation, and clinical data or access publicly available datasets (e.g., The Cancer Imaging Archive) through direct links. Researchers have access to a library of file conversion, segmentation, quantitative image feature extraction, and machine learning algorithms. An interface is also provided to allow users to upload their own algorithms in Docker containers. The QIFP gives researchers the tools and infrastructure for the assessment and development of new imaging biomarkers and the ability to use them for single and multicenter clinical and virtual clinical trials.
Diagnosing Interstitial Lung Diseases (ILD) is a difficult task. It requires experienced chest radiologists that may not be available in less-specialized health centers. Moreover, a correct diagnosis is needed to decide for an appropriate treatment and prognostic. In this paper, we focus on the classification of 3 common subtypes of ILDs: Usual Interstitial Pneumonia (UIP), Non-Specific Interstitial Pneumonia (NSIP) and Chronic Hypersensitivity Pneumonitis (CHP). We propose a graph model of the lungs built from a large dataset. The structure of the graph is inspired from medical knowledge of disease predominance, where the nodes correspond to 24 distinct regions obtained from lateral, anterior-posterior and vertical splits of the images. The adjacency matrix is built from distances between intensity distributions of distinct regions. Graphs models are interpretable and were successfully used in neuroimaging. However, to the best of our knowledge, this is the first attempt to use a graph model of the lungs for classifying ILDs. In the particular case of ILDs, graph methods are relevant for the following reasons. In order to differentiate between the subtypes, not only the types of local patterns of the disease are important but also their anatomical location. Therefore, we hypothesize that the comparison between regional distributions of Hounsfield Unit (HU) values is relevant to discriminate between the considered ILD subtypes. For instance, typical UIP shows a spatial predominance of reticular abnormalities and honeycombing in the peripheral regions of the lung bases. Therefore, we expect a marked difference of HU distributions between the central and peripheral regions of the lung bases. Moreover, the construction of the graph leads to an interpretable patient descriptor. The descriptor led to encouraging area under the Receiver Operating Characteristic (ROC) curve in 0.6-0.8 for one-versus-one classification configurations, which also showed to outperform feature sets based on a simple concatenation of regional HU distributions.
KEYWORDS: Breast cancer, Performance modeling, Tumors, Magnetic resonance imaging, Data modeling, Wavelets, Statistical modeling, Image filtering, Statistical analysis, Breast
Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is sensitive but not specific to determining treatment response in early stage triple-negative breast cancer (TNBC) patients. We propose an efficient computerized technique for assessing treatment response, specifically the residual tumor (RT) status and pathological complete response (pCR), in response to neoadjuvant chemotherapy. The proposed approach is based on Riesz wavelet analysis of pharmacokinetic maps derived from noninvasive DCE-MRI scans, obtained before and after treatment. We compared the performance of Riesz features with the traditional gray level co-occurrence matrices and a comprehensive characterization of the lesion that includes a wide range of quantitative features (e.g., shape and boundary). We investigated a set of predictive models (∼96) incorporating distinct combinations of quantitative characterizations and statistical models at different time points of the treatment and some area under the receiver operating characteristic curve (AUC) values we reported are above 0.8. The most efficient models are based on first-order statistics and Riesz wavelets, which predicted RT with an AUC value of 0.85 and pCR with an AUC value of 0.83, improving results reported in a previous study by ∼13%. Our findings suggest that Riesz texture analysis of TNBC lesions can be considered a potential framework for optimizing TNBC patient care.
Multiple sclerosis (MS) is a neurological disease characterized by focal lesions and morphological changes in the brain captured on magnetic resonance (MR) images. However, extraction of the corresponding imaging markers requires accurate segmentation of normal-appearing brain structures (NABS) and the lesions in MR images. On MR images of healthy brains, the NABS can be accurately captured by MR intensity mixture models, which, in combination with regularization techniques, such as in Markov random field (MRF) models, are known to give reliable NABS segmentation. However, on MR images that also contain abnormalities such as MS lesions, obtaining an accurate and reliable estimate of NABS intensity models is a challenge. We propose a method for automated segmentation of normal-appearing and abnormal structures in brain MR images that is based on a locally adaptive NABS model, a robust model parameters estimation method, and an MRF-based segmentation framework. Experiments on multisequence brain MR images of 30 MS patients show that, compared to whole-brain MR intensity model and compared to four popular unsupervised lesion segmentation methods, the proposed method increases the accuracy of MS lesion segmentation.
We aim to develop a better understanding of perception of similarity in focal computed tomography (CT) liver images to determine the feasibility of techniques for developing reference sets for training and validating content-based image retrieval systems. In an observer study, four radiologists and six nonradiologists assessed overall similarity and similarity in 5 image features in 136 pairs of focal CT liver lesions. We computed intra- and inter-reader agreements in these similarity ratings and viewed the distributions of the ratings. The readers’ ratings of overall similarity and similarity in each feature primarily appeared to be bimodally distributed. Median Kappa scores for intra-reader agreement ranged from 0.57 to 0.86 in the five features and from 0.72 to 0.82 for overall similarity. Median Kappa scores for inter-reader agreement ranged from 0.24 to 0.58 in the five features and were 0.39 for overall similarity. There was no significant difference in agreement for radiologists and nonradiologists. Our results show that developing perceptual similarity reference standards is a complex task. Moderate to high inter-reader variability precludes ease of dividing up the workload of rating perceptual similarity among many readers, while low intra-reader variability may make it possible to acquire large volumes of data by asking readers to view image pairs over many sessions.
We developed a method for automated classification and detection of liver lesions in CT images based on image patch
representation and bag-of-visual-words (BoVW). BoVW analysis has been extensively used in the computer vision
domain to analyze scenery images. In the current work we discuss how it can be used for liver lesion classification and
detection. The methodology includes building a dictionary for a training set using local descriptors and representing a
region in the image using a visual word histogram. Two tasks are described: a classification task, for lesion characterization, and a detection task in which a scan window moves across the image and is determined to be normal
liver tissue or a lesion. Data: In the classification task 73 CT images of liver lesions were used, 25 images having cysts, 24 having metastasis and 24 having hemangiomas. A radiologist circumscribed the lesions, creating a region of interest (ROI), in each of the images. He then provided the diagnosis, which was established either by biopsy or clinical follow-up. Thus our data set comprises 73 images and 73 ROIs. In the detection task, a radiologist drew ROIs around each liver lesion and two regions of normal liver, for a total of 159 liver lesion ROIs and 146 normal liver ROIs. The radiologist also demarcated the liver boundary. Results: Classification results of more than 95% were obtained. In the detection task, F1 results obtained is 0.76. Recall is 84%, with precision of 73%. Results show the ability to detect lesions, regardless of shape.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.