KEYWORDS: Kidney, Image segmentation, Data modeling, Magnetic resonance imaging, 3D modeling, Performance modeling, Statistical modeling, 3D image processing, Tumor growth modeling, 3D acquisition
Purpose: Multiparametric magnetic resonance imaging (mp-MRI) is being investigated for kidney cancer because of better soft tissue contrast ability. The necessity of manual labels makes the development of supervised kidney segmentation algorithms challenging for each mp-MRI protocol. Here, we developed a transfer learning-based approach to improve kidney segmentation on a small dataset of five other mp-MRI sequences.
Approach: We proposed a fully automated two-dimensional (2D) attention U-Net model for kidney segmentation on T1 weighted-nephrographic phase contrast enhanced (CE)-MRI (T1W-NG) dataset (N = 108). The pretrained weights of T1W-NG kidney segmentation model transferred to five other distinct mp-MRI sequences model (T2W, T1W-in-phase (T1W-IP), T1W-out-of-phase (T1W-OP), T1W precontrast (T1W-PRE), and T1W-corticomedullary-CE (T1W-CM), N = 50) and fine-tuned by unfreezing the layers. The individual model performances were evaluated with and without transfer-learning fivefold cross-validation on average Dice similarity coefficient (DSC), absolute volume difference, Hausdorff distance (HD), and center-of-mass distance (CD) between algorithm generated and manually segmented kidneys.
Results: The developed 2D attention U-Net model for T1W-NG produced kidney segmentation DSC of 89.34 ± 5.31 % . Compared with randomly initialized weight models, the transfer learning-based models of five mp-MRI sequences showed average increase of 2.96% in DSC of kidney segmentation (p = 0.001 to 0.006). Specifically, the transfer-learning approach increased average DSC on T2W from 87.19% to 89.90%, T1W-IP from 83.64% to 85.42%, T1W-OP from 79.35% to 83.66%, T1W-PRE from 82.05% to 85.94%, and T1W-CM from 85.65% to 87.64%.
Conclusions: We demonstrate that a pretrained model for automated kidney segmentation of one mp-MRI sequence improved automated kidney segmentation on five other additional sequences.
KEYWORDS: Kidney, Image segmentation, Magnetic resonance imaging, 3D modeling, Data modeling, 3D image processing, Tumor growth modeling, Algorithm development, Tissues, Cancer
Multi-parametric magnetic resonance imaging (mp-MRI) is a promising tool for diagnosis of renal masses and may outperform computed tomography (CT) to differentiate between benign and malignant renal masses due to superior soft tissue contrast. Deep learning (DL)-based methods for kidney segmentation are under-explored in mp-MRI which consists of several pulse sequences, including primarily T2-weighted (T2W) and contrast-enhanced (CE) images. Multi-parametric MRI images have domain shift due to differences in acquisition systems and image protocols, leading to lack of generalizability of methods for image segmentation. To perform similar automated kidney segmentation on another mp- MRI sequence, the model needs a large dataset with manual segmentations to train a model from scratch, which is labor intensive and time consuming. In this paper, we first trained a DL-based method using 108 cases of labeled data to contour kidneys using T1 weighted-Nephrographic Phase CE-MRI (T1W-NG). We then applied a transfer learning approach to other mp-MRI images using pre-trained weights from the source domain, thus eliminating the need for large manually annotated datasets in target domain. The fully automated 2D U-Net for kidney segmentation in source domain containing total 108 3D images of T1W-NG, yielded Dice-similarity coefficient (DSC) of 0.91 ± 0.07 on test cases. The transfer learning of pretrained weights of T1W-NG model on the smaller target domain T2W dataset containing total 50 3D images for automated kidney segmentation generated DSC of 0.90 ± 0.06 (p<0.05), which was an improvement of 3.43% in DSC by compared to the without transfer learning approach (T2W-UNet model).
Brachytherapy (BT) combined with external beam radiotherapy (EBRT) is the standard treatment for cervical cancer. Accurate segmentation of the tumor and nearby organs at risk (OAR) is necessary for accurate radiotherapy (RT) planning. While OAR segmentation has been widely studied, showing promising performance, accurate tumor and/or corresponding clinical target volume (CTV) segmentation has been less explored. In cervical cancer RT, magnetic resonance (MR) imaging is used as the standard imaging modality to define the CTV, which is very challenging as the microscopic spread of tumor cells is not clearly visible even in MRI. We propose a two-step convolutional neural network (CNN) approach to delineate CTV from T2-weighted (T2W) MR images. First, a human expert needs to select a seed point inside the CTV region, from which the MR volume is cropped to produce a region of interest (ROI) volume. The ROI volume is then fed to an attention U-Net to produce CTV segmentation. A total of 213 MR datasets from 125 patients was used to develop and evaluate the proposed methodology. The network was trained using 2-dimensional (2-D) slices extracted in the axial direction from 183 MR datasets and augmented using translation operation. The proposed method was tested on the remaining 30 MR datasets and yielded Mean±SD dice similarity coefficient (DSC) of 0.80±0.06 and Hausdorff distance (95th percentile) of 3.30±0.58 mm. The performance of our method is superior to the standard U-Net-based method (pvalue< 0.005). Although the proposed method is semi-automatic, the observer variability coefficient of variation (CV) was reported as 2.86% that demonstrated the high reproducibility of the algorithm.
Purpose: The objective of this study is to develop and evaluate a fully automated, deep learning-based method for detection of COVID-19 infection from chest x-ray images.
Approach: The proposed model was developed by replacing the final classifier layer in DenseNet201 with a new network consisting of global averaging layer, batch normalization layer, a dense layer with ReLU activation, and a final classification layer. Then, we performed an end-to-end training using the initial pretrained weights on all the layers. Our model was trained using a total of 8644 images with 4000 images each in normal and pneumonia cases and 644 in COVID-19 cases representing a large real dataset. The proposed method was evaluated based on accuracy, sensitivity, specificity, ROC curve, and F1-score using a test dataset comprising 1729 images (129 COVID-19, 800 normal, and 800 pneumonia). As a benchmark, we also compared the results of our method with those of seven state-of-the-art pretrained models and with a lightweight CNN architecture designed from scratch.
Results: The proposed model based on DenseNet201 was able to achieve an accuracy of 94% in detecting COVID-19 and an overall accuracy of 92.19%. The model was able to achieve an AUC of 0.99 for COVID-19, 0.97 for normal, and 0.97 for pneumonia. The model was able to outperform alternative models in terms of overall accuracy, sensitivity, and specificity.
Conclusions: Our proposed automated diagnostic model yielded an accuracy of 94% in the initial screening of COVID-19 patients and an overall accuracy of 92.19% using chest x-ray images.
COVID-19 is a highly contagious infectious disease that has infected millions of people worldwide. Polymerase Chain Reaction (PCR) is the gold standard diagnostic test available for COVID-19 detection. Alternatively, medical imaging techniques, including chest X-ray (CXR), has been instrumental in diagnosis and prognosis of patients with COVID-19. Enabling the CXR with machine learning-based automated diagnosis will be important for rapid diagnosis of the disease by minimizing manual assessment of images by the radiologists. In this work, we developed a deep learning model that utilizes the transfer learning approach using a pre-trained Residual Network model. The Residual Network 50 (ResNet50) is trained from scratch by utilizing the initial architecture and pre-trained weights to provide the classification results. Two types of classification (two-class and three-class) is performed using the developed model. A cascaded approach is adopted for two-class classification where the classification is performed in two phases. The dataset used for training and evaluating the model comprises of 8,254 images in total out of which 1651 images were considered for testing the cascaded model (15 COVID-19) and three-class classification (51 COVID-19). The model was evaluated using accuracy, sensitivity, specificity, and F1-score metrics. Our cascaded model yielded an accuracy of 91.8% for classification of abnormal and normal cases and 97.9% for the classification of pneumonia and COVID-19 images. In the three-class classification, our model reported an accuracy of 92% in classifying normal, pneumonia (bacterial and viral) and COVID-19 cases.
Hirschsprung’s disease is a motility disorder that requires the assessment of the Auerbach’s (myenteric) plexus located in muscularis propria layer. In this paper, we describe a fully automated method for segmenting muscularis propria (MP) from histopathology images of intestinal specimens using a method based on convolutional neural network (CNN). Such a network has the potential to learn intensity, textural, and shape features from the manual segmented images to accomplish distinction between MP and non-MP tissues from histopathology images. We used a dataset consisted of 15 images and trained our model using approximately 3,400,000 image patches extracted from six images. The trained CNN was employed to determine the boundary of MP on 9 test images (including 75,000,000 image patches). The resultant segmentation maps were compared with the manual segmentations to investigate the performance of our proposed method for MP delineation. Our technique yielded an average Dice similarity coefficient (DSC) and absolute surface difference (ASD) of 92.36 ± 2.91% and 1.78 ± 1.57 mm2 respectively, demonstrating that the proposed CNNbased method is capable of accurately segmenting MP tissue from histopathology images.
Diabetic retinopathy (DR), which is a major cause of blindness in the world is characterized by hard exudate lesions in the eyes as these lesions are one of the most prevalent and earliest symptoms of DR. In this paper, a fully automated method for hard exudate delineation is described that could assist ophthalmologists for timely diagnosis of DR before disease progress to a level beyond treatment. We used a dataset consist of 107 images to develop a U-Net-based method for hard exudate detection and segmentation. This network consists of shrinking and expansive streams in which shrinking path has the same structure as conventional convolutional networks. In expansive path, obtained features are merged with those from shrinking path with the proper resolution to generate multi-scale features and accomplish distinction between hard exudate and normal tissue in retinal images. The training images were augmented artificially to increase the number of samples in the dataset and avoid overfitting issues. Experimental results showed that our proposed method reported sensitivity, specificity, accuracy, and Dice similarity coefficient of 96.15%, 80.77%, 88.46%, and 67.23 ± 13.60% on 52 test images, respectively.
Segmentation of optic disk (OD) from retinal images is a crucial task for early detection of many eye diseases, including glaucoma and diabetic retinopathy. The main goal of this research is to facilitate early diagnosis of certain pathologies via fully automated segmentation of the OD from retinal images. We propose a deep learning-based technique to delineate the boundary of OD from retinal images of patients with diabetic retinopathy and diabetic macular edema. In our method, we first localized OD within a region of interest (ROI) using random forest (RF). The RF is an ensemble algorithm, which trains and combines multiple decision trees to produce a highly accurate classifier. We then used a convolutional neural network (CNN) based model to segment OD from chosen ROIs in the retinal images. The developed algorithm has been validated on 480,249 image patches extracted from 49 images of public Indian diabetic retinopathy image dataset (IDRiD). This dataset includes images with large variability in terms of the spatial location of OD and presence of other eye lesions that resemble the contrast of OD. Validation metrics including average of Dice and Jaccard indexes (DI and JI), Hausdorff distance (HD), and absolute surface difference (ASD) were reported as 82.62 ± 11.07%, 71.78 ± 14.87%, 13.19 ± 10.90 mm, and 22.74 ± 19.78%, respectively. As compared to other alternative methods, such as K-nearest neighbors (KNN), deformable models, graph-cuts, and image thresholding, our method yielded higher accuracy for OD segmentation in comparison to manual expert delineation. The algorithm-generated results demonstrate the usefulness of our proposed method for automated segmentation of OD from retinal images.
Myocardial tissue characterization on 3-dimensional late gadolinium enhancement magnetic resonance (3D LGE MR) is of increasing clinical importance for the quantification and spatial mapping of myocardial scar, a recognized substrate for malignant ventricular arrhythmias. Success of this task is dependent upon reproducible segmentation of myocardial architecture in 3D-space. In this paper, we describe a novel method to segment left ventricle (LV) myocardium from 3D LGE MR images using a U-Net convolutional neural network (CNN)-based model. Our proposed network consists of shrinking and expanding paths, where image features are captured and localized through several convolutional, pooling and up-sampling layers. We trained our model using 2090 slices extracted and artificially augmented from 14 3D LGE MR datasets, followed by validation of the trained model on ten 3D LGE MR unobserved test datasets inclusive of 926 slices. Averages of Dice index (DI) and absolute volume difference as a percentage versus manual defined myocardial volumes (NAVD) on the test dataset were obtained, providing values of 86.61 ± 3.80 % and 12.95 ± 9.56%, respectively. These algorithm-generated results demonstrate usefulness of our proposed fully automated method for segmentation of the LV myocardium from 3D LGE MR images.
Accurate segmentation of the myocardial fibrosis or scar may provide important advancements for the prediction and management of malignant ventricular arrhythmias in patients with cardiovascular disease. In this paper, we propose a semi-automated method for segmentation of myocardial scar from late gadolinium enhancement magnetic resonance image (LGE-MRI) using a convolutional neural network (CNN). In contrast to image intensitybased methods, CNN-based algorithms have the potential to improve the accuracy of scar segmentation through the creation of high-level features from a combination of convolutional, detection and pooling layers. Our developed algorithm was trained using 2,336,703 image patches extracted from 420 slices of five 3D LGE-MR datasets, then validated on 2,204,178 patches from a testing dataset of seven 3D LGE-MR images including 624 slices, all obtained from patients with chronic myocardial infarction. For evaluation of the algorithm, we compared the algorithmgenerated segmentations to manual delineations by experts. Our CNN-based method reported an average Dice similarity coefficient (DSC), precision, and recall of 94.50 ± 3.62%, 96.08 ± 3.10%, and 93.96 ± 3.75% as the accuracy of segmentation, respectively. As compared to several intensity threshold-based methods for scar segmentation, the results of our developed method have a greater agreement with manual expert segmentation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.