2.1.

Samples

A total of 200 samples of urinary stones were selected from a library of more than 1400 units. The urinary stones were obtained either from natural expulsion or surgical removal from patients of the urology service at the Hospital Universitari de Bellvitge, Barcelona (Spain). The samples underwent no other treatment than ethanol and water rinsing for cleaning and convenient drying for their proper conservation. Note that in this work, sample stands for a urinary stone (or various stones or stone fragments) collected from a patient in a single episode. If a sample consists of several pieces coming from the cleavage of the stone, they are not considered independent. Any piece of stone obtained directly from the patient (e.g., after a lithotripsy treatment) or after cutting the stone for its analysis is considered a fragment.

The selection criterion was to include the widest variability within each class of stones in terms of structure and composition considering both pure and mixed components. For the analysis of the samples used to compare both methodologies, a few representative fragments were used from each stone sample. The methodology is intended to prove its efficacy for the analysis of some fragments instead of selecting all the pieces. The fragments were chosen so that they were representative for the structure of the whole stone (core components, surface deposits, etc.).

The sample preparation differs for each of the analysis techniques used in this work. Stone samples were selected using microscopy as described in Sec. 2.2 and later analyzed using IR (Sec. 2.3) and NIR-HSI (Sec. 2.4). Note that NIR-HSI measurements were performed first because HSI is a nondestructive technique.

Microscopy only required cutting the samples for the observation of the outer and inner parts of the stone.

For the measurement of the IR spectra of the samples, a representative fragment of each sample was dried in an oven for 24 h at 40°C prior to the analysis. The sample was ground in an Agatha mortar and 0.9 mg of the sample was then mixed with 0.3 g of KBr powder. A pellet was prepared by setting this mixture under 10 Tm pressure in a manual press (Perkin Elmer, MA, USA).

Different from IR, HSI is fundamentally a surface analysis technique and requires a relatively flat surface for the measurements. Reflectance spectra are measured, so an uneven surface would lead to an important signal loss. All samples were cut with a surgical knife, thus fulfilling the requirements of HSI and allowing the acquisition of spectra from the inner part of the stones, which usually carry important information related to the stone history. Due to the characteristics of the technique, the whole exposed surface was analyzed after placing the sample on a plastic holder with the area of interest facing the detector. For those samples in which the inner and outer parts were clearly different, both were analyzed for better accuracy of the result.

2.2.

Microscopy

Stereoscopic microscopy was used for the selection of samples (training and test sets) prior to the IR and NIR-HSI measurements, according to morphological features such as texture, color, and structure. These features allow the achievement of the sample selection criteria indicated in Sec. 2.1. Microscopy analyses were performed as widely described in Refs. 10 and 11.

For those samples not clearly classified using optical microscopy, scanning electron microscopy was used since it allowed the close visualization of the structure of the stones and the performance of elemental analysis. The equipment used was a JEOL JSM-6300 scanning electron microscope (Japan), coupled to an Oxford Instruments Link ISIS-200 x-ray dispersive energy spectrometer (UK).

Microscopy should be considered in this work as a support methodology used for sample selection, because it only gives qualitative information. The numerical comparison between IR and NIR-HSI was done on all the samples and standards after this selection step.

2.3.

IR Methodology

IR spectroscopy has been used as reference methodology to test the results of NIR-HSI. The main advantages of considering IR as the standard are the robustness and wide acceptance of the technique and the calculation of a numerical value for the sample composition. The IR spectra of the prepared pellets were collected using an IR spectrometer Spectrum BX (Perkin Elmer, MA, USA). The spectral range measured ran from 450 to $4400{\mathrm{cm}}^{-1}$ (which corresponds to a wavelength range from $\sim 2200$ to 22,000 nm), with a spectral resolution of $4{\mathrm{cm}}^{-1}$.

For the determination of the sample composition, a comparison of the measured spectra with those in Ref. 21 was done. This collection of spectra considered pure compounds, combination of two species, and even combinations of three components for the common cases of calcium oxalate mono and dihydrate and carbonate apatite, including different gradations for the amounts of each component in a 10% (w/w) grading.

2.4.

NIR-HSI Device and Architecture Setup

HSI technique, also known as chemical imaging, can be regarded as the stacking of pictures of the same surface, each of them taken at a different wavelength.²² This group of images defines the so-called hypercube, a three-dimensional dataset that is analyzed as a single image containing a full spectrum for each pixel.²³ This technique, therefore, leads to the complete characterization of every pixel of the image. Compared to IR, HSI offers not only one spectrum per sample, but several hundreds while consuming the same amount of time (a few seconds) for the measurements.

The HSI device used was configured at DICMA, Sapienza–Università di Roma (Rome, Italy) and is precisely described in the previous work.¹⁹ The main feature of the equipment is an imaging spectrometer ImSpector™ N17E (Specim, Finland) with a spatial resolution $<15\mu \mathrm{m}$. The spectral resolution was 7 nm, covering the spectral range from 1000 to 1700 nm, which yielded a total of 121 wavelengths (data points/spectrum) measured. The spectrometer was coupled to a 50 mm lens with an image resolution of 320 pixels. The software Spectral Scanner™ v.2.3 (DV Optics srl, Italy) was used for the acquisition, preprocessing, and visualization of the images. The reflectance signal was calculated as a function of the calibrated black and white limits.

It is interesting to note that both IR and NIR are included within the IR spectral range but they do not overlap. The equipment used for those measurements, therefore, gives spectrally independent signals. This fact makes it clear that the chemical properties (i.e., bond vibrations or rotations) observed for the classification are different for each case. NIR represents an alternative spectroscopic approach, which offers results comparable to IR, yet requires little sample pretreatment.

When using HSI, the radiation penetration depth is defined as the depth at which the incident light is reduced by 99%. It varies, in fact, according to sample characteristics, surface attributes, and investigated wavelengths (depth is lower when wavelength increases). In this study, this magnitude can be estimated as 10 to $20\mu \mathrm{m}$.

The pixel resolution is much smaller than the size of any urinary stone (which generally ranges from a few millimeters to 2 cm), and is also smaller than most surface imperfections or porosity. This resolution prevents any remarkable signal loss or pixel misclassification, because the area surrounding the surface defect would still be correctly classified.

2.5.

Data Analysis

The data treatment included the handling of the spectra, creation of the classification (reference) model, and sample treatment.

The NIR reflectance spectra were directly analyzed as obtained, i.e., no derivatives or other data pretreatment were applied. However, due to the large amount of information available from the HSI measurements, the reduction of the dimensionality of the system was required so that only the most influential variables (wavelengths) were taken into consideration. For this issue, a randomized principal component analysis (PCA) algorithm was used to select the 10 principal components, a figure chosen by cross-validation. This way each pixel is represented by a vector of 10 components instead of more than 100 spectral measurements. This decreases the number of parameters for the classifier to learn, thus preventing overfitting to a great extent. Since the variables that had little classification power were not included in the model, the noise of the system was likewise decreased.²⁴

The classification model included a variable number between six and eight stone fragments of each stone type with homogeneous composition. Each stone fragment was collected from a different patient, so model overfitting was reduced. Note that although only a few samples were taken in the training set according to the characteristics of the HSI technique (i.e., acquisition of a spectrum for each pixel of the image sample), thousands of pixels were used for training purposes. The number of samples in the model was large enough to cover all the naturally occurring components in urinary stones.

The image analysis of the training and test sample sets was performed using SciPy and Scikit-learn, two open source Python libraries for scientific analysis.^25,26

A series of classifiers were tested for the creation of a model able to categorize the samples. A classifier is any mathematical function able to assign a label “class” to a given sample when it has been learned on a specific training set.²⁷ Following the criteria of the best classification rate, quadratic discriminant analysis (QDA) was selected. This classifier is a classic Bayesian method, generally used on machine learning. It assumes the normal distribution of the data within each class and different covariances within classes.²⁸ Due to the complex nature of the sample set used in this work, QDA is a good approach since it can establish boundaries between classes based on quadratic algorithms, so it is more flexible than linear classifiers. This classification scheme showed a better performance than others tested, including k-nearest neighbor, random forest, and support vector machines.

The creation of the model included a test step using the leave-one-out approach. The vectors of 10 principal components of every pixel of each fragment are placed in a pool. Then, each time a certain fragment has to be analyzed, that is when we want to assign a class label to each of its pixels, they are removed from the pool which now is the training set. The QDA classifier parameters are learned and the classification on the given fragment vectors, which act as the testing set, is performed. This process is iterated for every fragment. In pattern classification literature, this procedure is known as leave-one-out, although in this case it was done at a fragment level.²⁴

When the classification model created using the standards was applied to the samples, the structural analysis of the stone was achieved. The physical structure of the stone was reconstructed based on the results for the analysis of each pixel. These procedures lead to the creation of chemical maps of the stones which represent the spatial resolution of the technique.

3.1.

Performance of the HSI-NIR Model

The training set was analyzed using IR spectroscopy (reference values) and also using NIR-HSI. In this case, only homogeneous samples were considered, since the objective of this step was to train the classifier in the recognition of stone classes using a large number of pure pixels. It was easier to isolate pure pixels in homogeneous stones. The advantages in spatial resolution of HSI are not relevant in this step. The graphical results obtained for the HSI model performance are shown in Fig. 1.

Fig. 1

Results for the classification using the training set. Not all the samples used for the creation of the model are shown. Upper row: reconstructed image from the original stones, obtained from the Spectral Scanner software. The different fragments (with different origins) can be appreciated. Lower row: results of the classification model, where each pixel is assigned a class and, so, a different color (from MATLAB® software).

The overall correct classification rate of the model was calculated to be 90.4% in terms of correctly classified pixels compared to the results obtained by IR spectroscopy. The specific efficacy of the model on the classification of each stone type is shown in the normalized confusion matrix (Table 1). The diagonal of the matrix shows how, for most classes, only a minor number of pixels are confused with other classes, with the exception of carbonate apatite [${\mathrm{Ca}}_{10}({\mathrm{PO}}_4){({\mathrm{CO}}_3,\mathrm{OH})}_6{(\mathrm{OH})}_2$] (CAP) and struvite ($\mathrm{Mg}·{\mathrm{NH}}_4{\mathrm{PO}}_4·6{\mathrm{H}}_2\mathrm{O}$) (STR). Indeed, the confused pixels for the rest of the stone classes represent only residual values. Note the composition difference between CAP and hydroxylapatite (lacking any carbonate ion in the lattice), sometimes misclassified as the main component of CAP stones.

Table 1

Normalized confusion matrix. The efficiency of the model can be measured as a percentage of correct classification for each stone class.

It can be appreciated that the general behavior of the model is very efficient, with the overall results $>95\%$ of correct prediction for most stone classes. The exception is represented by CAP and STR, which are confused with each other in an important percentage (30 to 40%). These results can be attributed to the similar features their NIR spectra share and the related appearance of both types of urinary stones. The model shows a good general result, even considering the remarkable variability within groups seen in the spectra, as plotted in Fig. 2. The general trend of the spectra in each class and the unique spectral bands for some components lead to a proper classification result. The mean spectra of the training set shows such specific bands (see Fig. 2). As specified in Sec. 2.4, NIR does not overlap with the IR spectral range. Despite this important difference, most of the relevant bonds in organic components can be identified in the NIR range, including: C─H, S─H, N─H, O─H, or O─H linked to an aromatic ring. The sensitivity of those bonds (which are also identified in IR spectroscopy) in the NIR range allows for proper recognition of urinary stones’ mineralogical components, as discussed here.

Fig. 2

First and second rows: accumulation of all the spectra of the training set, plotted as reflectance intensity versus wavelength (nm). The mean spectra are plotted at the bottom of the image.

The distribution of pixels (depending on the size and number of stones used in the model) of each class used for creating the model is shown in Fig. 3. Only uric acid dihydrate (${\mathrm{C}}_5{\mathrm{H}}_4{\mathrm{N}}_4{\mathrm{O}}_3·2{\mathrm{H}}_2\mathrm{O}$) (UAD) counts had a smaller number of pixels for the creation of the model; it was difficult to find stones rather pure in UAD and with no contamination of uric acid (${\mathrm{C}}_5{\mathrm{H}}_4{\mathrm{N}}_4{\mathrm{O}}_3$) (UA), since UAD rapidly transforms into UA in contact with urine.²⁹ The computer model, though, is able to perfectly discriminate ($>99\%$, as seen in Table 1) between the two derivatives of uric acid, and also to distinguish them from any other component to a great extent.

Fig. 3

Frequencies histogram. The model is created from such a distribution of pixels. The total count of pixels is related not only to the number of fragments considered for each group but also to the size of those fragments.

3.2.

Conventional IR Analysis of Samples

From the test samples group, only one sample was discarded from the experiment using IR spectroscopy. Its composition, according to the IR analysis, was 100% dittmarite, corresponding to the chemical composition ${\mathrm{MgNH}}_4{\mathrm{PO}}_4·{\mathrm{H}}_2\mathrm{O}$. This mineral corresponds to the dehydration of STR, the hexahydrated derivative, which may take place during storage periods of these stones.³⁰ The NIR-HSI test set did not include such unexpected minerals as standards, so the sample was not considered.

3.3.

Implementation of the NIR-HSI Model

The created model was rapidly implemented on the 200 test samples by using the described data analysis software. Two types of output were demanded from this software: the classification of each pixel into a group from the standards and a reconstructed picture of the stone, from which the location of all components and their spatial relationships are easily recognizable. These pictures represent the spatial resolution, that is, the possibility of performing a topological assessment of the detected species in the sample offered by this methodology: the added value that NIR-HSI offers compared to IR spectroscopy.

The samples in the test set were analyzed using the NIR-HSI model and the results, expressed as percentage for each component, were compared to the composition measured by IR spectroscopy. Assuming an error $<10\%$ for the IR classification due to the component gradations in the consulted bibliography, the results from NIR-HSI were considered to correlate with the IR results if the NIR-HSI composition matched, considering $\pm 10\%$ deviation.

A thorough analysis of the results shows the composition obtained by NIR-HSI to generally fit that from IR (achieving 70% correlation), with some variations that require a more careful analysis. It should be taken into account that the sensitivity of minor components is higher for the NIR-HSI technique than for IR spectroscopy. Minor components could cause only a small IR spectral band to grow or appear, so the spectrum would hardly be changed. IR spectroscopy strongly relies on the analyst’s experience, so such small changes might be hard to quantify. Instead, since each pixel is analyzed individually by NIR-HSI, minor components are identified with the same precision as main constituents of the sample. Therefore, as will be seen later, NIR-HSI can give more information in some cases than IR spectroscopy does.

The description of the classification results and their correlation to the IR analyses is listed as groups based on their mineralogical composition and is summarized in Table 2.

Table 2

Overview of the classification results obtained by the near-infrared hyperspectral imaging (NIR-HSI) methodology and their correlation to IR spectroscopy for each type of stone. Results correspond to a total of 200 urinary stone analyses using both techniques.

3.4.

Spatially Defined Analysis

As already stated, the main advantage of the NIR-HSI technique over the conventional IR spectroscopy is the spatial resolution of the analysis (i.e., topological assessment of the different mineralogical species), not available when the sample is ground for the IR measurements. Thus, NIR-HSI offers the possibility to describe all the stages in the stone formation and the urine conditions at each point of the process, information that can direct a treatment that is precisely adapted to every patient. This is the advantage that the examples presented in this section highlight.

Among the total amount of samples that were analyzed, some specific representative examples will be accurately described in order to illustrate the performance of the developed model and to highlight the differential results that both techniques offer (Fig. 4). The results for the composition of these examples are listed in Table 3.

Fig. 4

Examples of some results for stone analysis comparing near-infrared hyperspectral imaging and infrared spectroscopy. First row: color, real pictures of the samples. Second row: reconstructed images corresponding to the samples listed in the first row; these images are the output of the imaging analysis software used (the legend shows the correspondence between colors and components).

Table 3

Mineralogical composition corresponding to the samples listed in Fig. 4, according to the two methodologies used. The composition values are expressed as percentage.

The first example [Fig. 4(a)] shows a calcium oxalate mixed stone, with a similar composition for both methodologies. This stone presents a singular structure: it is basically a COM stone, totally covered by a COD layer. Thus, the order of precipitation is important; COD (which usually precipitates due to a high concentration of Ca ion and oxalate in urine³²) precipitated exclusively at the end of the stone episode. In this case, it should be appreciated that the higher percentage of COD quantified by the NIR-HSI technique is due to the analysis of both the inner part and surface at the same time so the COD surface is bigger than that for COM. HSI is a surface technique, so attention shall be paid to which part of the stone is being analyzed. Once this is clear, HSI allows the description of two steps in the stone formation, while IR can only give a general composition ratio of these two phases.

The stone in Fig. 4(b) corresponds to a very common composition, calcium oxalate and CAP. In this case, the stone was formed as COD and the core transformed into COM due to the higher stability of the latter.¹⁰ We can observe this phenomenon as a different color in the stone nucleus. CAP did precipitate in this stone as a few deposits within the stone structure. Both techniques could identify the same three components; the higher percentage of COM in the IR should be attributed to the difference of relative surface (NIR-HSI) and relative weight in the stone (IR) of this component. NIR-HSI allows the location of CAP not only in the stone core so the stone formation steps become clearer when image analysis is used.

The mixed stone of UA and COM in Fig. 4(c) yielded a very similar percentage composition when analyzed by the two methodologies suggested in this work. However, NIR-HSI allowed the location of COM in the core of the stone, so the acidic urine conditions needed for the precipitation of UA can be located in the latter stages of the stone formation. This separate analysis of core and shell parts requires the addition of steps to the stone analysis process by IR spectroscopy (the measurement of two spectra), so the time for the analysis is increased.

The stone in Fig. 4(d) is a common mixed stone, which includes CAP and STR. These stones usually lack any defined structure, and this exact situation can be seen in the picture. The quantification of the stone components is really similar using both techniques.

In Fig. 4(e), UAD is the kinetic derivative of UA, and, as seen in the calcium oxalate case, it also slowly transforms into the most stable UA if it remains in contact with urine.²⁴ While the composition according to both techniques is basically the same, HSI also gives the distribution of components, which appears to be different in the two analyzed fragments. HSI is able, in this case, to define which fragments are UAD, so the description of the sample is totally precise for this particular patient.