Computational spectral imaging systems have been gaining momentum in recent years thanks to their ability to acquire information using a smaller number of measurements alongside emerging powerful Deep Learning algorithms. Recently [1] we developed a multispectral imaging system based on 25um liquid crystal retarder; however, its relatively large thickness limits the speed. In this work, we will present a multispectral imaging system based on a 2um thick Fabry-Perot spectral modulator filled with nematic liquid crystal combined with a deep learning algorithm. The system combined with a novel deep learning algorithm, designed to operate at 9 narrow spectral bands covering the VIS and NIR, holds a fast response and provides reliable spectral information, which makes it a potential candidate for various applications.
[1] Doron Pasha, Marwan J. Abuleil, Isaac August, Ibrahim Abdulhalim, Faster multispectral imager based on thin liquid crystal modulator and 3D neural network lattice, Lasers&Photonics Review 2200913(8p) (2023).
SignificanceHyperspectral microscopy grants the ability to characterize unique properties of tissues based on their spectral fingerprint. The ability to label and measure multiple molecular probes simultaneously provides pathologists and oncologists with a powerful tool to enhance accurate diagnostic and prognostic decisions. As the pathological workload grows, having an objective tool that provides companion diagnostics is of immense importance. Therefore, fast whole-slide spectral imaging systems are of immense importance for automated cancer prognostics that meet current and future needs.AimWe aim to develop a fast and accurate hyperspectral microscopy system that can be easily integrated with existing microscopes and provide flexibility for optimizing measurement time versus spectral resolution.ApproachThe method employs compressive sensing (CS) and a spectrally encoded illumination device integrated into the illumination path of a standard microscope. The spectral encoding is obtained using a compact liquid crystal cell that is operated in a fast mode. It provides time-efficient measurements of the spectral information, is modular and versatile, and can also be used for other applications that require rapid acquisition of hyperspectral images.ResultsWe demonstrated the acquisition of breast cancer biopsies hyperspectral data of the whole camera area within ∼1 s. This means that a typical 1 × 1 cm2 biopsy can be measured in ∼10 min. The hyperspectral images with 250 spectral bands are reconstructed from 47 spectrally encoded images in the spectral range of 450 to 700 nm.ConclusionsCS hyperspectral microscopy was successfully demonstrated on a common lab microscope for measuring biopsies stained with the most common stains, such as hematoxylin and eosin. The high spectral resolution demonstrated here in a rather short time indicates the ability to use it further for coping with the highly demanding needs of pathological diagnostics, both for cancer diagnostics and prognostics.
The parametrization of light rays in form of light fields (LF) have become the standard and probably the most common way for the representation, analysis and processing of rays emitted from 3D objects or from 3D displays. Essentially, the LFs are 4D maps representing the spatial and angular distribution of the intensity of the rays. Nowadays, with the increasing availability of spectral imagers, the conventional LF can be augmented with the spectral information, yielding to what we call spectral light fields (SLFs). Spectral light fields refer to a 5D distribution of spatial, angular and spectral ray’s distribution. Thus, the SLF can be viewed as spectral radiance over a 2D manifold, or as 5D parameterization of a plenoptic function. In this paper we show the utility of the SLFs for digital 3D reconstruction. We show that the additional spectral domain provides important information that can be utilized to overcome 3D reconstruction artefacts caused by ambiguities in commonly captured LFs. We demonstrate the utilization of the SLFs for profilomety and refocusing.
Light fields are four-dimensional parametrizations of rays, extensively used for the representation of the rays emitted from three-dimensional objects. With the increasing availability of spectral imagers, conventional light fields can be augmented by the additional spectral information, yielding a five-dimensional ray parametrization referred to as a spectral light field. We demonstrate that utilization of the spectral channel information can improve the profilometric performance of light-field cameras.
During the past years, several compressive spectral imaging techniques were developed. With these techniques, an optically compressed version of the spectral datacube is captured. Consequently, the information about the object and targets is captured in a lower dimensional space. A question that rises is whether the reduction of the captured space affects the target detection performance. The answer to this question depends on the compressive spectral imaging technique employed. In most compressive spectral imaging techniques, the target detection performance is deteriorated. We show that our recently introduced technique, dubbed Compressive Sensing Miniature Ultra-Spectral Imaging (CSMUSI), yields similar target detection and false detection rates to that of conventional hyperspectral cameras.
Recently we presented a Compressive Sensing Miniature Ultra-spectral Imaging System (CS-MUSI)1 . This system consists of a single Liquid Crystal (LC) phase retarder as a spectral modulator and a gray scale sensor array to capture a multiplexed signal of the imaged scene. By designing the LC spectral modulator in compliance with the Compressive Sensing (CS) guidelines and applying appropriate algorithms we demonstrated reconstruction of spectral (hyper/ ultra) datacubes from an order of magnitude fewer samples than taken by conventional sensors. The LC modulator is designed to have an effective width of a few tens of micrometers, therefore it is prone to imperfections and spatial nonuniformity. In this work, we present the study of this nonuniformity and present a mathematical algorithm that allows the inference of the spectral transmission over the entire cell area from only a few calibration measurements.
We review two compressive spectroscopy techniques based on modulation in the spectral domain that we have recently proposed. Both techniques achieve a compression ratio of approximately 10:1, however each with a different sensing mechanism. The first technique uses a liquid crystal cell as a tunable filter to modulate the spectral signal, and the second technique uses a Fabry-Perot etalon as a resonator. We overview the specific properties of each of the techniques.
Compressive sensing theory was proposed to deal with the high quantity of measurements demanded by traditional hyperspectral systems. Recently, a compressive spectral imaging technique dubbed compressive sensing miniature ultraspectral imaging (CS-MUSI) was presented. This system uses a voltage controlled liquid crystal device to create multiplexed hyperspectral cubes. We evaluate the utility of the data captured using the CS-MUSI system for the task of target detection. Specifically, we compare the performance of the matched filter target detection algorithm in traditional hyperspectral systems and in CS-MUSI multiplexed hyperspectral cubes. We found that the target detection algorithm performs similarly in both cases, despite the fact that the CS-MUSI data is up to an order of magnitude less than that in conventional hyperspectral cubes. Moreover, the target detection is approximately an order of magnitude faster in CS-MUSI data.
The interest in liquid crystal devices for photonic non-display devices has grown recently due to their mature quality and the continuous improvement of their speed combined with the rising nanoscale and optoelectronic technologies. Of particular interest is their application in imaging systems as compact devices to manipulate the wavefront, wavelength, phase or polarization.
Recently we have been developing variety of specially designed LC devices integrated into imaging systems for specific spectro-polarimetric imaging applications using small number of LC devices. These included: (i) wide dynamic range tunable filters for hyperspectral imaging and frequency domain optical coherence tomography, (ii) discrete narrowband tunable filter for multispectral imaging, (iii) compact polarization rotator for polarimetric imaging, (v) wideband achromatic waveplate for polarimetric camera, (vii) polarization independent LCFP tunable filter, and lately (vii) single LC retarder for hyperspectral imaging. In this report we shall present the main concepts of these devices and their functionality into spectro-polarimetric imaging systems such as in skin cancer diagnosis, and imaging oximetry [1-4].
Selected Publications:
1. S. Isaacs et.al, Applied Optics 53, H91-H101 (2014).
2. M. AbuLeil et.al., Optics Letter 39, 5487-90 (2014).
3. I. August, et.al., Scientific Reports, communicated 2016.
4. M. AbuLeil et.al., in preparation.
The theory of compressive sensing (CS) has opened up new opportunities in the field of imaging. However, its
implementation in this field is often not straight-forward and the optical imaging system engineer encounters
several hurdles on the way of compressive imaging (CI) realization. The principles of CI design may differ
drastically from the principles used for conventional imaging. Analytical tools developed for conventional imaging
may not be optimal for compressive imaging. Nor are the conventional imaging components. Therefore often the CI
designer needs to develop new tools, and imaging schemes. In this paper we overview the main challenges that
might arise in the design of compressive imaging systems. The challenges are demonstrated through four tasks and
systems: compressive two dimensional (2D) imager, compressive motion detection, compressive spectral imaging
and compressive holography.
Recently we introduced a Compressive Sensing Miniature Ultra-Spectral Imaging (CS-MUSI) system. The system is based on a single Liquid Crystal (LC) cell and a parallel sensor array where the liquid crystal cell performs spectral encoding. Within the framework of compressive sensing, the CS-MUSI system is able to reconstruct ultra-spectral cubes captured with only an amount of ~10% samples compared to a conventional system. Despite the compression, the technique is extremely complex computationally, because reconstruction of ultra-spectral images requires processing huge data cubes of Gigavoxel size. Fortunately, the computational effort can be alleviated by using separable operation. An additional way to reduce the reconstruction effort is to perform the reconstructions on patches. In this work, we consider processing on various patch shapes. We present an experimental comparison between various patch shapes chosen to process the ultra-spectral data captured with CS-MUSI system. The patches may be one dimensional (1D) for which the reconstruction is carried out spatially pixel-wise, or two dimensional (2D) - working on spatial rows/columns of the ultra-spectral cube, as well as three dimensional (3D).
Hyperspectral imagery involves capturing and processing a tremendous amount of data, which sets severe system resource requirements. This has motivated the application of compressive sensing for different spectroscopic and spectroscopic imager systems. Several new compressive hyperspectral architectures have been designed to stretch the common limitations of classical systems. However, the application of the compressive sensing framework involves design of system architectures that differ significantly from the conventional ones. Since compressive sensing differs essentially from conventional sensing, it cannot be implemented for hyperspectral imaging by simply modifying one of the components of a conventional hyperspectral system, rather it requires a complete new design. In this work we present a comparison between four compressive hyperspectral architectures to conventional architectures. The compressive hyperspectral sensing compared are: Coded Aperture Snapshot Spectral Imaging (CASSI), Compressive HS Imaging by Separable Spatial And Spectral Operators (CHISSS), (Liquid-crystal Compressive spectral Imager) LiCSI and (Spectral Single-Pixel) SSP systems. Those methods are compared to conventional spatial/spectral scanning hyperspectral such as pushbroom, whiskbroom and color filter techniques. A fundamental comparison between these architectures is presented in terms of optical system volume and radiometric efficiency.
Recently we introduced a hyperspectral compressive sensing scheme that uses separable projections in the spatial and spectral domains. The separable encoding schemes facilitates the optical implementation, reduces the computational burden dramatically, and storage requirements. Owing to these benefits we have been able to encode the hyperspectral cube in all three dimensions. In this work we present a comparison between various reconstructions methods applied to the hyperspectral data captured with our separable compressive sensing systems.
Compressive hyperspectral imaging is based on the fact that hyperspectral data is highly redundant. However, there is no
symmetry between the compressibility of the spatial and spectral domains, and that should be taken into account for
optimal compressive hyperspectral imaging system design. Here we present a study of the influence of the ratio between
the compression in the spatial and spectral domains on the performance of a 3D separable compressive hyperspectral
imaging method we recently developed.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.