Many questions in developmental biology and neuroscience require tissue-scale measurements of multiple cell parameters. Recent progress in in-depth and large-scale microscopy approaches are transforming brain imaging, but generally lack efficient color contrast modalities. After introducing the basic concepts of multiphoton microscopy, the talk will discuss ongoing work aiming at augmenting the information content of multiphoton microscopy of tissues. First, we will present chromatic multiphoton serial (ChroMS) microscopy (1), a recent approach combining trichromatic two-photon (2P) excitation through wavelength mixing and microtome-based serial block-face image acquisition to acquire large-volume images of color-labelled brain tissue. This approach provides micrometric color imaging with constant resolution over the entire imaged volume. We illustrate its potential of ChroMS for several types of measurements, such as color-based morphological, clonal and connectivity analyses. Then, we will discuss three-photon (3P) microscopy, a recent approach providing superior imaging depth than 2P microscopy in highly scattering tissues, but requiring the development of novel infrared MHz OPA sources (2). Finally, we will discuss the combination of multiphoton fluorescence imaging with coherent contrast modalities providing complementary information such as third harmonic generation (THG) (3,4). References (1) Abdeladim et al, Nat Comm (2019) (2) Guesmi et al, Light Sci App (2018) (3) Morizet et al, Optica (2019) (4) Morizet et al, Optica (2021)

Confocal fluorescence microscopy is a privileged tool for life imaging, but can generate phototoxicity due to the prolonged sample illumination. When cells are organized along sheets lying on 2D surfaces curved in a 3D volume (e.g. epithelial cells), we propose a new approach allowing to automatically estimate the surface on which these cells are distributed from a small number of acquisitions (typically 0.1% of the voxels). This allows to concentrate thereafter the illumination around the surface of interest and thus to scan only a small portion (typically between 1% and 5%) of the volume containing the sample.

This conference presentation, “Ultrafast imaging in flow” was recorded for the Unconventional Optical Imaging III conference at SPIE Photonics Europe 2022.

we present the very first fully integrated streak camera realized with a specific ultrafast CMOS sensor. This sensor is composed of a vector of 200 photodiodes and each photodiode signal is sampled on an on-chip memory that can record 200 samples. It results on a sampling matrix of 200x200 samples that stores the values of the slit at the different sampling time. The sampling rate can be set up to 4 GS/s, i.e., the maximal total sampling rate of the matrix is 800 GS/s. It is possible to post trigger the streak camera and to extract the past samples.

Stray light characterization using ultrafast time of flight imaging was demonstrated recently for the testing of refractive telescopes, using a streak tube with a femtosecond laser. It was shown that individual contributors such as ghost reflections and scattering could be measured individually and identified through their optical path length with a time-resolved measurement. This allows unprecedented understanding of stray light properties in optical instruments, especially for high-end space telescopes. In this paper, we demonstrate the extension of this method for the validation and improvement of stray light rejection in an optical calibration facility for large space instruments. Here, the stray light paths to be characterized have long range, up to 20 meters. Therefore, the ultrafast sensor considered is a single photon avalanche diode (SPAD) and the illumination is achieved by a picosecond laser. We demonstrate that we are able to measure the individual stray light contributors, for example the scattering on the optical surfaces or the multiple scattering events occurring between baffles and vanes. As each contributor can be identified, the time-of-flight results are used to determine how to improve the optical calibration facility. Moreover, while the measurement is affected also by the detection system intrinsic contribution, this effect can be removed so that the final result only shows the performance of the optical facility. This new approach is extremely useful as it allows pushing further the achievable performances of space telescope characterization, where conventional methods were currently reaching a plateau.

Holographic transmission matrix measurements of optical multimode fibre communication channels enable optical precoding on the transmitter-side exploiting the physical layer. This approach can achieve a crucial SNR advantage for legitimate communication participants over illegitamte ones, such as an eavesdropper. We present measurements on a 55-mode fibre channel including transmitter Alice and receiver Bob, to which an eavesdropper Eve is physically coupled. Alice performs a programmable optical precoding through an SLM based on singular value decomposition enabling space division multiplexing. Demonstration of confidential data transmission is based on measurements from both channels Alice/Bob and Alice/Eve, which are used for determining the achievable amount of securely exchanged data between Alice and Bob, i.e. secrecy rate. Results are shown on step-index fibres of up to 100 m length.

Three-dimensional, fluorescence imaging methods with ~1 MHz frame rates are needed for high-speed 3D imaging flow cytometry and volumetric imaging of neuronal activity. The frame rates of current fluorescence imaging methods are limited to kHz by the photon budget, slow camera readout, and/or slow laser beam scanners. I will present a new fluorescence imaging method called LEAD (line excitation array detection) microscopy, capable of providing 0.8 million frames per second. This method performs 0.8 MHz line-scanning of excitation laser beam using a chirped signal-driven longitudinal acousto-optic deflector to create a virtual light-sheet, and images the field-of-view with a linear photomultiplier tube array to generate a 66×14 pixel frame each scan cycle. I will also present an implementation of the LEAD microscopy as a blur-free 3D flow cytometer for Caenorhabditis elegans moving at 1 m/s with 3.5-micron resolution and signal-to-background ratios >200. Signal-to-noise measurements indicate LEAD fluorescence microscopy can reach higher resolutions and pixels per frame without compromising frame rates.

Endoscopy through coherent fiber bundles plays a significant role in industrial and medical imaging. By using a diffractive optical element in form of a diffuser on the distal side, the information of the measurement volume is encoded as 2D speckle patterns on the camera. This allows minimally invasive single-shot 3D imaging through a flexible low-cost endoscope with a diameter of less than 1 mm. Neural networks are employed to get fast image reconstructions from the speckle patterns with high quality. Moreover, multispectral speckle coding is used to increase the amount of transferred information and thus improving imaging capabilities.

Imaging through an opaque layer has been a longstanding problem, which has typically been addressed by separating the extremely weak light field that is not scattered from the overwhelming amount of background light that has scattered multiple times such as in optical coherence tomography. A recent paradigm shift is harnessing spatial information from the multiply-scattered or diffused light for imaging applications. The key to harnessing spatial information from multiply-scattered light is the hidden correlations of random speckles formed by the interference of scattered light. The most celebrated one of such hidden correlations is known as the angular “memory effect”: when the incident wavefront of a coherent beam on a diffusive medium is tilted by a small angle, the transmitted wavefront is tilted in the same direction by the same amount. However, it has certain constraints such as finite angular correlation range and predetermined direction that correlations are observed. Here, we propose and experimentally demonstrate a general approach to arbitrarily modify the angular memory effect in opaque scattering media. Introducing an angular memory operator by applying a certain transformation to the field transmission matrix, we show that the eigenvectors of such an operator have perfect “memory” for arbitrarily chosen angles and tilt directions of input and output wavefronts. Our work paves the way to customize the memory effect of both classical and quantum waves for imaging, metrology, and communication applications in complex scattering systems.

We introduced several approaches of terahertz wavefront phase retrieval from intensity measured in a volumetric grid. Our developments include several experimental solutions for the registration of multiple intensity distributions spaced along the optical axis for two types of terahertz sources, namely Gunn diode with frequency multiplication chain and quantum cascade laser. We implemented several measurement modes: (i) sequential raster scanning by single Schottky diode with two lock-in amplifiers, complimentary tuned to different sensitivities for high dynamic range recording; (ii) step by step registration on matrix photodetectors, with averaging over several images for every measurement plane; (iii) continuous measurement during the displacement of the motorized translation stage. The high dynamic range data acquisition allowed us to successfully implement single-beam terahertz surface profilometry in the reflection, while the on-the-go recording ensures the shortest measurement times. In addition, we experimentally appraised two matrix detectors (INO and I2S) and applied several phase retrieval algorithms which proved their effectiveness in various experimental conditions, namely for the intensity registration in various diffraction zones and axial measurement plane allocations.

Optical imaging has become an invaluable tool in life science. Among the variety of available techniques, selective plane illumination microscopy (SPIM) allows for fast (x,y,z) imaging of fluorescent samples with reduced photobleaching. SPIM directly acquire the (x,y) slice corresponding to a thin light sheet that illuminates the sample, while the third spatial dimension is scanned. Promoted by the open source SPIM project, many designs variants are now available. This enables the study of various samples such as fly embryos, zebrafish embryos and others. The study of multi-labeled specimens implies to unmix the fluorophores, which usually relies on optical filters. As most of the light is rejected, this approach has a major drawback, as a large amount of information is lost (e.g., fluorophores with overlapping spectra cannot be unmixed). Therefore, there is a need for 3D imagers with hyperspectral capabilities, which can exploit the full-emission spectrum of a fluorescent sample. We will describe a computational hyperspectral light sheet microscope inspired from Hadamard spectroscopy. We generate structured light sheets using a digital micromirror device and focus the fluorescence signal onto the entrance slit of an imaging spectrometer. Then, we reconstruct the full hypercube from the raw data acquired for multiple structured light patterns. Our technique enjoys excellent spectral resolution and allows resolving overlapping fluorophores with up to nanometer resolution. Furthermore, the Hadamard patterns used for illumination allow maximizing the collected signal compared to previous hyperspectral SPIM setups. To reduce the acquisition time, we consider undersampled measurements for which we will present reconstruction results obtained using an algorithm based on a deep convolutional network.

Spectral imaging is at the cornerstone of many fields, including astronomy, environmental monitoring, food processing, agriculture, and biomedical imaging. While most current spectral techniques lack imaging speed, we describe a computational approach that allows for fast high spectral resolution imaging. Our set-up maps the image of the scene into the fiber of a compact spectrometer through a digital micromirror device (DMD), where a series of multiplexing patterns are uploaded. After a reconstruction step, the hypercube of the scene can be recovered. DMDs represent the fastest technology (e.g., >20 kHz) for spatial light modulators. The raw data is acquired for a (x,y,λ) hypercube of size 64x64x2048 in about 12 s, while the Hadamard inversion takes <1 s. The acquisition speed can be further reduced by limiting the number of Hadamard patterns; however, the resulting imaging reconstruction problem is turned into an underdetermined inverse problem, which requires regularization techniques to be used to obtain acceptable solutions, in particular, in the presence of noise. Deep learning is a very efficient framework to solve inverse problems in imaging. Following a recent trend, several convolution neural network architectures have provided a link between deep and optimization-based image reconstruction methods. Contrary to the initially proposed “black box” networks, these deep-learning methods rely on a forward operator and lead to more interpretable networks. Here, we review deep architectures for single-pixel image reconstruction and show that the network can be trained easily, in a end-to-end manner, using databases such as STL-10 or ImageNet. We present reconstruction results from simulated and experimental single-pixel acquisitions. We show that EM-Net generalizes very well to noise levels that are unseen during the training, despite having fewer learned parameters than alternative methods. The proposed EM-Net generally reconstructs images with fewer artifacts and with higher signal-to-noise ratios, particularly in high-noise situations.

Using deep neural networks, we demonstrate an ultra-fast scanning approach that extracts sharp pictures from motion blurred videos. This strategy allows us to complete image acquisitions at up to 58x the frame rate of similar stop-and-stare systems. This method is implemented on a Zeiss AXIO Observer Z1 microscope in both brightfield and phase contrast modes at stage speeds of 5,000 μm/s to 10,000 μm/s. We validate the proposed method by imaging tissue slides and live cell cultures over time. These results comprise the first application of recurrent neural networks for microscopy motion deblurring. The presented approach not only retrieves the clear image, but also corrects any defocusing that occurs over scanning operations due to height irregularities in samples. This technique will significantly accelerate microcopy measurements and will be particularly advantageous for imaging extensive sample types.

Nanoparticle-based applications require increasingly sensitive screening tools, necessitating single-particle, multi-parameter analysis techniques in the native liquid environment. In this work, we present a laser-scanning camera using acousto-optic deflectors and a single-photon counting module. By scanning a highly focused laser beam (532 nm) across a grid of pixels, and by detecting the associated emitted photons with high sensitivity, fluorescent nanoparticles of 100 nm and below can be imaged at frame rates up to 12 kHz.

While white light laparoscopy (WLL) has been established as a method that yields improved detection of peritoneal metastatic lesions over radiographic imaging, its sensitivity and specificity remain sub-optimal, leading to thousands of cancer patients that are over- or under- treated every year. We present studies that establish the sensitivity of polarization enhanced (PEL) laparoscopy to changes in the cross-section and alignment of scattering centers using phantoms and report on our initial in vivo human feasibility trial. The design of our PEL probe is simple and highly compatible with current laparoscopes, the standard of patient care and the established procedure flow for WLL. It comprises of a sheath inserted on the WLL laparoscope to provide linearly polarized illumination and a modified detection optical path that allows for recording of co- and cross-polarized images relative to the incident illumination. The sum of these images is equivalent to WLL, while their difference corresponds to PEL. Using PDMS phantoms with features that contain either TiO2 and ZnO2 particles, we demonstrate the enhanced sensitivity of PEL to scattering cross-section differences. Measurements performed with collagen gels that are being stretched reveal improved sensitivity of PEL (over WLL) to collagen fiber alignment. Our initial feasibility study results support the improved diagnostic potential of PEL. In fact, a Monte Carlo-based analysis approach that utilizes the color content of the PEL images indicates that PEL yields improved contrast for detection and differentiation of benign and malignant metastatic lesions. These studies provide strong motivation for further development of PEL imaging as an approach that may improve significantly the detection of peritoneal metastatic lesions.

The cervix plays a crucial role in pregnancy and childbirth. Indeed, the mechanical properties of the cervix change significantly as the gestational age of pregnant women increases. In particular, during full-term gestation, the cervix undergoes a gradual softening up to 37 weeks of amenorrhea without substantial changes in its anatomy and mechanical strength, allowing the growing fetus to remain in the uterus. This process accelerates after 37 weeks of amenorrhea, resulting in shortening and dilation of the cervix in preparation for delivery, which usually takes place around 41 weeks of amenorrhea. The change in the mechanical properties of the cervix is due to a remodeling process of its microstructure involving connective tissue, one of the main microscopic components of which is collagen. Several studies have shown that this process is due, among other things, to a decrease in the organization and density of collagen, as well as to an increase in its extractability and solubility. Due to the lack of techniques to probe the microstructure of the cervix in vivo during pregnancy, the process of cervical remodeling remains poorly understood in full-term pregnancy and even more so in preterm delivery, which is the leading cause of perinatal mortality worldwide. Several imaging techniques have been used to explore the microstructural changes of the cervix in pregnant women, such as X-ray, magnetic resonance, optical coherence tomography (OCT), and second harmonic generation (SHG). However, the in vivo use of X-rays and magnetic resonance is limited due to their potential danger to the patient and the fetus. In addition, OCT and SHG, despite their micrometric resolution, probe the cervical tissue with a small field of view and limited penetration depth, making these techniques difficult to use for in vivo applications. In recent years, polarimetric imaging has shown great potential for exploring the structure of various types of biological tissues, including the cervix. In particular, this technique has shown great sensitivity in characterizing the density and organization of cervical collagen. In this study, we used Mueller polarimetry to probe the cervical microstructure of pregnant women. The great advantage of this technique is that it can be performed with a macroscopic field of view, which is crucial for in vivo applications, while providing information on the microstructure of the tissue being explored. Furthermore, it has shown great promise in characterizing the density and orientation of cervical collagen. We implemented this technique in vivo on a group of pregnant women at term. In particular, the anisotropy and scattering properties of the cervix have been characterized as a function of gestational age by measuring linear birefringence and total depolarization, respectively. The use of a multispectral approach allowed the characterization of these properties at different depths. The most relevant polarimetric parameters have been determined to monitor the remodeling of the cervical microstructure during pregnancy.

Scattering of light in complex samples such as biological tissue renders most samples opaque to conventional optical imaging techniques, a problem of great practical importance [1]. However, although random, scattering of coherent light generates speckle patterns with universal statistics and angular and spatial correlations, that allow computational retrieval of diffraction-limited imaging [2,3]. Moreover, the random temporal fluctuations of speckle patterns can be exploited for super-resolution photo-acoustic and acousto-optic imaging, deep inside scattering samples [4-7]. I will present the fundamental principles and limitations of these novel approaches, as well as some of our recent efforts in undoing scattering by wavefront-shaping through image-metrics based optimization [8]. References [1] Z. Merali, Optics: Super vision, Nature 518, 158 (2015). [2] O. Katz et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nature photonics, 8(10), 784-790 (2014). [3] T. Yeminy, O. Katz, Guidestar-free image-guided wavefront shaping, Science Advances, 7, 21 (2021). [4] T. Chaigne et al. Super-resolution photoacoustic fluctuation imaging with multiple speckle illumination, Optica 3 (1), 54-57 (2016). [5] E. Hojman et al. Photoacoustic imaging beyond the acoustic diffraction-limit with dynamic speckle illumination and sparse joint support recovery, Optics Express 25 (5), 4875-4886 (2016). [6] D. Doktofsky et al. Acousto optic imaging beyond the acoustic diffraction limit using speckle decorrelation, Communications Physics 3 (1), 1-8 (2020). [7] M. Rosenfeld, D. Doktofsky, G. Weinberg, Y. Li, L. Tian, O. Katz, Acousto-optic Ptychography, Optica 2021 (in press) [8] T.Yeminy, O.Katz, Guide-star free image-guided wavefront shaping, Science Advances, 7 : eabf5364 (2021)

Light scattering has proven to be a hard limitation in a wide range of sensing applications, such as astronomical or biological imaging. In microscopy systems, the random perturbations introduced to the wavefront limit the achievable spatial resolution and imaging depth. In the past, several methods have been proposed to control how light interacts with the medium, allowing focusing and imaging through multiple scattering media by using wavefront shaping techniques. However, non-invasively imaging objects behind scattering media over large fields of view remains a challenging feat. Here, we introduce a novel approach that allows to recover fluorescent extended objects behind scattering layers well beyond the optical memory effect (ME) range without the use of neither adaptive optics nor wavefront shaping techniques. To do so, we project a collection of unknown random illumination speckle patterns through the scattering medium by using a simple rotating diffuser. For each position of the rotating diffuser, a different incoherent sum of speckle patterns is recorded by the camera. Even though these images are low-contrast, random, and seem to carry no information at all, they contain the information about the position of the emitters. Here we show that, if enough images are measured, it is possible to use Non-negative Matrix Factorization to demix all the information and to retrieve the relative position of each fluorescent emitter in the sample. As a proof of the technique, we show experimental results with both sparse and continuous objects, covering fields-of-view of up to three times the optical memory effect range

Optical imaging in complex media is a challenging task: due to multiple scattering, ballistic light is exponentially attenuated. This prevents conventional microscopy techniques from retrieving information beyond a millimeter inside biological tissues. We present an innovative way of focusing and imaging through scattering media using a model-based computational approach: a 2-layer neural network. This technique allows to retrieve transmission matrices of the system and thus reverse the scattering phenomenon. We are then able to retrieve the position of fluorescent beads through holographic diffusers. This approach is versatile and appliable to more challenging scenarios, like other scattering media or non-linear phenomena.

Regularized inverse approaches are now widely used for the three dimensional reconstructions in Tomographic Diffractive Microscopy. This family of methods consists in the minimization, under some constraints, of a cost function often composed with a data fidelity term and one or several regularization terms. On one hand, the data fidelity term is based on the likelihood of the data. On the other hand, the regularization terms are based on physical a priori (morphology, values,…). It is important to keep a good trade off between data fidelity and regularization. The contribution of these last ones is weighted through hyperparameters which require a precise tuning. The minimization of the Generalized Stein’s Unbiased Risk Estimator (GSURE) is efficient to select a set of optimal hyperparameters however it requires an accurate approximation of the data formation. In this work, we compare the efficiency of an unsupervised regularized approach using GSURE depending of the data model approximation. We compare the optimal reconstructions obtained using the first Born approximation on one side and the Beam Propagation Method on the other side. We chose the regularization of the Total Variation, which favor piece-wise constant structures. For the reconstruction, we used the Primal-Dual Condat-Vũ algorithm with backtracking. We apply both reconstruction methods on experimental data. Our results show that our unsupervised regularized method manages in both cases to find an optimal reconstruction.

We present a CNN-based quantification pipeline for the imaging and analysis of adherent cell cultures. The imaging part features two CNNs dedicated to lens-free microscopy performing accelerated holographic reconstruction and phase unwrapping. The analysis part features CNNs estimating several cellular metrics. These CNNs maps phase image into 2D quantitative representations of cell positions and measurements. The outputs images are processed by a local maxima algorithm to obtain a list of cell measurements. Here, we discuss the performance and limitations of this CNN-based quantification pipeline. The advantage is the fast processing time, i.e. the analysis of ~10.000 cells in 10 seconds.

Circulating tumor cell clusters (CTCCs) are associated with high metastatic potential and poor patient prognosis. However, they are difficult to detect and isolate because of their extremely low numbers. Here, we report on the use of machine learning based analysis to achieve highly accurate detection of CTCCs in flowing whole blood samples relying on the confocal detection of endogenous light scattering and fluorescence signals. Our custom flow cytometer utilizes laser excitation at 405, 488, and 633 nm and confocal detection of the corresponding light scattering signals as well as fluorescence in the 525 25nm and 67020nm range. Samples consist of whole blood isolated from mice or rats spiked with varying concentrations of CTCCs consisting of 2-15 cell CTCCs, flowed through the channels of a microfluidic device. The CTCCs utilized in this initial study express GFP, so that we can detect the strong GFP signal using the 525 nm detector and use that signal as the ground truth for assessing the performance of algorithms relying on the endogenous signals detected by the other detectors. Our data is acquired during 18 independent experiments with data from 13 days used for training and five days used for testing. There are over 6,000 true positive and over 60,000 false negative peaks in this data set. We consider narrow neural network, fine k-nearest neighbors and ensemble bagged tree (EBT) models and we find that an EBT with gentle boost model yields optimal performance. Using the data from the three light scattering channels and the autofluorescence channel results in CTCC detection with purity, sensitivity, specificity and accuracy that exceed 90% in the test data. These promising results motivate further development of label-free flow cytometry using non-GFP expressing CTCCs for in vitro and in vivo applications.

Label-free quantitative phase imaging (QPI) of highly scattering samples like embryos, spheroids etc., have been studied recently using gradient light interference microscopy (GLIM). [1-3] While GLIM images are quantitative, they lack the specificity brought on by fluorescence staining. However, fluorescence staining has its own drawbacks ranging from phototoxicity to extensive sample preparation and small window of observation owing to the photobleaching effects. [4] To introduce specificity in the quantitative phase images from GLIM, we use machine learning in a technique called phase imaging with computational specificity (PICS) [5]. Combining GLIM with a confocal microscopy setup and PICS gave rise to a new imaging capability, referred to as artificial confocal microscopy (ACM). [6] In ACM, the input images are obtained by transmission GLIM imaging, which is label-free. Through deep learning and training on fluorescence confocal data as ground truth, we obtain ACM images from GLIM data, but with axial resolution and chemical specificity associated with confocal. Thus, ACM is an ideal, nondestructive tool for characterizing live embryos. In this study, we used ACM to image 99 blastocyst stage mouse embryos. These embryos were stained with 7-AAD for nuclear detection. We trained a deep neural network (U-Net with efficient-net B0 encoder) on pairs of GLIM image (phase) and nucleus fluorescence images. Our model predictions can highlight the positions of the nuclei inside the cells, with SSIM and Pearson correlation coefficient values of 0.833 and 0.924, respectively. This 3D structure is then analyzed to determine the “structure function” associated with the nuclear position in 3D. The “form function” is given by the average shape of the nuclei. [7] We use biophysical markers, as well as form and structure functions derived from the 3D structure to grade the embryos and determine their viability. Funding: NSF: 1353368; 1652150; References [1] M. E. Kandel et al., "Epi-illumination gradient light interference microscopy for imaging opaque structures," Nature communications, vol. 10, no. 1, pp. 1-9, 2019. [2] M. J. Fanous, Y. Li, M. E. Kandel, A. A. Abdeen, K. A. Kilian, and G. Popescu, "Effects of substrate patterning on cellular spheroid growth and dynamics measured by gradient light interference microscopy (GLIM)," Journal of biophotonics, vol. 12, no. 12, p. e201900178, 2019. [3] T. H. Nguyen, M. E. Kandel, M. Rubessa, M. B. Wheeler, and G. Popescu, "Gradient light interference microscopy for 3D imaging of unlabeled specimens," Nat Commun, vol. 8, no. 1, p. 210, Aug 08 2017, doi: 10.1038/s41467-017-00190-7. [4] R. A. Hoebe, C. H. Van Oven, T. W. Gadella, Jr., P. B. Dhonukshe, C. J. Van Noorden, and E. M. Manders, "Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging," Nat Biotechnol, vol. 25, no. 2, pp. 249-53, Feb 2007, doi: 10.1038/nbt1278. [5] M. E. Kandel et al., "PICS: Phase Imaging with Computational Specificity," arXiv preprint arXiv:2002.08361, 2020. [6] X. Chen et al., "Artificial confocal microscopy for deep label-free imaging," arXiv preprint arXiv:2110.14823, 2021. [7] G. Popescu, Quantitative phase imaging of cells and tissues. McGraw-Hill Education, 2011.

Fiber-based lensless endoscopy is powerful tool for minimally invasive tissue in clinical practice. However, the inherent honeycomb-artifact reduce the resolution and increases diagnosis difficulty. We proposed an end-to-end resolution enhancement and classification network for fiber bundle imaging. Comparing with conventional interpolation and filtering methods, the average peak signal to noise ratio (PSNR) can be improved 2~6 dB. Then we trained a VGG-19 classification network on label-free multiphoton images of 382 human braintumor 28 nontumor brain samples. The results show the classification accuracy of enhanced images is up to 91%, while the fiber bundle images are only 67% accurate. The method paves the way to in vivo histologic imaging through miniaturized endoscopic probes, and gives rapid and accurate determination for intraoperative diagnosis.

Light propagation in complex media, such as paint, clouds, or biological tissues, is a very challenging phenomenon, encompassing fundamental aspects in mesoscopic and statistical physics. It is also of utmost applied interest, in particular for imaging. Wavefront shaping has revolutionized the ability to image through or in complex media. I will discuss how computational tools and machine learning allows to develop further wavefront shaping for imaging applications, and conversely discuss how the same complexity can be leveraged for optical computing tasks.

The evolution of the light intensity along an optical waveguide is evaluated by analyzing far-field right-angle Rayleigh light scattering. The method is based on point-by-point-spectral mapping distributed along the waveguide with a micrometric spatial resolution given by a confocal microscope, a high spectral resolution given by a spectrometer, and a high signal-to-noise ratio given by a highly cooled detector. This non-destructive and non-invasive experimental method allows the observation, in linear or nonlinear regime of propagation, the general Rayleigh scattering profile of the optical waveguide in a nominal operation, i.e., whatever the power or the wavelength of the light source (CW or pulsed laser), and can be applied to micrometer-scale waveguides of several centimeters in length, for which the longitudinal characterization is challenging. Applied to a tapered optical fiber with submicrometer final diameter and several centimeters long (called nanofiber), this method has proved its capacity in linear regime to collect different optical characteristics such as optical losses, mode beatings, transition from core-cladding to cladding–air guidance for different modes, localization of punctual defects, leaking of high order modes no longer guided by the fiber. Furthermore, the experimental results are successfully compared to measurements provided by the state-of-the-art Optical Backscatter Reflectometer system, and to numerical simulations. Moreover, coupled to the spectral resolution of the spectrometer, the method have allowed in the nonlinear regime the distributed measurements of a Raman cascading process along the nanofiber, for the first time to our knowledge. Our experimental technique is complementary to other characterization methods, generally focused on the optical parameters of the waveguide input or output. This technique can also be extended to others waveguides whatever its geometry which represents a strong interest for deepen optical characterization of photonics waveguides, or for other optical regimes characterized by spectral evolution of the field propagating along the waveguide.

Absorption spectroscopy and photoluminescence (PL) decay measurement count among the most used spectroscopic methods to characterize optical materials. Therefore, an extension of these techniques into the imaging mode – hyperspectral imaging (HSI) and fluorescence lifetime imaging (FLIM) – is immensely useful and commonly used. The measurement of transmittance HSI and FLIM is seldom combined in a single system due to the necessity to use very different working conditions. For HSI, high light intensity needs to illuminate a sample as its image is typically scanned across a slit. On the contrary, the most common FLIM technique, time-correlated single-photon counting (TCSPC), uses low-light conditions and laser beam scanning. We present a versatile multimodal microscopic system that combined transmittance HSI with FLIM using a single setup based on compressed sensing, namely on the so-called single-pixel camera (SPC). This method makes it possible to reconstruct an image by using only a fraction of the measurements necessary compared to the number of pixels. The use of the SPC concept allowed us to create multidimensional HS and FLIM maps, which are inherently matched. Moreover, their spectral and temporal resolution follows the resolution of the used spectrometer and TCSPC setup. Our system is also highly versatile with respect to image magnification and field of view. Owing to the use of a digital micromirror device, we can switch without any modification of the setup, from the large field of view (3.5 × 2.6 mm) to the imaging on the micrometer scale limited only by the microscope lens. We demonstrate the functionality of the setup in both HS and FLIM mode on the testing samples of fluorescent dyes, luminophore (LuAG:Ce), and the cells of Convallaria.

We used a simple setup for phase and fluorescence time-lapse imaging of thousands of cells in parallel directly in the incubator. These images allow us to make a population based cell-cycle analysis comparable to flow-cytometry, but also give us an access to the analysis on the level of individual cells followed over several cell cycles. We can observe a thousand cells in parallel. We show a FUCCI marked HeLa cell culture observed over several days directly in the standard incubator. We compare our analysis to the flow cytometer data and show that we can produce a statistically relevant time-resolved measurement.

Intensity diffraction tomography (IDT) is a 3D phase imaging technique that enables the reconstruction of the refractive indexes (RIs) and absorption of a sample. IDT targets biological imaging in a label-free manner using the optical variation within the sample and multiple tilted imaging to reconstruct the 3D map of RIs. However, standard IDT techniques reveal several drawbacks in terms of limited field of view and feasibility of imaging living samples in time-lapse conditions. We focused on time-lapse imaging of large sample (>100µm) without the need of large NA objective or immersion oil. The challenge created by the absence of the phase information (intensity only measurements) as well as the limited illumination angle (low NA due to low magnification) has been solved using a Beam Propagation Method (BPM) embedded inside a deep leaning framework, that we call “physical neural network”. This network layers are encoding the 3D optical representation of the sample. Besides, we included in the forward model the effect of the spherical aberration introduced by the optical interfaces, which gave a strong impact on measurements under oblique illumination in terms of 3D spatial resolution. Using this framework, we achieved 3D reconstructions of mouse embryos (>100µm) in time-lapse conditions over 7 days, observing the intrinsic embryonic development from single cell (low-scattering sample) to the blastocyst level (highly scattering sample). Such time-lapse yields quantitative information on the development and viability of embryos in view of the sub-cellular imaging capacities. Our technology opens up novel opportunities for 3D live cell imaging of whole organoids in time-lapse.

The self-assembly of native structural proteins into fibrils or amyloid spherulite-like structures is associated with a spectrum of neurodegenerative diseases. we have a new super-resolution method (REPLOM), to observe directly and in real-time the formation of insulin spherulites and quantify the existence and abundance of diverse morphologies as well as their heterogeneous growth kinetics. Our results surprisingly revealed insulin spherulite growth is not exclusively isotropic, but it may also occur anisotropically. Combined with Machine learning we associated growth rates to specific morphological transitions and provided energy barriers and the energy landscape for each aggregation morphology.

The inherited bone disease Osteogenesis Imperfecta (OI) caused by genetic mutation can lead to brittle bones that fracture easily and serious skeletal deformities due to collagen defects. To understand this disease, in vitro models of 2D monolayer fibroblast cultures obtained from subdermal skin biopsies were developed by worldwide research groups, and nowadays were extended to 3D models to further mimic this disease. The models allow for studying of the collagen formation and possible abnormalities in extracellular matrix (ECM) and will enable studies of the mechanisms underlying defects in the quality and quantity of collagen fibres in OI skin fibroblasts when compared to the healthy cells. The most common methods for measuring the components in fibroblasts cultures are collagen immunostaining, which is time consuming and prohibits time-lapse measurements, or second harmonic generation microscopy (SHG) which only enables visualization of collagen fibres but not fibroblasts. Here by our label-free higher harmonic generation microscopy with 1050nm / 1070nm laser, we have extended SHG to with third harmonic generation microscopy (THG) to visualize fibroblasts cell morphology and their nuclei, also with multiphoton autofluorescence (MAF) signal from the cytoplasm of the fibroblasts. Our techniques allow for the visualisation of the 3D cultures in real-time, resulting in the possibility of making time-lapse videos. For both OI patient and control fibroblast cultures, fibroblasts and their nuclei were clearly visualised by forward THG signal; the collagen fibres by the forward SHG channel, and the cytoplasm signal with the backward autofluorescence channel. First results on the comparison between cultures of OI patients and controls reveal differences in fibroblasts orientation, collagen orientation and density.

We present a new method to achieve digital autofocus in holography. This method relies on the insertion of calibrated beads into the studied sample. Reconstructing the position and the radius of the beads using Inverse Problems Approach, based on Mie Model, makes it possible to accurately locate the slide on which the sampled is placed. Numerical focusing can then be performed using the standard backpropagation method or regularized reconstruction. Because the reconstruction plane can be chosen objectively with respect to the position of the slide, this numerical autofocus is reproducible whatever is the type of the observed biological sample.

The real-time positioning of an object on a microscopic scale is a significant challenge and remains difficult to apply. Many traditional imaging techniques exist but their axial resolution and/or their measurement range is often limited. We develop a novel high‐profile technology based on three pillars to meet these challenges. Using digital holography, we determine the correct focus distance on a large scale. Secondly, a new generation transformer neural networks processes the hologram giving in real-time (~30 frames per seconds) a submicrometric axial resolution, exceeding therefore the diffraction limit of the depth of field. Finally, the spatial structuring of the object allows us a nanometric lateral positioning by classical techniques, which will be sped up by a machine learning technique. Such high frame rates enable real-time processing in many different application scenarios.

Compressive sensing (CS) is a new acquisition technique that can potentially open the way to multi- and hyper-spectral imaging in wide spectral regions with a simplified optical scheme. In this framework, we are studying different approaches applied to Earth observation in different contexts. We describe the design of a vis-NIR telescope, where the image sampling is performed in its focal plane with a Digital Micromirror Device (DMD), and the hyperspectral acquisition is made possible by a compact spectrometer with a multispectral detector. We show the approach for a pushbroom multispectral acquisition from space, working in both the visible and thermal bands, to monitor the status of the cryosphere. Finally, we describe a compact wideband system working in the 0.4- 2.5 micron spectral range, based on three parallel spectrometers, where the acquisition is performed with a pushbroom scanning and a rotating coding mask. Opportunities and constraints given by the CS approach in these different contexts are highlighted.