In this undergraduate research project, we use LiDAR mapping for object detection and further combine AI and computer vision algorithms to enable robots to safely drive the vehicle in a given environment. AI and computer vision technologies allow the robot to identify lanes and intersections, enabling vehicle navigation, while LiDAR mapping quickly and accurately determines the depth between the vehicle and objects entering a specific area. This capability allows the robot to temporarily stop the vehicle, preventing collisions with objects. Through these technologies, our goal is to prevent collisions that may occur during driving, ensuring pedestrian safety and enabling safe robot-driven vehicle operation in crowded places. Simulation and test have been conducted to verify the proposed methods.
KEYWORDS: Image restoration, 3D image processing, Integral imaging, 3D image reconstruction, 3D metrology, Reconstruction algorithms, MATLAB, 3D displays, Signal to noise ratio, Roads
In this undergraduate research work, we present studies on computational volumetric reconstruction and depth detection approaches. Computational integral imaging algorithm could provide 3D reconstructed images which include both infocus and out-of-focus pixels. Using two image analysis indicators, peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM), we discussed the accuracy and performance of 3D depth detection by comparing 2D captured images with 3D reconstructed images along a depth range.
This paper presents an overview of previously published reports on three-dimensional (3D) Profilometry for visualization of objects using Integral Imaging (InIm). The method was initially proposed for imaging in free space to map color, depth, and texture of multiple objects on a 3D surface. The advantage of 3D profilometric reconstruction versus conventional InIm planar reconstruction is that the former can optically slice the entire object depth within a range of voxels whereas the latter presents the depth of a single plane. Later, this method was studied for flexible sensing integral imaging for occlusion removal. This work which combined two-view geometry for camera pose estimation and 3D occluded object recognition demonstrated that 3D profilometric reconstruction not only can mitigate the effects of occlusion but is also perceptually better than conventional InIm reconstruction.
In this review paper, we present a previously proposed approach for the estimation of degree of polarization under low illumination conditions. To avoid the saturation problems, zeros from the denominator of the Degree of polarization calculation are excluded which changes the distribution of the photon detection from Poisson to a Truncated Poisson distribution. 3D polarimetric imaging experiments had been conducted via light field under low illumination environments to verify the proposed approach.
KEYWORDS: Integral imaging, 3D image processing, Reconstruction algorithms, 3D image reconstruction, Image restoration, 3D surface sensing, Three dimensional sensing, Image sensors, Optical sensing, Lens arrays
In this undergraduate research work, we studied 3D optical sensing and volumetric computational reconstruction algorithm based on conventional integral imaging and axially distributed sensing architectures. Imaging sensors are distributed along x-y-z dimensions for multi-perspective sensing, corresponding reconstruction algorithms are developed for volumetric visualization. Experiments were conducted to verify the feasibility of 3D sensing system and developed algorithms.
Accurate object detection and depth estimation is critical for a variety of applications such as autonomous driving and robotics. In the context of object avoidance, one may use a LiDAR sensor to determine the position of nearby objects but, due to a lack of resolution, these sensors cannot be used to accurately categorize and label the object being detected. To contrast this, RGB cameras can provide rich semantic information, which can be used to categorize and segment an object but cannot provide accurate depth data. To overcome this, an abundance of algorithms has been created which are capable of fusing the two sensors, among others, allowing for accurate depth detection and segmentation of a given object. The problem with many of these systems is that they are complex in their approach and create 3D bounding boxes, which can result in an agent taking a less optimal path due to the size of the perceived object. The proposed approach in this paper simply determines the position of an object in an RGB image, using a CNN, and then translates two dimensions, found through the center pixel of the bounding box, to a point cloud to identify and segment point clusters.
In this paper, we develop an algorithm for calibrating a camera array for 3D image reconstruction and depth detection. The developed algorithm uses the functionality of MATLAB’s Computer Vision Toolbox and allows for the calibration of any number of cameras. The resulting intrinsic and extrinsic parameters (x, y, and z position; focal length of the lens; and pixel size of the sensor) will be formatted and saved into a matrix for further computations. This increases the efficiency when conducting 3D image reconstruction and depth detection. The accuracy and precision of these calculations were experimentally verified using simulated scenes created in Autodesk 3Ds Max. Future work includes: adding functionality to make use of rotation parameters, testing with real-world data sets, and modification to increase the accuracy and efficiency of calculations.
KEYWORDS: 3D image processing, Integral imaging, 3D image reconstruction, Visualization, 3D displays, Three dimensional sensing, Reconstruction algorithms, Algorithm development, Cameras, Stereoscopy
We have developed a project-based learning approach with the aim of teaching, education, and undergraduate research in optics and photonics. The proposed project-based learning process is focused on the development of hands-on experiments with 3D light field integral imaging technologies. The research projects enable our undergraduate engineering school students with different levels and majors to gain a deep understanding to optics and photonics through early research experience and student-faculty engagement.
We overview our recently published multi-dimensional integral imaging-based system for underwater optical signal detection. For robust signal detection, an optical signal propagating through the turbid water is encoded using multiple light sources and coded with spread spectrum techniques. An array of optical sensors captures video sequences of elemental images, which are reconstructed using multi-dimensional integral imaging followed by a 4D correlation to detect the transmitted signal. The area under the curve (AUC) and the number of detection errors were used as metrics to assess the performance of the system. The overviewed system successfully detects an optical signal under higher turbidity conditions than possible using conventional sensing and detection approaches.
KEYWORDS: Polarimetry, 3D image processing, Cameras, Integral imaging, Signal to noise ratio, Polarization, Stereoscopy, Sensors, Imaging systems, Image sensors
We overview a previously reported three-dimensional (3D) polarimetric integral imaging method and algorithms for extracting 3D polarimetric information in low light environment. 3D integral imaging reconstruction algorithm is first performed to the originally captured two-dimensional (2D) polarimetric images. The signal-to-noise ratio (SNR) of the 3D reconstructed polarimetric image is enhanced comparing with the 2D images. The Stokes polarization parameters are measured and applied for the calculation of the 3D volumetric degree of polarization (DoP) image of the scene. Statistical analysis on the 3D DoP can extract the polarimetric properties of the scene. Experimental results verified the proposed method out performs the conventional 2D polarimetric imaging in low illumination environment.
We overview a previously reported method for spatial-temporal human gesture recognition under degraded environmental conditions using three-dimensional (3D) integral imaging (InIm) technology with correlation filters. The degraded conditions include low illumination environment and occlusion in front of the human gesture. The human gesture is captured by passive integral imaging, the signal is then processed using computational reconstruction algorithms and denoising algorithms to decrease the noise and remove partial occlusion. Gesture recognition is finally processed using correlation filters. Experimental results show that the proposed approach is promising for human gesture recognition under degraded environmental conditions compared with conventional recognition algorithms.
KEYWORDS: 3D image reconstruction, 3D image processing, Signal to noise ratio, Cameras, Integral imaging, Photons, Visualization, Optical sensors, Facial recognition systems, Sensors
We overview a recently published work that utilizes three-dimensional (3D) integral imaging (InIm) to capture 3D information of a scene in low illumination conditions using passive imaging sensors. An object behind occlusion is imaged using 3D InIm. A computational 3D reconstructed image is generated from the captured scene information at a particular depth plane, which showed the object without occlusion. Moreover, 3D InIm substantially increases the signal-to-noise ratio of the 3D reconstructed scene compared with a single two-dimensional (2D) image as readout noise is minimized. This occurs due to the 3D InIm reconstruction algorithm being naturally optimum in the maximumlikelihood sense in the presence of additive Gaussian noise. After 3D InIm reconstruction, facial detection using the Viola-Jones object detection framework is successful whereas it fails using a single two-dimensional (2D) elemental image.
KEYWORDS: 3D image processing, Cameras, 3D image reconstruction, Integral imaging, 3D surface sensing, 3D modeling, Object recognition, Sensing systems, Nonlinear dynamics, Visualization
We overview a previously reported method for three-dimensional (3D) profilometric reconstruction with occlusion removal based on flexible sensing integral imaging. With flexible sensing, the field-of-view of the image system can be increased by randomly distributing a camera array on a non-planar surface. The camera matrices are estimated using the captured multi-perspective elemental images, and the estimated matrices are used for 3D reconstruction. Object recognition is then implemented on the reconstructed image by nonlinear correlation to detect the 3D position of the object. Finally, an algorithm is proposed to visualize the 3D profile of the object with occlusion removal.
We present recent progress of the previously reported Multidimensional Optical Sensing and Imaging Systems (MOSIS) 2.0 for target recognition, material inspection and integrated visualization. The degrees of freedom of MOSIS 2.0 include three-dimensional (3D) imaging, polarimetric imaging and multispectral imaging. Each of these features provides unique information about a scene. 3D computationally reconstructed images mitigate the occlusion in front of the object, which can be used for 3D object recognition. The degree of polarization (DoP) of the light reflected from object surface is measured by 3D polarimetric imaging. Multispectral imaging is able to segment targets with specific spectral properties.
We overview a previously reported head tracking integral imaging three-dimensional (3D) display to extend viewing angle accommodated to a viewer’s position without the crosstalk phenomenon. A head detection system is applied to obtain the head position and rotation of a viewer, and a new set of elemental images is then computed using the smart pseudoscopic-to-orthoscopic conversion (SPOC) method for head tracking 3D display. Experimental results validate the proposed method for high quality 3D display with large viewing angle.
In this paper, a three-dimensional (3D) integral imaging display for augmented reality is presented. By implementing the pseudoscopic-to-orthoscopic conversion method, elemental image arrays with different capturing parameters can be transferred into the identical format for 3D display. With the proposed merging algorithm, a new set of elemental images for augmented reality display is generated. The newly generated elemental images contain both the virtual objects and real world scene with desired depth information and transparency parameters. The experimental results indicate the feasibility of the proposed 3D augmented reality with integral imaging.
KEYWORDS: 3D displays, 3D image processing, Integral imaging, Imaging arrays, Displays, 3D image reconstruction, Tablets, Image processing, Image quality, Distortion
In this paper, we present a technique to generate an elemental image array to match display devices for three dimensional integral imaging. Experimental results show that our technique can be used to accurately match different display formats and improve the display results.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.