This PDF file contains the front matter associated with SPIE Proceedings Volume 11785 including the Title Page, Copyright information, and Table of Contents.

Near-eye displays (NEDs) for augmented and virtual reality (AR/VR) are spotlighted because they have a possibility to provide much more immersive experiences never possible before. With the virtue of recent progress in sensors, optics, and computer science, several commercial products are already available, and the consumer market is expanding rapidly. However, there are several challenging issues for AR and VR NEDs to become closer to our lives. Here, we will explore these issues and important topics for AR and VR, and introduce some of the ideas to overcome them: diffractive optical elements (DOEs), retinal projection displays, and 3D display with focus cues. First, unlike VR with a simple optical system, AR that needs to merge an artificial image with an outer scene requires additional optics. The diffractive elements have the merit of being thin and transparent, suitable for the image combiner. Among them, holographic optical elements (HOEs) have great potential as they can record the desired volume grating from the simple lens to the complex wavefront using light interference. Second, in order to wear the NEDs for a long time, it must deal with the visual fatigue as well as the form factor. Retinal projection display can effectively prevent the vergence-accommodation conflict problem even with a simple optical design. In the retinal projection display, the light rays from the display are adjusted to converge into a small point using a lens. It ensures a wide depth range in which the images are clearly visible. Furthermore, it is possible to provide observers with accurate focus cues for the alleviation of visual fatigue via multi-layer displays and holographic displays. Recently, we conceived tomographic NED that can reproduce dense focal planes. We confirm that this system provides quasi-continuous focus cues, semi-original contrast, and considerable depth of field. The experimental results of our prototypes are explained. We also explain the recent activities of using deep learning in holographic NED system.

In the framework of the non-destructive structural health monitoring we have developed an innovative combination of remote and in situ geophysical techniques aimed at assessing the structural properties of the infrastructure (i.e. eigenfrequencies, equivalent viscous damping factors and related modal shapes) and its interactions with foundation soils both in static and dynamic conditions. We have applied and validated this approach on the Gravina bridge, a bow-string bridge located on outcropping calcarenites in the city of Matera (Southern Italy) that develops for 144 m along a single-span steel-concrete deck. The non-invasive, non-destructive and cost-effective seismic and electromagnetic techniques have been fine-tuned for a multi-sensing monitoring at a multi-scale and multi-depth levels (i.e. with different degrees of spatial resolution and different soil subsurface depths). The foundation soil characteristics have been evaluated by means of three high-resolution geo-electrical tomographies, one bi-dimensional seismic array processed through Extended Spatial Auto Correlation (ESAC) and two single station seismic noise measurements analyzed by means of Horizontal to Vertical Noise Spectral Ratio (HVNSR). The main structural characteristics of the bridge have been estimated through permanent and on-demand monitoring by using seismic and electromagnetic sensing. The great potential of this approach concerns the synergy, correlation and integration of different geophysical sensing to determine the main static and dynamic structures/infrastructures parameters and their interaction with soil. Moreover, the convergence of results obtained by different technologies and independent data sets constitutes a validation procedure of the presented approach.

The integration of load tracking systems for structural health monitoring of road infrastructures is a big leap towards future smart cities, allowing efficient policy making for sustainable development and maintenance, also, supporting fast decision taking with respect to unforeseen events. To facilitate deployment, the load sensing system should be low-cost, easily applicable to any type of road infrastructure, highly durable, and require minimum maintenance. In this context, the authors introduce a novel weigh-in-motion (WIM) system for load sensing and vehicle characterization. The integration of the proposed WIM system with proper signal processing techniques enables the real-time alert of traffic conditions, for example overloads. The sensor is a smart road pavement that generates voltage signals upon the passage of a vehicle. The voltage signals are generated through the piezoresistive effect of the material produced by the inclusion of carbon micro fibers. These signals are processed to generate information about vehicle weight, speed, and class. Preliminary results from field tests show the promise of this technology at WIM sensing.

The LBM (Lattice Boltzmann Method) is often used in CFD (Computational Fluid Dynamics) for efficient fluid flow simulations. Computation of the permeability of a porous media from direct simulations is a common application which benefits from the ability of the LBM (Lattice Boltzmann Method) to embed porosity parameters. The MM (Mathematical Morphology) is widely used in image processing as the theoretical aspects guaranty robust algorithms for geometrical characterization of shapes appearing in images. The MM is commonly used to compute porosity from porous media images. The union of these two methods has been recently done through the LB3M (Lattice Boltzmann Method for Mathematical Morphology). The present work extends the LB3M to the extraction of porosity and pores segmentation from images. In order to benefit from the full capacity of the LB3M, it is necessary to reformulate and adjust the algorithms in a new paradigm. Thus, the underlying concept and algorithms required for computing the different previous information are detailed. Moreover, a comparison is provided between the permeability resulting from the CFD and MM both implemented by using the LBM. To sum up, this work emphasizes the full capacity of the LB3M to obtain complex transformations and operations issued from the MM theory through completely new and innovative algorithms. The herein challenge is to highlight the abilities of the LB3M to match with physical phenomenons. Indeed, the LB3M keeps the advantages from the MM such as a complete theory, fast convergence, scalability, robustness, etc. while adding the power of the LBM: statistical physics origins, partial differential equation solver, intrinsic properties of parallelization, efficiency, etc.

In the last years, there is an increasing attention to the management and monitoring of critical infrastructure with a particular focus on the transport ones. In fact, several infrastructures have experienced (especially bridges) significant safety criticalities, which in few cases led to collapse with significant social and economic impacts. In this context, Structural Health Monitoring (SHM) is necessary in order to provide information, which can support the strategies for increasing the life-time of the structure, correctly planning maintenance interventions, so to ensure safety conditions of the infrastructure under any environmental condition and hazard occurrence. At the sight of the above considerations, at present, significant research efforts are on-going towards development of SHM systems able to get the opportunity of technological advances in sensing and, at the same time, to be economically sustainable. In particular, there is the necessity to design and implement integration strategies, where the modern sensing technologies, coming from several research fields, are combined with the usual tools and models exploited in civil engineering. Therefore, the present contribution deals with a unified approach to the design and development of systems for early warning, monitoring and quick damage assessment of the built environment and critical infrastructures and networks. In this context, the integration is the key factor for exploiting the synergy among different kind of sensing/diagnostics technologies (including new concept of operation, such as the citizen as a sensor and sensors no sensors) and advanced Spatial Data infrastructure and ICT architectures . It is also important to assimilate monitoring data and indicators coming from the sensing into civil engineering; this is crucial to identify actions and strategies for an effective and economically sustainable management of the infrastructure. In addition, the integrated SHM has to be designed and implemented according to several stages exploiting different sensing technologies, which are organized according to a temporal and logical workflow characterized by different levels of observation and knowledge, where the information gained at a level is used to decide whether to activate the next level. At conference, several examples of the proposed integrated SHM approach will be presented, with a focus on transport infrastructure and built environment.

Many civil engineering infrastructures including bridges and buildings have been constructed several years ago. They are facing increasing challenges due to climate change and exposed to various external loads such as earthquakes. Extensive research works have been carried out to enable structural and health monitoring (SHM). Modal identification is a crucial part of SHM. Conventionally, it has been accomplished by accelerometers mounted on the structure. Their use may be extremely accurate. However, only a few sensors is usually set up on the structure which may limit modal identification and SHM performance. Vision-based techniques gained increased acceptance as cheaper and easier solution to perform long-range vibration measurements. Video cameras offer the capacity to collect high-spatial resolution data from a distant scene of interest. Various image processing techniques have been developed to extract motion from subtle time changes in the image brightness. Commonly, these motions are then used for modal identification. Instead, this paper explores the use of pixels intensity variations only to perform SHM. Our new approach can be divided in two parallel steps. The first one deals with processing video image flow to effectively selecting the 'active pixels' or pixels belonging to the structure edges. The pixel selection process relies on the power spectral density and an energy criterion. Second, stochastic subspace identification based method that takes into account uncertainty bounds for modal parameters is adopted. So, vision modal parameters from the vision data are recovered. Comparison of identification results with motion signals is finally studied. For that, we have assessed a methodology for extracting motions based on cross-correlation analysis and Taylor-based subpixel refinement. Experiments in a laboratory setting on a cantilever beam are performed to verify the approach. The beam was imaged by a high speed camera and excited on a shaking table.

Research in various natural ecosystems has had a greater interest in recent years, for an evaluation, detection, and measurement of the changes that occur between the ecosystem and the species that inhabit it, all with the aim of knowing if these changes are consequences natural or the result of deliberate or unintentional human interventions. For this reason, different tools have begun to be used to carry out a detailed analysis of some of these places and, therefore, of certain species. An example of the implementation of these tools is the use of radio frequency identification systems (RFID) for species monitoring, which has been used to control the number of species in a specific area. However, the RFID systems that are implemented in these studies are usually placed in fixed places outdoors, also, no data has been found on the network on a detailed analysis of the behavior of RFID readers when submerged in water. Therefore, the interest of this research is to perform an analysis of the RFID low-frequency readers in two different test scenarios, with the aim of being able to implement them for the monitoring of marine species, which are affected using frequencies. higher than 150kHz that cause damage to their nervous tissues and temporary loss of hearing. For this research, two test scenarios are proposed, to make a comparison between both scenarios and observe the feasibility of being able to use RFID readers and passive tags in environments where they must be submerged in water.

In this paper, an approach for the estimation of relative positions and orientations of multiple rigidly coupled RGB-D sensors is presented. The proposed methodology does not impose constraints on the relative pose among the cameras, except for the rigidity: relative poses among cameras do not change in time. We assume that each camera captures an individual sequence which is afterward processed by a Simultaneous Localization And Mapping (SLAM) algorithm resulting in positions and orientations describing the trajectory followed by each sensor. Several simulations have been run considering a two-camera set having known input trajectories, affected by increasing levels of white noise. The results prove the capability of the algorithm in estimating the relative translations and rotations between the two cameras. Furthermore, experimental laboratory tests corroborate the effectiveness of the proposed method.

This paper proposed an efficient method to provide a robust occupancy grid useful for robot navigation tasks. An omnidirectional indoor robot accomplishing logistics tasks, has been equipped with stereocameras for detecting the presence of moving and fixed obstacles. The stereocamera provides a 3D point cloud. Starting from the tridimensional information, the occupancy map can be computed. Nevertheless, the point cloud often owns unstable points mainly due to low accurate disparity map and to light reflections on the floor that produce mismatching during the stereo matching phase. The point cloud has been opportunely filtered by using a cascade approach in order to get more robust occupancy grids. Passthrough filters are applied to remove the too far 3D points. Since high reflective floors produce unwanted 3D points, a color filter is also used to remove those points having saturated intensity values. The remaining floating points related always to the floor are then filtered out by taking advantage of the knowledge about the camera tilt. At this stage, a preliminary 2D occupancy grid is built to sample the point cloud. Each bin of occupancy map is then processed. In case the cell under investigation contains points, a distribution analysis about the point spread is performed. If the height of the highest point is under a determined threshold value, the cell value is set to zero. The unwanted floor points are thus furtherly removed. The cells containing a low number of points are also cleared. Finally, the isolated cells of occupancy grid and the cells that do not have enough valid neighboring cells are reset. The noisy points and the edge points of objects do not concur to produce inaccurate occupancy maps. Final outcomes prove as the proposed methodology enables to provide robust occupancy maps ensuring high performance in terms of processing time.

Composites represent the evolution of the material science and technologies. They are obtained by combining two or more materials of different nature with the aim of exploiting any synergies between the characteristic performances of the raw materials. Their properties, in fact, are influenced by those of the starting components but also by the quality of the interface generated between the combined phases as well as by their mutual distribution. The interphase, even if of minimal extension with respect to the main phases constituting the composite, plays a significant role in the control of the damage mechanisms, determines the breaking strength and the stress / deformation behavior of composite materials. In this work we study the effect of low pressure plasma treatment on intrinsically hydrophilic flax fiber fabrics to improve their adhesion to a hydrophobic polypropylene matrix. The fibers are treated using nitrogen (N₂) plasma with four different exposure times. The interfacial adhesion actually achieved was indirectly quantified by interlaminar shear strength measurements. After this, the damaged areas were measured with non-destructive techniques, i.e. Electron Speckle Pattern Interferometry and Lock-in thermography.

The contactless estimation of vital signs based on conventional color cameras and ambient light can be affected by motions of patients and changes in ambient light. In this work, a multimodal 3D imaging system with an irritation-free controlled illumination was developed in order to mitigate these both problems. In the developed system the real-time high-precision 3D imaging is combined with VIS-NIR multispectral imaging and thermal imaging. Based on 3D data and color images, an approach was proposed for the efficient compensation of head motions, and novel approaches based on 3D regions of interest were developed for the estimation of heart rate, oxygen saturation, respiration rate, and body temperature from NIR multispectral video data and thermal video data. A proof-of-concept for the developed imaging system and algorithms can be delivered with first experimental results.

Work-related musculoskeletal disorders are causing occupational diseases. While allowing them to control the execution of their picking task, one solution to physically relieve workers performing trunk flexions is to assist them with an exoskeleton. This preliminary study focuses on the determination of the appropriate measurements systems (motion capture and electromyography) to characterize dedicated trajectories of movements responsible of lower back pathologies. Three tests have been realized with and without a back-assisting exoskeleton for loaded box picking. The results of the study help us to understand which body strategies were used to perform the same task from one individual to another under different conditions. This preliminary study validates this multimodal approach to show the use of a back-assisting exoskeleton for posture harness in dynamic conditions.

Achieving precise information on the position of a subject without changing the luminaire infrastructure is a big challenge in positioning approaches that rely on visible light positioning. Achieving high positioning accuracy on a centimeter scale is done by implementing complex receiver unit designs or adapting the existing luminaries. In this context, we suggest a visible light positioning based approach that can determine the position of a person in certain areas of a room without the need of lighting infrastructure modifications. With this approach, one can identify the position with the help of the existing luminaires for the obligatory room lighting. The receiver, represented by a RGB sensitive photodiode, is positioned in an optimized way in order to support both the positioning task as well as the comfort of the user. Based on received signal strength measurements in the red, green and blue channels, we achieve the positioning task by a segmentation of the room into different areas corresponding to the respective impinging light and by utilizing machine learning clustering. Our results show the influence of different segmentation strategies and parameters on the number and size of the distinguishable areas inside the room. Then, we demonstrate the achievable accuracy of our solution approach in real world experiments. Our results show that such light-based positioning data can be fused with IMU sensor data for recognizing human activity.

imultaneous 3D scene reconstruction and semantic segmentation are required in many applications such as autonomous driving, robotics, and optical metrology. Classic 3D reconstruction methods usually perform such operations twofold. Firstly, a 3D scanner or laser scanner acquires a point cloud. Secondly, semantic segmentation of the point cloud is performed. Recently a new kind of 3D model representation was proposed that utilizes the trapezium-shaped voxels that are aligned with the camera’s frustum and pixels [1]. Frustum voxel models proved to be effective for monocular 3D scene reconstruction and segmentation from monocular images [2]. Still, many existing 3D scanning systems readily provide stereo cameras. The performance of frustum voxel model-based methods for stereo input remains an open question. This paper is focused on the evaluation of the 3D reconstruction quality of a volumetric neural network with a monocular and stereo input. We leverage an SSZ [2] volumetric neural network as a starting point for our research. We develop its modified version that we term Stereo-SSZ that receives a stereo pair as an input. We compare the performance of the original SSZ model and our Stereo-SSZ model on different real and synthetic 3D shape datasets. Specifically, we generate a stereo version of the SemanticVoxels [2] dataset and capture stereo pairs of multiple real objects using a structured light scanner. The results of our experiments are encouraging and demonstrate that the model with a stereo input outperforms the original monocular SSZ network. Specifically, the frustum voxel models generated by our Stereo-SSZ model have lower surface distance errors and demonstrate fine details in the reconstructed 3D models.

Nowadays, thanks to the development of new advanced driver-assistance systems (ADAS) able to help drivers in driving tasks, autonomous driving is becoming part of our lives. This massive development is mainly due to the great possibilities of guaranteeing higher safety levels that these systems can offer to vehicles that every day travel on our roads. At the heart of every application in this area, there stands the environment’s perception, guiding the vehicle’s behavior. This counts in autonomous driving field and all the applications characterized by a system that moves in the 3D real-world like robotics, augmented reality, etc. For this purpose, an effective 3D perception system is necessary to accurately localize all the objects that compose the scene and reconstruct it in a 3D model. This issue is often faced using LIDAR sensors, which allow an accurate 3D perception offering high robustness in unfavorable light and weather conditions. But these sensors are generally expensive, and thus do not represent the right choice for low-cost vehicles and robots. Moreover, they need to be used in a particular position that does not permit integrating it on the car changing both the appearance and the aerodynamics. Besides, their output is a point cloud data that, due to its structure is not easily manageable with deep learning models that promise outstanding results in various similar predictive tasks. For these reasons, in some applications, it is better to leverage other sensors like RGB cameras to estimate 3D perception. For this purpose, more classic approaches are based on stereo-cameras, RGB-D cameras, and stereo from motion, which generally can reconstruct the scene with less accuracy than LIDARS, but still produce acceptable results. In recent years, several approaches have been proposed in literature which aim to estimate the depth from a monocular camera leveraging deep learning models. Some of these methods use a supervised approach,1, 2 however they mainly rely on annotated datasets which in practice can be labor-expensive to be collected. Thus, some works3, 4 use, on the contrary, self-supervised training procedure leveraging reprojection error. Notwithstanding their good performance, most of the proposed approaches use very deep neural networks, that are power- and resource- consuming and need high-end GPUs to produce results in real-time. For these reasons, these approaches cannot be used in systems with power and computational limits. In this work, we propose a new approach based on a standard CNN proposed in the literature to deal with the image segmentation problem created not to be highly resource-dependent. For the training, we used the knowledge-distillation method using an out of shell pre-trained network as teacher network. We execute large scale experiments to qualitatively and quantitatively compare our results with those obtained with baselines. Moreover, we propose a deep study of inference times using both general-purpose and mobile architectures.

Technological progress in uncooled infrared focal plane array sensors has contributed significantly to enlarge the scope of applications of such sensing technique in many domains: Leisure, Manufacturing, Process Survey, Building insulation diagnostic, Civil Engineering, Road works, etc… Different outdoor situations and objects of interest monitored by two in-house designed measurement architecture are presented. Measurements correction from varying environmental conditions and geometrical considerations are discussed. A first step toward joint estimation of emissivity and temperature is introduced. Finally conclusion and perspectives are proposed.

Nowadays the importance of Unmanned Aerial Vehicle (UAV) based sensing technologies is globally recognized. Indeed, thanks to the ability of investigating large areas in a very short time and at very reduced cost, the UAV sensing technology has been widely used in multiple application contexts, including security and surveillance inspections, environmental monitoring, geology, agriculture, archeology and cultural heritage. Actually, the widespread remote sensing technologies mounted on-board UAVs are mainly optical, thermal and multi-spectral sensors, which are passive technologies designed to measure the signals emitted into the optical and (near and far) infrared portions of the electromagnetic spectrum thus providing useful 2D and 3D information about the observed scene. Radar systems represent an important complementary solution. Indeed, radar system is an active system which transmits and receives electromagnetic signals at microwave frequencies, thus offering the advantages of performing inspections in free space and through-obstacle scenarios. However, UAV based radar imaging is not yet a well consolidated technology due to the significant challenges related to the acquisition modality and data processing strategies. Since both transmitting and receiving radar units must be installed on-board the UAV, this introduces not trivial issues related to payload and assets constrains. Moreover, in order to obtain reliable and easily interpretable images, a high precision UAV trajectory reconstruction must be deployed. As a contribution to this topic, an UAV imaging system prototype based on a microwave tomographic approach was recently proposed. Experimental tests at the Archaeological Park of Paestum (SA) has been recently carried out. During the survey, the UAV platform was piloted in path-planning mode, i.e. “autonomous flight” on a predefined rectangular grid and a novel imaging strategy which exploits multiple measurement lines has been developed.

Differential Synthetic Aperture Radar Interferometry (DInSAR) represents by now an established tool for the remote monitoring of the ground displacements occurring on our planet. In addition, this technique is benefiting from the recent advancements in radar, navigation and aeronautical technologies. These indeed allow us on one side to avail of wide-coverage SAR satellites to observe larger and larger areas of the Earth’s surface. On the other side, they make it possible to profit of cost-efficient aerial SAR systems, whose operative flexibility can play a crucial role to observe phenomena possibly hidden to satellite sensors and for the prompt monitoring in emergency scenarios. In this work, we provide the results of a ground deformation analysis performed over the municipality of Vazante, Brazil, based on the DInSAR processing of the data acquired by both the space-borne C-band Sentinel-1A and the drone-borne multi-frequency (P/L/C-bands) DBSS SAR systems.

Diatoms are one of the largest groups of microalgae present in marine, freshwater and transitional environments and their reactivity to environmental changes makes them suitable to be employed as biomarkers for monitoring tasks. Anyway, their presence in a large number of species makes it arduous to perform diatoms taxonomy during monitoring tasks considering that, to date, analysis is conducted by marine biologists on the basis of their own experience and, hence, in a subjective way. Hence, the need for automatic and objective methodologies for the identification and classification of diatoms samples rises. Research efforts in the field of Computer Vision led to a plethora of highly effective deep learning strategies surpassing human capabilities for image classification, as showed in the recent Imagenet challenge editions where they were initially introduced. Despite the very promising results of the proposed solutions, the difficulty arises to determine which technique is most suitable among them for real tasks and in particular for diatoms classification. This work proposes an end-to-end pipeline for automatic recognition of diatoms, acquired by means of holographic microscopy in water samples, employing deep learning techniques. In particular the most recently introduced Convolution Neural Networks (CNNs) architectures have been deeply investigated and compared in order to highlight the pros and cons of each of them. Moreover, in order to feed the CNNs training stages with a suitable amount of labeled data, a strategy to build a synthetic dataset, starting from a single image per class available from commercial glass slides specifically prepared for taxonomy purposes, is introduced. Besides, models ensembling strategies, in order to improve the single model scores, have been exploited. Finally, the proposed approach has been validated employing a dataset built up of holographic images of diatoms sampled in natural water bodies.

The application of Deep Learning Neural Networks to Object Detection grants the localization of object instances from several classes within images. The inference of the locations of the items is not always precise and the positional error of bounding boxes can result being in the pixel range. This research aims at analysing the object localization uncertainty of Deep Convolutional Neural Networks (DCNNs), by considering MVTec HALCON and open-source Neural Networks. The metrological outcomes are presented. Since matching algorithms are still used to identify shapes, contours and edges, a comparison with them sounds like an obligation. The results of both Deep Learning and Shape-Based Matching (SBM) algorithms are compared with a sub-pixel precise reference labelling. The labelling consists of specific bounding boxes around the objects of each image of the dataset. It was obtained manually performing rectangular boxes on a single image and propagating them to the entire dataset according to the camera pose, precisely determined by keeping the target fixed and moving the camera with a measuring machine. The intent of this work is to establish the best detection methodology with respect to different testing conditions, considering also the effect of possible Gaussian noise and imprecise labelling.

In this study we analyzed deep learning methods for point clouds semantic segmentation. We compared PointNet and PointNet++ on data with different characteristics, coming from distinct domains, in order to understand their behavior. Finally, we exploited the so gained knowledge to improve the performance of the models on railway data. In particular, we properly updated the training protocol and altered the PointNet++ architecture, in order to perform transfer learning by leveraging the models previously trained in the first experiments. Results on both state-of-the-art datasets and on a custom dataset specifically acquired for this scope demonstrate that transfer learning can effectively boost the performance of the models in terms of prediction accuracy and convergence rate in the railway context.

Wheel-terrain interaction plays a critical role for vehicle mobility on natural terrain, such as in agricultural, planetary exploration and off-road settings. Estimation of the terrain characteristics and the way they affect traversability is essential for the vehicle to better plan its safest and energy-efficient path. This work proposes a novel approach to learn and predict from a distance the motion resistance encountered by a robotic vehicle, while traversing natural soil, by using visual information from a stereovision device. To this end, terrain appearance and geometry information are first correlated to resistance torque measurements during a learning phase via two alternative regression approaches, namely Least-Squares Boosting and Long-Short Term Memory Recurrent Neural Network. Then, such a relationship is exploited to predict motion resistance remotely, based on visual data only. Results obtained in preliminary experimental tests on ploughed and compact terrain are presented to show the feasibility of the proposed method.

Plant phenotyping is the complete evaluation of complex plant traits such as its growth, development, tolerance or resistance, measured on the basis of quantitative and individual parameters of the plant itself. The evaluation process should be automated and non-destructive, suggesting computer vision as a key enabling technology to perform this task. In this paper, we propose a computer vision software pipeline for the analysis of the roots system of a plant. Two main contributions are provided: first, a deterministic procedure to assemble a roots panorama image starting from multiple shots of a rotating rhizotron; second, the automatic extraction of a binary mask representing the observed roots in the image. Results on more than 20.000 RGB images demonstrate the robustness and feasibility of our approach, reporting 77% median sensitivity and 99% median specificity in the roots segmentation task. This study can be seen as the first step towards the automation of labelled data to be used in complex deep learning architectures devoted to higher level applications, such as the automatic data-driven feature extraction as well as high-throughput applications.

Laser cleaning based on laser ablation processes has the potential to remove dirt, over painting and other surface contaminants without touching the surface and without using any chemicals. It is already widespread in the semiconductor, automotive, and aerospace industries due to its lower costs, higher throughput and reduced environmental and toxicity problems compared to other cleaning methods. Currently, portable hand-held laser cleaning is the mostly used device for coating removal. The advantages of the portable hand-held laser cleaning lie in flexibility and convenience of use. However, with this mode of operation, it is not possible a precise control of the geometric parameters of the ablation process (focal length, angle of incidence, etc) or others such as the scanning speed, which are decisive to ensure effective, controlled and reproducible cleaning, especially in the case of 3D workpieces. In this sense, it must be said that laser processing of 3D surfaces (cleaning, micromachining, etc) is not a completely solved issue nowadays, and most of the proposed solutions are based on quasi 3D or 2.5D approximations. In this paper it is presented a laser cleaning robot system for the processing of 3D surfaces. It is based on the integration of a 6-degree-of-freedom (DOF) robotic arm with a laser source and a three-dimensional surface capture system to automatically generate the laser cleaning path. The manipulator allows displacements and changes of orientation of the object to be processed following the programmed paths in a full 3D solution. Furthermore, the hardware and software to coordinate the robot and the laser on/off was developed. Experimental results show the effectiveness of the proposed method which improves the efficiency of the automatic laser cleaning of 3D objects and further promotes the application to other ablation processes.

Currently, wind turbine blades are designed following a safe-life paradigm where manufacturing defects and operational damages are not taken into account. However, industry competition steers innovation towards typical solutions to maximize the economical output. The harsh environment, especially for offshore wind turbines, significantly degrades the turbine structural integrity and its performance. One of the major factors that lowers energy production due to aerodynamic losses is the erosion of the leading edges. Nowadays this erosion is monitored visually, either directly or with drone-based camera systems. We expect a quantitative characterization of the current erosion state and long term degradation monitoring to provide crucial information for wind farm maintenance and possible recoating of wind turbine blades. In this project, a customized long-range laser line scanner is used to measure the profiles of the blades at distances of up to 2 m. Individual scans are assembled into a 3D point cloud which is processed and analysed for defects and erosion. Further in the project, a custom-developed heavy drone will be used to scan the blades offshore. Current results include the detection of eroded areas, impact damages and surface defects in laboratory conditions with the sensor positioned with a scanning frame. Two processing approaches, namely deterministic and machine learning are compared and benchmarked with an ambition to perform erosion detection at a submillimeter scale. The deterministic approach includes local analysis of the surface variation, thresholding and clustering of the defects. Present controlled conditions allow investigation of how predicted drone vibrations, possible water droplets and salt contamination affect the inspection in a real offshore environment. This development is done within the smart maintenance innovation project AIRTuB: Automatic Inspection and Repair of Turbine Blades where the inspection expertise is combined with drone and wind-turbine manufacturers to change the turbine design paradigm from safe life to damage-tolerant.

Contactless, non-destructive testing has always been an important pillar in crucial tasks performed in industrial applications, including post-assembly testing oriented toward aircraft manufacturing. This work will examine the topic from the point of view of the quality check for the lining of aircraft interiors, such as sidewalls and hatracks, with the aim of improving safety of operation and comfort during flights, using an automatic approach guided by the usage of computer vision system. In particular, it will present a multimodal approach using a 3d snapshot sensor and a color camera, for identifying defects and anomalies that belong to two distinct categories, such as geometrical and surface defects, where, in the aircraft manufacturing sectors, due to the low-volume production, when compared with the automotive industry, such quality check operation tasks are still done manually, with a low level of automation. The proposed approach is showing potential and has been demonstrated on a proof-of-concept system prototype funded through a European Commission Horizon 2020 research project under the Clean Sky 2 umbrella aimed at the growth of the aviation sector.

Automatic 3-D recovery from multimodal images can be extremely useful for information extraction for the robot navigation application. In most cases, such a scene contains missing holes on depth maps that appear during the synthesis from multi-views. This paper presents an automated pipeline for processing multimodal images to 3-D digital surface models. The proposed approach uses the modified exemplar-based technique in quaternion space. We also perform depth completion by fusing data from multiple recorded multimodal images affected by occlusions. We propose an algorithm using the concepts of a sparse representation of quaternions, which uses a new gradient to calculate the priority function by integrating the structure of quaternions with local polynomial approximation - the intersection of confidence intervals). Moreover, the color information incorporates into the optimization criteria to obtain sharp inpainting results. Compared with state-of-the-art techniques, the proposed algorithm provides plausible restoration of the depth map from multimodal images, making them a promising tool for an autonomous robot navigation application.

Calibrating the kinematic parameters of a mobile robot is a time consuming and mandatory procedure, since the mechanical tolerances and the assembly procedures may introduce a large inaccuracy in the nominal parameters. A small error in the calibration might lead to severe inconsistencies in tasks that rely on precise positioning, such as localization, mapping and navigation in general. In a wheeled mobile robot this consists of estimating the odometry parameters, that are required to convert wheel encoder ticks in a relative motion of the mobile base on a local plane, for then computing its position. To tackle this problem, we propose the use of the unscented Kalman filter for estimating the geometrical kinematic parameters of the dynamic model of the mobile robot, while using an external camera, to precisely measure the mobile robot positions in the environment, via a visual tag fixed to the robot.  The inclusion of the visual tag in the model corresponds to an eye-to-eye problem, where the fixed transform between the visual tag frame and mobile robot frame is identified. The usage of the unscented version of the Kalman filter, compared to the extended Kalman filter, assures robustness in the estimation of parameters and state in strongly nonlinear system, and to parameters initial values, due to the usage of the unscented transform to propagate the state uncertainty, rather than employing linearization. We validate this method on a 4 mecanum-wheel mobile robot internally developed using a Kinect2 to track the mobile robot movement trough a reference chessboard. The experiments has shown a great improvement in the odometry computation of the two platforms, and a fast convergence of the estimated parameters, furthermore the fixed transform of the visual tag, which corresponds to ah hand-to-eye problem is well identified

The increasing request of flexible robotic applications asks for systems more capable in perceiving and interacting with the environment, to adapt the robot behavior to possible changes. The challenge is the easy design of the robotic system around the application. This becomes crucial in particular when the objects to be manipulated are bulky, and the relative positions between the robot and the objects to be manipulated are uncertain and high precision is required to successfully complete a task. In this paper a possible guideline to design a system capable to localize itself, identify a target, bulky, object and manipulate it, is presented. Vision systems techniques and technologies are briefly analyzed in order to select the most suitable solution under restrictive constraints. The objects considered for this approach are indeed bulky, texture-less, without strong geometrical features, with uncertain positions and rotations in the 3D space and strict accuracy tolerances. Since the presented work involves object manipulation and assembly, also force feedback is necessary when interaction is involved. This allows the robot to perceive the interacting environment through touch and avoid possibly dangerous huge forces exchange. With this purpose, a method for tuning the impedance control parameters is derived, to keep interaction forces below maximum allowed values. The autonomous localization, grasping and assembly of a sidewall panel of an airplane is used as test. Such scenario involves different challenges for both the vision system and the manipulation, indeed the environment is very narrow and cluttered, lighting is variable, assembly tolerances are very tight (10 mm) considering huge objects (1000x1500 mm). Experiments show that the success rate of completing a task increases, combining vision perception and force control, with respect to the single use of visual localization and position control. The coupled action of these two techniques makes the entire application more reliable.

The diagnostic of artworks represents a topic of high interest and at the same time a critical task. The development of effective tools based on advanced technologies for investigating and monitoring health-state of item of interest for cultural heritage is essential to assuring their preservation and restoration. In fact, detecting and preventing the formation of damages or assessing them for an accurate pre-restoration analysis are the main objectives of non-destructive investigation. In this study the combined use of Electronic Speckle Pattern Interferometry and Pulse Thermography Analysis, as complementary tools for cultural heritage artifacts diagnostics have been employed on poplar panels painted reproducing original painting methods adopted during the 16th and 17th centuries. Both techniques are full field, contactless, non-invasive and can provide relevant information about state of conservation. The concurrent investigations, allow to expand understanding the results of each individual technique, from which, in many cases, no exclusive assumption is possible. The applications include detection of detachments, micro-cracks, inclusions, and hidden damages. It is also possible the real time monitoring of the behaviour of the object according to the environment thermo-hygrometric changes. The porpoise of this work is to optimize the parameters of such NDT methods on the samples that were specially made by reproducing in high fidelity the structural properties and materials of the artwork. Results show that the employed diagnostic protocols, by speckle interferometry and pulse thermography, are a powerful tool in assessing the pre-restoration health-state and suitable for in situ analysis of wood artworks. Examples of analysis on different artworks are shown.

For decades, satellite and aerial imagery have been the central element of mapping the earth's surface. The need for fast and frequent imaging, especially of smaller areas with unmanned aerial vehicles (UAVs) is gaining importance in various industries. Regular monitoring of agricultural areas, forests, coastal regions and urban areas are of utmost interest. Using UAVs to examine the worlds surface allows high resolution imagery captured in short time intervals without high costs as with the use of airplanes. This time series monitoring allows a quick detection of changes in the study area. The FraunhoferInstitute for Physical Measurement Techniques IPM presents a compact and lightweight multispectral camera system. A 12 MP RGB-sensor and two 5.5 MP grayscale sensors are simultaneously capturing images. High frame rates, ultralightweight design, and the ability to easily adjust the aperture angle and wavelengths to be recorded open up many new applications. This ensures the use on a wide range of UAVs. Even with fast flying fixed wing UAVs, images can be acquired with high overlap. Further, we present results of a first measurement campaign. Further, we present results of laboratory tests and measurement campaigns.

The impact of climate change on snow cover evolution is evident. Increasing amounts of winter precipitation as well as rising temperatures are causing the winter snow cover to change more and more rapidly within one season. To quantify the direct effects on hydrological cycles, spatially and temporally high-resolution information on snow height and the amount of water stored as snow (Snow Water Equivalent, SWE) is required on watershed scales. This paper presents the concept for a novel airborne light detection and ranging (LiDAR) system combining high-resolution snow height mapping with co-registered spatial information on the water content of the snowpack. Based on the optical characteristics of snow, we outline a detailed plan for dual-wavelength LiDAR sensor working at wavelengths of 1030 nm and 515 nm. By comparing the intensities received in the two channels, snow cover parameters like the effective grain size can be inferred. By means of recent snow hydrological models, from these data and the topographic snow depth maps then high resolution SWE maps can be deduced. We supplement our outline with conceptual LiDAR snow depth and reflectance measurements using a commercially available system, pointing out the impact of view angle dependence of received intensity and general applicability for future airborne LiDAR surveys.