|
|
1.INTRODUCTIONIn recent years, the global warming situation has become more and more serious, relevant studies[1] show that there is a positive correlation between economic growth and carbon dioxide emissions, how to reduce carbon emissions while developing the economy has become a major challenge facing the world in the future. Studies have shown that forestry is an important option for reducing carbon emissions, free savings of 50 PPMV equivalent to 14 degrees Celsius by 2100, meeting the same carbon limits in other ways would cost an additional $3 trillion[2],Forest protection is an important way to alleviate global warming. On the one hand, human beings need to increase the intensity of forest protection, on the other hand, we need to improve the maximum and reasonable utilization of forest resources. Compared to the production of plastics, concrete, steel and other materials, wood production and processing is very energy efficient. In the process of logs production and processing, the logs are divided according to different caliber specifications, which can improve the utilization rate of wood, thus improving economic and ecological benefits. With the rapid development of computer hardware level, computer image processing ability has been greatly improved, image processing algorithm is changing from the original traditional image processing technology to artificial intelligence technology. The application of artificial intelligence technology in the forestry production process can promote the improvement of production efficiency and the high-quality development of forestry industry[3]. However, at this stage, the production, sales and transportation processes of wood are not highly intelligent and informative. For example, wood volume as a basic economic indicator of wood management and utilization, the accuracy of wood volume calculation directly affects the interests of both sides of the transaction. At present, in order to get the volume of the whole vehicle logs, the vast majority of enterprises to calculate the volume of the scheme is usually the manual way to measure the root by root. This method is inefficient, and the volume accuracy can not be guaranteed due to subjective factors. Some scholars have tried to develop a smarter, faster and more accurate method to replace the manual scale check. Sprechgesang N et al.[4] proposed a method combining SSD model and FCN semantic segmentation model to achieve vehicle-mounted eucalyptus image segmentation. Yella et al.[5] introduced a method that uses photos taken by drivers, then uses color information and geometric operators in multiple color Spaces to segment images and extract relevant information, and finally implements automatic detection, counting and classification of wood by Circular Hough Transform (CHT) algorithm. The algorithm is robust to external factors such as lighting conditions and camera angles, but it is poor at identifying logs with large color differences caused by mud or snow cover. Yang Pan et al.[6] used the Mask R-CNN instance segmentation model to recognize the wood end face images collected from the vehicle. The above scholars have done a lot of work in the field of wood detection and segmentation and achieved considerable experimental results, but they have not conducted relevant research on how to obtain high-definition log images. Small differences in log diameter measurements can determine the grade of the log, which can affect the price and economic losses of the industry. In order to solve the problem of log diameter measurement, Budiman F et al.[7] built a simple handheld device with a fixed length iron rod, camera and raspberry PI. They put the iron rod against the end face of the measured log and used the camera to collect images. Then the image is transferred to the Raspberry PI for compression, gray transformation, contour search, circle fitting and a series of operations to get the measured wood diameter. The scheme proposed in this paper has the advantages of easy portability and easy operation, and the error of the proposed method is controlled within 3%. However, this method needs to be used with LED light source, and the robustness of the system is insufficient when the experimental environment is under natural lighting conditions. For machine vision system, the image quality directly affects the image processing results. The main factors affecting image quality are illumination, exposure time and gain, resolution, target distance and focal length[8]. Usually after the hardware design of machine vision system is completed, the light source, lens and focal length have been fixed and will not be easily changed. Therefore, in order to obtain high-quality target images under natural light conditions, the “gain” and “exposure time” of the camera are generally set to compensate for the unstable ambient light conditions. However, with the rise of machine learning technology, the robustness of the algorithm can be improved by adding training samples under different lighting conditions, and the influence of lighting on image quality is not particularly important. In target tracking applications, the object distance (the distance from the target to the camera) is constantly changing and the focal length is fixed. The camera’s image is clear only when the objects are at the right distance. Relevant studies[9] show that images with different definition have a great impact on the error between the measurement results of the target algorithm and the truth value, so judging the target distance has become the key to obtain high-quality images in the application of target tracking and detection. 2.THE SYSTEM DESIGN2.1System framework designIn traditional automatic industrial detection, the camera image acquisition scheme can be divided into target image acquisition based on sensor and target image acquisition based on detection algorithm according to the different target detection methods. The target image acquisition based on sensor need external sensors, photoelectric sensors, ultrasonic sensor, laser sensor, etc.), when external sensors detected signals triggered when the object to be tested to the camera, the camera a frame target image was taken from the received signal, the last PC camera image data read again for subsequent image processing[10]. In the application of fixed focal length industrial camera, this acquisition scheme can accurately control the distance between the target and the camera, so as to obtain high-quality clear images. Compared with software trigger method, the process of image acquisition using hardware trigger method based on sensor is much shorter and has high real-time performance. However, the sensor is prone to interference from the outside environment. For example, when non-target objects pass by, the sensor can also trigger the signal, resulting in invalid pictures taken by the camera. Relative to the target image acquisition method based on sensor, based on the detection algorithm of target image acquisition mode does not require an external transducer, PC directly read the camera’s video streaming, through each frame of video image in image processing, judge whether the presence of target objects, if there is a goal to save the current image[11]. This image acquisition method directly analyzes and determines the images obtained by the camera through detection algorithm to save the images, which is not easy to save invalid target images due to the interference of non-target objects, and is easy to deploy. But this method requires both robustness and real-time performance of the algorithm. At the same time, the target position information detected by this method is two-dimensional, and the distance between the target and the camera cannot be obtained. Therefore, it is impossible to judge whether the target object is on the focal length, and the image resolution obtained is not the highest. Based on the above scheme, in order to obtain a high-quality image containing the target with a certain clarity without stopping the volume vehicle, this paper intends to use the method of combining laser ranging sensor and Yolov3 deep learning target detection model to achieve it. The experimental device of the system is shown in the figure, which consists of a laser ranging sensor, a 200W industrial camera and a Xilinx AXU3EG development platform. The system can detect the image in real time. When the wood end face of the vehicle appears, the laser ranging sensor is activated to obtain the distance between the target and the camera, and the image is saved when the target is on the focal length of the camera. The system consists of a ZYNQ chip, a USB camera and a laser ranging sensor. The ARM Cortex-A53 on the ZYNQ chip acts as the core of the task scheduling and coordinates the data transfer. First, ARM reads the image data of USB camera, and then sends the image data to PL terminal for YOLOv3 inference to judge whether there is a whole vehicle log. At the same time, open the laser ranging sensor to read the distance between the end face of the vehicle log and the camera to judge whether the end face of the vehicle log is on the ideal focal length. Finally, the car log end face image that meets the requirements is saved to the hard disk. 2.2System workflow1) System initialization: connect the laser ranging sensor module, set the detection range to 10 meters, set the starting point of measurement to the top, and open the continuous measurement mode; 2) Connect the camera; 3) Loading images; 4) Call DPU API; 5) Judge whether there is a vehicle log end face in the collected image, if so, perform step 6; otherwise, go back to Step 3 to continue image detection of the next frame; 6) Read the data cache of the laser ranging module to obtain the target distance; 7) Judge whether the target distance is within the best shooting distance, if so, perform step 8; otherwise, go back to Step 3 to continue image detection of the next frame; 8) Save the image. 3.PRINCIPLES OF KEY MODULES3.1Principle of laser moduleIn order to obtain high-definition vehicle wood end face image, distance parameters (depth information) from the target to the camera lens need to be obtained. At present, the existing methods to obtain depth information include binocular camera ranging, infrared depth camera ranging, ultrasonic ranging, laser ranging and so on. The principle of binocular ranging is to directly measure the distance of the target by using the parallax between two images of the same target[12]. This method requires pixel by pixel matching of two images, which requires a large amount of computation and high complexity, so it is not suitable for real-time systems. The ranging method based on infrared depth camera[13] is easy to receive the influence of ambient light, the measurement distance is unstable, and it is not suitable for outdoor scenes. The ranging method[14] based on ultrasonic wave is slow in measuring speed and requires that the measured object must be planar, so it is not suitable for the measurement of the distance of the end face of the complex and rough wood on the vehicle. Based on the laser ranging method, because it is not easy to be interfered by the sun, high precision, detection fast break, suitable for rough surface objects and other advantages, is the ranging scheme adopted in this paper. Phase laser measurement is realized by modulating laser intensity and calculating the phase difference between the intensity of the transmitted light and the received light[15]. By measuring the phase difference, the target distance can be calculated, as shown in the specific formulas (1) and (2) (d stands for target distance; c is the propagation speed of light in air: 3×108 m/s; Δt is the total time required for laser propagation from the transmitting end to the receiving end; φ is the total phase difference generated by a transmission of laser from the transmitting end to the receiving end; is the part less than in the laser transmission from the transmitting end to the receiving end; is the number of full-cycle wavelengths in a transmission from the transmitting end to the receiving end; is the period involved in the modulation of light; represents the frequency involved in modulating the light): To improve the detection speed of the system, a predictive volume approach is used to reduce the impact of laser ranging on the overall system time consumption. The specific steps are first the host computer sends a continuous measurement command to the laser rangefinder before performing the target detection, then the rangefinder automatically performs the distance measurement and saves the latest distance measurement results into the cache. When the host computer needs distance data, it can directly read the results in the cache without waiting for a whole measurement cycle. 3.2Application of target detection modelDue to the improvement of hardware computing power, the performance of target detection models has been greatly improved, and they are widely used in industrial inspection. Two-stage target detection algorithms are mainly classified into one-stage and two-stage. two-stage target detection algorithms first generate a series of sample candidates and then classify them by convolutional neural networks. Typical two-stage network models include Fast R-CNN, R-FCN and FPN. The one-stage target detection algorithms such as YOLOv1[16], YOLOv2[17], SSD[18], and YOLOv3[19] can obtain the category probability and location coordinate values of the object in just one calculation, which is faster than the two-stage target detection algorithms and is suitable for detection scenarios with high real-time performance[20]. In this paper, yolov3 is chosen as the detector responsible for processing the input video stream. Compared with yolov2, yolov3 adopts darknet-53 as the backbone network, while borrowing from the Resnet network to set the residual structure, which can avoid the gradient disappearance phenomenon, thus enhancing the learning ability of the network[19]. yolov3 outputs feature maps at three scales of 13×13, 26×26 and 52×52, corresponding to the detection of large, medium and small targets, respectively. targets and small targets, respectively. yolov3 divides the images into S × S grids, and each grid needs to detect three anchor boxes, each of which contains coordinate information (horizontal coordinate, vertical coordinate, width, height), confidence and class information using one-hot encoding, so its output is constructed as S × S × 3 × (5 + class_number)[21]. In the detection task of this paper, only one class of vehicle wood end face needs to be identified, so the feature maps of 13×13×18, 26×26×18, and 52×52×18 are output on the three scales. In this paper, we choose to deploy the yolov3 model on the edge hardware platform of xazu3eg-sfvc784-1-i of Xilinx ZYNQ series. The floating-point model weight file in .pb format is first exported after the training of the yolov3 neural network is completed, and then the floating-point model is quantified using the quantization tool provided by Xilinx and the test set data before importing this parameter information. Then the DPU Kernel is compiled, which requires the appropriate DPU configuration file and hardware architecture description file. After the kernel is compiled, the .elf file is cross-compiled to generate a .so dynamic link library file and copied to the SD card. The .so file contains the yolov3 kernel, and the yolov3 inference can be performed by running the driver of the corresponding application on the target board to get the target detection results. 4.EXPERIMENT4.1Experimental environmentThe application scenario of this system is to perform high quality image acquisition work on the end face of a full truckload of logs loaded on a timber volume vehicle. Figure a is a picture of logs loaded on site in a forestry field. In order to increase the stability of the whole vehicle, let the whole vehicle mass center as much as possible in the middle, so in the log loading is divided into two layers, the lower layer of wood small diameter facing outward, the upper layer of wood large diameter facing outward. The experimental environment was built according to the actual situation of log loading on site as shown in Figure b. The system hardware platform is built according to the method designed in this paper, as shown in the figure, which is mainly composed of (1) development board, (2) laser ranging module and (3) camera. (1) The development board adopts Xilinx ZYNQ series XAZU3EG-SFVC784-1-I edge hardware platform, which can be divided into two parts, PS (Processing System) and PL (Programmable Logic). Data is transmitted between PS and PL through the Advanced Extensible Interface (AXI) protocol. At the same time, the development board also provides a large number of peripheral interfaces, such as USB, DP (Display Port), gigabit Ethernet, Micro SD, CAN communication interface and 485 communication inter-face, this paper only used the USB3.0 interface, DP interface and SD card. (2) It is a laser ranging sensor module with a measurement frequency of 50Hz and an accuracy of less than 1cm. It can exchange data with the development board through USB serial port. (3) It is a COMS industrial camera with 200W pixels and a 6mm focal length lens. The development board reads the image data of the camera through USB. 4.2Experiment and analysis of laser ranging moduleAs shown in Table 1, are the test data of the laser ranging sensor module adopted in this paper. When the actual distance is 0.4m, 1M, 2m, 3m, 4M, 5m and 6m, the measurement data is read for ten times respectively. It can be seen from the data that the maximum relative error between the measured value and the theoretical value is not more than 0.3%, the measurement distance is more than 6 meters, and the measurement accuracy is ±1cm, which meets the requirements of the system for accuracy. Secondly, the maximum relative error measured by the sensor at 4M and 5m is the smallest, and the optimal distance of the sensor is between 4 and 5 meters. Therefore, the object distance is assumed to be 4.5m when determining the shooting distance between the target and the camera in this paper. Table 1.Test data of laser ranging module.
4.3The training of object detection modelThis paper uses a Window laptop with Intel I5-8300H CPU and NVIDIA GeForce GTX 1050Ti graphics card. Software configuration is Python3.6 programming language, tensorflow1.6.0 deep learning framework, Cuda9.0 accelerator. A total of 1230 original target images taken under different weather conditions were collected in this paper, including 850 on sunny days, 280 on cloudy days and 100 on rainy days. The 1230 images were divided into training set and validation set with a ratio of 9:1. According to Pascal VOC2007 standard format, this paper uses labelimg software to annotate the data set, get the coordinates of the target area and put preset labels, and finally get the XML file. Because this paper only identifies the car wood end face, there is only one type of preset label, named woods_face. In the training stage, first of all, to facilitate the model training, the image needs to be converted to 416*416 before data input to the model. Then, 4 images were used as a batch size for mini-batch training, and the momentum value was set as 0.9. In order to speed up the training time of the model, two learning rates are used to train the model. By default, 200 iterations of the training model and the first 50 iterations of the network are carried out, and the training learning rate is 1E-3. After that, the learning rate is 1E-4 for subsequent training. In order to prevent overfitting, the iteration is terminated in advance when the loss curve of the training set decreases and the loss curve of the validation set rises. The training loss curve is shown in the figure. It can be seen that the learning rate of the first 50 iterations adopts the 0.001 model for rapid convergence. The loss curve of the training set decreases to about 35, and the loss of the validation set decreases to about 40. After that, the loss appears unstable oscillation, indicating that the model no longer converges. Then, the learning rate of 0.0001 is used for training, and the loss decreases again. When the training reaches 111 times, the loss of both the training set and the validation set will not decrease, and will be stable at about 7. At this time, the iteration will be terminated in advance to prevent overfitting. In the test phase, 100 pictures were collected again for testing, including 30 pictures taken on cloudy days, 60 on sunny days, and 10 on rainy days. The test results were as follows. From the test of the pictures taken on cloudy days, sunny days and rainy days, the recognition rate of the pictures taken on sunny days was the highest 100%, followed by 96.7% on cloudy days, and the worst was on rainy days. The proportion of overcast, sunny and rainy days in the training set is 22.8%, 69.1% and 8.1%, indicating that the sample size of the training set affects the final recognition rate. In order to improve the recognition rate of wood face in cloudy and rainy days, it is necessary to increase the corresponding sample size. Table 2.Identification effect of vehicle wood face under different environment.
4.4System testThe camera video stream obtained by the system through the USB serial port is deduced by the YoloV3 target detection model, and the result is obtained. At this time, there are two kinds of target detection results. In the first case, there is no vehicular wood end face in the current frame image, and “Nok” is displayed in red at the upper left corner of the image. In the other case, there is a car wood end face in the current frame image, and the green “OK” character is displayed in the upper left corner of the picture. The serial port reads the cache information of the optical ranging sensor and obtains the distance between the wood end face and the camera. By judging whether the distance meets the condition of focal length, the image is saved. The focal length is set as 4.5m. When the target distance is between 4.4m and 4.6m, it meets the focal length range, as shown in the figure below is the overall test result of the system in the experimental environment. A) The target distance read by the laser ranging sensor is 5.13m, not within the range of 4.4m to 4.6m (red), and no Nok is detected on the wood face (red); B) The target distance read by the laser ranging sensor is 4.52 m, in the range of 4.4 m to 4.6 m (green), and no Nok is detected on the wood end face (red); C) The target distance read by the laser ranging sensor is 6.71 m, not within the range of 4.4 m to 4.6 m (red), and the wood end face shows OK (green); D) The laser ranging sensor reads the target distance of 4.5 m, in the range of 4.4 m to 4.6 m (green), and detects the wood end face showing OK (green). The system can save the image only when the target distance meets the requirements and the end face of the vehicle wood is detected. At this time, the saved image has the highest definition. Two images (c) and d) containing wood end faces of different clarity in Figure 8 were used for wood counting, and the model inference results obtained are shown in Figure 9. The first figure shows the detection results of fuzzy images obtained by using only the object detection algorithm. A total of 74 recognition boxes are detected, among which 71 are really wood recognition boxes and 3 are misidentified ones. The second figure shows the detection results of the clear wood end image obtained by the proposed method. A total of 100 recognition frames are detected, among which 97 are real wood and 3 are misidentified. Compared with the fuzzy image in the first figure, the number of real wood roots identified in the image obtained by the method designed in this paper increases from 71 to 97, and the recognition effect is improved by 36.6%. 5.CONCLUSIONIn this paper, in order to solve the problem of how to accurately collect high-resolution images of vehicle log face without using a higher resolution camera in natural scenes, a method based on the combination of laser ranging sensor and YOLOV3 target detection model algorithm is proposed. Compared with other image acquisition schemes, the scheme described in this paper can accurately capture target images in real time and avoid capturing other non-target images. At the same time, without using a higher resolution camera, by judging whether the distance from the target to the camera meets the focal length requirement, the captured image can have the highest definition. REFERENCEZheng Z.,
“Re-calculation of responsibility distribution and spatiotemporal patterns of global production carbon emissions from the perspective of global value chain,”
Science of The Total Environment, 773 145065
(2021). https://doi.org/10.1016/j.scitotenv.2021.145065 Google Scholar
Tavoni M, Sohngen B, Bosetti V.,
“Forestry and the carbon market response to stabilize climate,”
Energy Policy, 35
(11), 5346
–5353
(2007). https://doi.org/10.1016/j.enpol.2006.01.036 Google Scholar
Ma X, Chen X, Geng Y.,
“Regional forestry economic evaluation based on neural network and fuzzy model,”
Journal of Intelligent and Fuzzy Systems, 40
(8), 1
–12
(2020). Google Scholar
Samdangdech N, Phiphobmongkol S.,
“Log-End Cut-Area Detection in Images Taken from Rear End of Eucalyptus Timber Trucks,”
1
–6
(2018). Google Scholar
Yella S, Dougherty M.,
“Automatically Detecting the Number of Logs on a Timber Truck,”
Journal of Intelligent Systems, 22
(4), 417
–435
(2013). https://doi.org/10.1515/jisys-2013-0026 Google Scholar
Yang Pan, Zheng Jishi, et al.,
“Research on detection and segmentation methods of dense-stacked logs using Mask R-CNN,”
Journal of Forestry Engineering, 7
(02), 135
–142
(2022). Google Scholar
Budiman F, Mardiyanto R, Tasripan, et al.,
“A handy and accurate device to measure smallest diameter of log to reduce measurement errors,”
in 2016 International Seminar on Intelligent Technology and Its Applications (ISITIA),
(2016). https://doi.org/10.1109/ISITIA.2016.7828697 Google Scholar
Zhou Z, Liang J, Duan S, et al.,
“Research on Autofocus Recognition of the LAMOST Fiber View Camera System Under Front and Back Illumination,”
Publications of the Astronomical Society of the Pacific, 134
(1032), 025001
(2022). https://doi.org/10.1088/1538-3873/ac4e1e Google Scholar
Ke F, Liu H, Zhao D, et al.,
“A high precision image registration method for measurement based on the stereo camera system,”
Optik, 204 164186
(2020). https://doi.org/10.1016/j.ijleo.2020.164186 Google Scholar
Palander T.,
“Utilization of Image, LiDAR and Gamma-Ray Information to Improve Environmental Sustainability of Cut-to-Length Wood Harvesting Operations in Peatlands: A Management Systems Perspective,”
ISPRS International Journal of Geo-Information, 10
(2021). Google Scholar
Ravindran P, Owens F C, Wade A C, et al.,
“Field-Deployable Computer Vision Wood Identification of Peruvian Timbers,”
Frontiers in Plant Science, 12
(647515), 1
–8
(2021). Google Scholar
Xu H, Xia L, Zhu C, et al.,
“A Real-Time Ranging Method Based on Parallel Binocular Vision,”
in 2017 10th International Symposium on Computational Intelligence and Design (ISCID),
(2017). https://doi.org/10.1109/ISCID.2017.33 Google Scholar
Gu T, Zhao H, Sun S.,
“Depth Estimation of Single Infrared Image Based on Interframe Information Extraction,”
Laser & Optoelectronics Progress,
(2018). Google Scholar
Niepiaowski K.,
“Impact of the Selected Disturbing Factors on Accuracy of the Distance Measurement with the use of Ultrasonic Transducers in a Hard Coal Mine,”
Energies, 15
(2021). Google Scholar
Qi K, Song Z, Dai J S,
“Safe physical human-robot interaction: A quasi whole-body sensing method based on novel laser-ranging sensor ring pairs,”
Robotics and Computer-Integrated Manufacturing, 75 102280
(2022). https://doi.org/10.1016/j.rcim.2021.102280 Google Scholar
Redmon J, Divvala S, Girshick R, et al.,
“You only look once: Unified, real-time object detection,”
in Proceedings of the IEEE conference on computer vision and pattern recognition,
779
–788
(2016). Google Scholar
Redmon J, Farhadi A.,
“YOLO9000: better, faster, stronger,”
in Proceedings of the IEEE conference on computer vision and pattern recognition,
7263
–7271
(2017). Google Scholar
Liu W, Anguelov D, Erhan D, et al.,
“SSD: Single Shot MultiBox Detector,”
(2015). Google Scholar
Redmon J, Farhadi A.,
“Yolov3: An incremental improvement,”
(2018). Google Scholar
Zhang W, Zeng Z.,
“Research Progress of Convolutional Neural Network and its Application in Object Detection,”
(2020). Google Scholar
Redmon J, Farhadi A.,
“YOLOv3: An Incremental Improvement,”
(2018). Google Scholar
|
