Unsupervised embedding learning aims to learn highly discriminative features of images without using class labels. Existing instance-wise softmax embedding methods treat each instance as a distinct class and explore the underlying instance-to-instance visual similarity relationships. However, overfitting the instance features leads to insufficient discriminability and poor generalizability of networks. To tackle this issue, we introduce an instance-wise softmax embedding with cosine margin (SEwCM), which for the first time adds margin in the unsupervised instance softmax classification function from the cosine perspective. The cosine margin is used to separate the classification decision boundaries between instances. SEwCM explicitly optimizes the feature mapping of networks by maximizing the cosine similarity between instances, thus learning a highly discriminative model. Exhaustive experiments on three fine-grained image datasets demonstrate the effectiveness of our proposed method over existing methods.

Edge-preserving image smoothing plays a vital role in the field of computational imaging. It is a valuable technique that has applications in various tasks. However, different tasks have specific requirements for edge preservation. Existing filters do not take into account the task-dependent smoothing behavior, resulting in visually distracting artifacts. We propose a flexible edge-preserving image filter based on a nonconvex Welsch penalty. Compared with the convex models, our model can better handle complex data and capture nonlinear relationships, thus providing better results. We combine deep unsupervised learning and graduated nonconvexity to solve our nonconvex objective function, where the main network structure is designed as a Swin transformer complemented with the locally enhanced feed-forward network. Experimental results show that the proposed method achieves excellent performance in various applications, including image smoothing, high dynamic range tone mapping, detail enhancement, and edge extraction.

Crowd counting aims to derive information about crowd density by quantifying the number of individuals in an image or video. It offers crucial insights applicable to various domains, e.g., secure, efficient decision-making, and management. However, scale variation and irregular shapes of heads pose intricate challenges. To address these challenges, we propose a scale-deformation awareness network (SDANet). Specifically, a scale awareness module is introduced to address the scale variation. It can capture long-distance dependencies and preserve precise spatial information by readjusting weights in height and width directions. Concurrently, a deformation awareness module is introduced to solve the challenge of head deformation. It adjusts the sampling position of the convolution kernel through deformable convolution and learning offset. Experimental results on four crowd-counting datasets prove the superiority of SDANet in accuracy, efficiency, and robustness.

Real-world image denoising is a critical task in image processing, aiming to restore clean images from their noisy counterparts captured in natural environments. While diffusion models have demonstrated remarkable success in image generation, surpassing traditional generative models, their application to image denoising has been limited due to challenges in controlling noise generation effectively. We present a general denoising method inspired by diffusion models. Specifically, our approach employs a diffusion process with linear interpolation, enabling control of noise generation. By interpolating the intermediate noisy image between the original clean image and the corresponding real-world noisy one, our model is able to achieve controllable noise generation. Moreover, we introduce two sampling algorithms for this diffusion model: a straightforward procedure aligned with the diffusion process and an enhanced version that addresses the shortcomings of the former. Experimental results demonstrate that our proposed method, utilizing simple convolutional neural networks such as UNet, achieves denoising performance comparable to that of the transformer architecture.

The most common form of image forgery is copy-move, which arises when an image region is duplicated and pasted onto another region of the same image. An effective algorithm for copy-move forgery detection based on binarized statistical image features (BSIF) and principal component analysis (PCA) is presented. Initially, the suspicious image is converted to grayscale and is subsequently partitioned into overlapping blocks. Feature vectors are extracted from these blocks using BSIF, followed by dimensionality reduction using PCA. Next, as a precursor to the matching step, the feature vectors are sorted lexicographically. Additionally, a morphological opening operation is applied to eliminate outliers. This algorithm offers not just forgery detection but also the ability to localize and identify duplicated regions. The proposed algorithm was assessed using three datasets: CoMoFoD, GRIP, and UNIPA. The experimental results show that this algorithm is fast and has high accuracy for forgery detection and localization. Moreover, it has high robustness under various postprocessing operations, such as brightness, contrast adjustments, and blurring. Furthermore, the proposed algorithm outperforms some recent approaches in overall performance.

Recognizing three-dimensional (3D) objects based on local feature descriptors is a highly challenging task. Existing 3D local feature descriptors rely on single-scale surface normals, which are susceptible to noise and outliers, significantly compromising their effectiveness and robustness. A multi-scale point pair normal encoding (M-POE) method for 3D object recognition is proposed. First, we introduce the M-POE descriptor, which encodes voxelized features with multi-scale normals to describe local surfaces, exhibiting strong distinctiveness and robustness against various interferences. Second, we present guided sample consensus in second-order graphs (GSAC-SOG), an extension of RANSAC that incorporates geometric constraints and reduces sampling randomness, enabling accurate estimation of the object’s six-degree-of-freedom (6-DOF) pose. Finally, a 3D object recognition method based on the M-POE descriptor is proposed. The proposed method is evaluated on five standard datasets with state-of-the-art comparisons. The results demonstrate that (1) M-POE is robust, discriminative, and efficient; (2) GSAC-SOG is robust to outliers; (3) the proposed 3D object recognition method achieves high accuracy and robustness against clutter and occlusion, with recognition rates of 99.45%, 94.21%, and 97.88% on the U3OR, Queen, and CFV datasets, respectively.

The utilization of chest X-ray (CXR) image data analysis for assisting in disease diagnosis is an important application of artificial intelligence. Supervised learning faces challenges due to a lack of large-scale labeled datasets and inaccuracies. Self-supervised learning offers a potential solution, but current research in this area is limited, and the diagnostic accuracy remains unsatisfactory. We propose an approach that integrates the self-supervised Bidirectional Encoder Representations from Image Transformers version 2 (BEiTv2) method with the vector quantization-based knowledge distillation (VQ-KD) strategy into CXR image data to enhance disease diagnosis accuracy. Our methodology demonstrates superior performance compared with existing self-supervised methods, showcasing its efficacy in improving diagnostic outcomes. Through transfer and ablation studies, we elucidate the benefits of the VQ-KD strategy in enhancing model performance and transferability to downstream tasks.

Video hashing is an efficient technique for tasks like copy detection and retrieval. This paper utilizes canonical polyadic (CP) decomposition and Hahn moments to design a robust video hashing. The first significant contribution is the secondary frame construction. It uses three weighted techniques to generate three secondary frames for each video group, which can effectively capture features of video frames from different aspects and thus improves discrimination. Another contribution is the deep feature extraction via the ResNet50 and CP decomposition. The use of the ResNet50 can provide rich features and the CP decomposition can learn a compact and discriminative representation from the rich features. In addition, the Hahn moments of secondary frames are taken to construct hash elements. Extensive experiments on the open video dataset demonstrate that the proposed algorithm surpasses several state-of-the-art algorithms in balancing discrimination and robustness.

In the current distributed environment, reversible data hiding in encrypted images has the disadvantages of low security and nonprogressivity. To address this problem, a homomorphic embedding algorithm is proposed based on polynomial secret sharing (PSS) and Chinese remainder theorem. First, the image owner encrypts the carrier image in streaming encryption and sends it to the data hider. Then, the data hider utilizes PSS to split the carrier image into n shares. At the same time, extra secrets after SS are embedded into the carrier shares using homomorphism. After splitting by Chinese remainder theorem, every share of the embedded data is divided into some sub-shares and then distributed to the participants. The participants that satisfy the threshold condition provide part or all of the sub-shares according to the authority of the data extractor. If each participant provides all sub-shares, the secrets and carrier image can be reconstructed completely. If each participant provides part of the sub-shares, the secrets and carrier image can be reconstructed partly. The experimental results show that the proposed scheme has progressivity, high security, and a large embedding rate (ER). Meanwhile, the ER is not affected by the carrier image.

The smooth regions in depth videos contain a significant proportion of homogeneous content, resulting in many spatial redundancies. To improve the coding efficiency of depth videos, this paper proposes a deep degradation-aware up-sampling-based depth video coding method. For reducing spatial redundancies effectively, the proposed method compresses the depth video at a low resolution, and restores the resolution by utilizing the learning-based up-sampling technology. To recover high-quality depth videos, a degradation-aware up-sampling network is proposed, which explores the degradation information of compression artifacts and sampling artifacts to restore the resolution. Specifically, the compression artifact removal module is used to obtain refined low-resolution depth frames by learning the representation of compression artifacts. Meanwhile, a jointly optimized learning strategy is designed to enhance the capability of recovering high-frequency details, which is beneficial for up-sampling. According to the experimental results, the proposed method achieves considerable performance in depth video coding compared with 3D-HEVC.

Radar imaging techniques, such as synthetic aperture radar, are widely explored in automatic vehicle recognition algorithms for remote sensing tasks. A large basis of literature covering several machine learning methodologies using visual information transformers, self-attention, convolutional neural networks (CNN), long short-term memory (LSTM), CNN-LSTM, CNN-attention-LSTM, and CNN Bi-LSTM models for detection of military vehicles have been attributed with high performance using a combination of these approaches. Tradeoffs between differing number of poses, single/multiple feature extraction streams, use of signals and/or images, as well as the specific mechanisms used to combine them, have widely been debated. We propose the adaptation of several models towards a unique biologically inspired architecture that utilizes both multi-pose and multi-contextual image and signal radar sensor information to make vehicle assessments over time. We implement a compact multi-pose 3D CNN single stream to process and fuse multi-temporal images while a dual sister 2D CNN stream processes the same information over a lower-dimensional power-spectral domain to mimic the way multi-sequence visual imagery is combined with auditory feedback for enhanced situational awareness. These data are then fused across data domains using transformer-modified encoding blocks to Bi-LSTM segments. Classification results on a fundamentally controlled simulated dataset yielded accuracies of up to 98% and 99% in line with literature. This enhanced performance was then evaluated for robustness not previously explored for three simultaneous parameterizations of incidence angle, object orientation, and lowered signal-to-noise ratio values and found to increase recognition on all three cases for low to moderate noised environments.

In recent years, underwater target detection algorithms based on deep learning have greatly promoted the development of the field of marine science and underwater robotics. However, due to the complexity of the underwater environment, there are problems, such as target occlusion, overlap, background confusion, and small object, that lead to detection difficulties. To address this issue, this paper proposes an improved underwater target detection method based on YOLOv8s. First, a lightweight backbone network with efficient transformers is used to replace the original backbone network, which enhances the contextual feature extraction capability. Second, an improved bidirectional feature pyramid network is used in the later multi-scale fusion part by increasing the input of bottom-level information while reducing the model size and number of parameters. Finally, a dynamic head with an attention mechanism is introduced into the detection head to enhance the classification and localization of small and fuzzy targets. Experimental results show that the proposed method improves the mAP0.5:0.95 of 65.7%, 63.7%, and 51.2% with YOLOv8s to that of 69.2%, 66.8%, and 54.8%, on three public underwater datasets, DUO, RUOD, and URPC2020, respectively. Additionally, compared with the YOLOv8s model, the model size decreased from 21.46 to 15.56 MB, and the number of parameters decreased from 11.1 to 7.9 M.

The detection of multiple faces in unconstrained environment in deep learning suffers from insufficient detection accuracy and inefficiency; at the same time, the detection of blurred, occluded, and very small faces is even more unsatisfactory. The detection of blurred, occluded, and very small faces in multiple face detection in unconstrained environment is a hard problem in face detection nowadays. It is difficult to balance the detection accuracy and real-time efficiency in face detection with the improved RetinaFace chosen in this study. Therefore, in order to improve the efficiency of detecting blurred, occluded, and very small faces among multiple faces in unconstrained environments, we introduce deformable convolution, feature pyramid networks (FPN), and coordinate attention (CA) attention mechanism based on RetinaFace algorithm. Deformable convolution can be dynamically adjusted according to the shape and deformation of the recognized object and is no longer limited to a fixed-size square receptive field to improve the image feature extraction capability of the convolutional layer. FPN enhances the feature semantic information of the lower layers with a small increase in computational effort and improves the robustness of the detection algorithm to detect targets of different sizes. CA is a novel, lightweight, and efficient attention mechanism module for improving model performance, which can be easily integrated into mobile networks to improve accuracy with little additional computational overhead. The improved ResRetinaFace algorithm does not increase the computational overhead too much while improving the recognition accuracy, and it can better combine the characteristics of multiple postures and deformations of faces in complex scenes, adapt to the deformation state of faces’ postures, and provide more effective features for face detection, so as to pay better attention to the detection target and enhance the network characterization ability. Meanwhile, the improved algorithm combines the feature pyramid with the context module, which improves the detection effect in the case of blurred, occluded, and very small faces. The experimental outcomes demonstrate that, in contrast to the method before enhancement, the accuracy rates for easy, medium, and hard classification scenarios on the WIDER FACE dataset, utilizing the ResNet50 backbone network, are 94.83%, 93.28%, and 84.99%, respectively. Accompanied by a frames-per-second rate of 7.704, this meets the precision and real-time criteria for face measurement tasks. Validation on the WIDER FACE dataset further affirms that ResRetinaFace consistently achieves reliable face detection while maintaining high detection efficiency.

When facing motion and complex environmental conditions, infrared videos captured by thermal imaging devices often suffer from blurring, leading to unclear or missing details and positional information about the targets. To improve this problem, this work proposes an improved deblurring method suitable for infrared videos based on a deep learning-based deblurring network originally designed for visible light images. This method is built upon the D²Net network by introducing a spatial and channel reconstruction convolution for feature redundancy, enhancing the network’s capability for image feature learning. In terms of the encoder-decoder module, a triple attention mechanism and fast Fourier transform are introduced to further improve the network’s deblurring performance. Through ablative experiments on infrared datasets, the results demonstrate a significant improvement in deblurring performance compared to the original D²Net. Specifically, the improved network achieved a 1.42 dB increase in peak signal-to-noise ratio and a 0.02 dB increase in structural similarity compared to the original network. In summary, this paper achieves promising results in infrared video deblurring tasks, demonstrating the effectiveness of the proposed method.

Aiming at the frequent misdetection and omission in the detection process of airport runway foreign object debris (FOD) and the difficulty of deploying the detection algorithm to embedded devices, we propose a lightweight FOD detection method called PGDIG-YOLO based on the improvement of YOLOv8n. First, a detection layer for detecting small-size objects is added and a large target detection layer is deleted to enhance the network’s ability to sense small-sized objects. Second, a dilation-wise residual module is introduced in the segmentation domain, and the C2FD module is proposed, which effectively solves the problem of misdetection and missed detection of FOD on airport runways. Third, the inner-WMPDIoUv3 is designed to replace the CIoU as a loss function to improve the regression accuracy of the detection frame. Finally, the model is pruned using the Group_sl method, which reduces the amount of computation, compresses the model size, and improves the model inference speed. The experimental results on the homemade dataset FOD-Z show that, compared with the benchmark model YOLOv8n, the model volume and computation of the PGDIG-YOLO network are only 6.6% and 44.4% of the original network, and the accuracy and recall are improved by 1.1% and 3.8%, respectively. Meanwhile, the mAP@0.5, mAP@0.75, and mAP@0.5:0.95 are increased to 99.1%, 93.7%, and 85.6%, respectively. Deploying PGDIG-YOLO to the NVIDIA Jetson Xavier NX 16 GB embedded device, the detection speed reaches 42 FPS, which can realize real-time FOD detection.

Compared with ordinary images, hyperspectral images (HSIs) consist of many bands that can provide rich spatial and spectral information and are widely used in remote sensing. However, HSIs are subject to various types of noise due to limited sensor sensitivity; low light intensity in the bands; and corruption during acquisition, transmission, and storage. Therefore, the problem of HSI denoising has attracted extensive attention from society. Although recent HSI denoising methods provide effective solutions in various optimization directions, their performance under real complex noise is still not optimal. To address these issues, this article proposes a self-modulated cross-attention network that fully utilizes spatial and spectral information. The core of the model is the use of deformable convolution to cross-fuse spatial and spectral features to improve the network denoising capability. At the same time, a self-modulating residual block allows the network to transform features in an adaptive manner based on neighboring bands, improving the network’s ability to deal with complex noise, which we call a feature enhancement block. Finally, we propose a three-segment network architecture that improves the stability of the model. The method proposed in this work outperforms other state-of-the-art methods through comparative analysis of experiments in synthetic and real data.

Most crowd counting methods rely on integrating density maps for prediction, but they encounter performance degradation in the face of density variations. Existing methods primarily employ a multi-scale architecture to mitigate this issue. However, few approaches concurrently consider both scale and timing information. We propose a scale-divided architecture for video crowd counting. Initially, density maps of different Gaussian scales are employed to retain information at various scales, accommodating scale changes in images. Subsequently, we observe that the spatiotemporal network places greater emphasis on individual locations, prompting us to aggregate temporal information at a specific scale. This design enables the temporal model to acquire more spatial information and alleviate occlusion issues. Experimental results on various public datasets demonstrate the superior performance of our proposed method.

Prefabricated steel pipes play a crucial role in prefabricated buildings, and maintaining their surface integrity is crucial to ensuring the safety of these buildings. We propose a surface defect detection algorithm for prefabricated steel pipes, D-YOLOv7-tiny, based on YOLOv7-tiny, to address the challenges of high parameter count and large computational requirements associated with traditional algorithms, making it difficult to deploy at resource-constrained terminals. By incorporating the squeeze-and-excitation attention mechanism into the backbone network, D-YOLOv7-tiny effectively minimizes the impact of redundant information and improves the network’s ability to extract features. In addition, distribution shifting convolution is implemented as a replacement for a portion of traditional convolution in the original effective layer aggregation network module network. This exchange reduces the computational workload of the model without affecting its expressive power. Subsequently, the lightweight and ubiquitous content-aware reassembly of features upsampling operator improved the feature merging of the network. Finally, an attention-based dynamic head was adopted to enhance the model’s robustness while minimizing parameter counts. Compared with YOLOv7-tiny, the mAP of D-YOLOv7-tiny was enhanced by 1.9% through experiments conducted on self collected datasets. During this process, the number of parameters and computational complexity decreased by 7.6% and 39.4%, respectively. The results show that this method achieves lightweight and meets the requirements of practical engineering accuracy and real-time performance.

In the field of computer vision and graphics, high-quality reconstruction of the human body in static scenes has been achieved in recent years by a single multilayer perceptron (MLP) in a number of approaches. However, MLPs have capacity limitations, requiring substantial training time and computational resources for dynamic scene reconstruction, and the quality of reconstruction is significantly constrained. We propose a method for effectively processing complex spatiotemporal signals in dynamic scene human three-dimensional (3D) modeling. The proposed method uses temporal residual neural radiance fields to achieve novel view rendering and new pose synthesis of human bodies. To address the problem of representing temporal signals in video sequences, we construct a temporal residual field that is not related to the MLP architecture. Second, to improve the reconstruction efficiency, we propose an integrated approach that reduces the trainable parameters and accelerates rendering, thereby enhancing the network’s feature representation capability. Finally, we design a multi-dimensional loss function to accurately measure the loss between predicted and actual spatial pixel values. The experimental results show that our proposed approach improves the peak signal-to-noise ratio and structural similarity index accuracy metrics compared with the latest representative methods. It maintains a similar accuracy to Anim-NeRF and Neural Body while achieving a nearly 780-fold increase in time efficiency.

A range of compression algorithms for generative adversarial networks (GANs) have been explored to alleviate the significant overhead involved in deploying GANs. However, in lightweight tasks, the role of the discriminator is usually ignored, resulting in an imbalance in training. Simultaneously, the ability of compact GANs to generate high-frequency information also lags behind teacher models. Addressing these challenges, we propose a wavelet-based knowledge distillation scheme with an adaptive discriminator (StyleWA). In StyleWA, the adaptive discriminator dynamically activates or inhibits convolutional channels based on the assessment of convolutional kernel contributions, ensuring more stable network training while preserving discriminator efficiency. Moreover, existing research primarily concentrates on the compression of conditional GANs, with limited exploration of unconditional GANs (uncGANs), known for their superior performance. We introduce wavelet analysis for the first time in the knowledge distillation of uncGANs, enhancing the model’s capability to control image details and augmenting distillation quality. Results demonstrate that the StyleWA scheme not only aids in maintaining Nash equilibrium between the discriminator and lightweight generator during training to mitigate pattern collapse but also facilitates the distillation model in learning intricate high-frequency details. This, in turn, enhances the overall generation quality and editing efficacy of the uncGAN model.

Multiscale features have attracted widespread attention in person reidentification due to their capability to enhance the model information processing. However, the use of multiscale features often leads to model redundancy and low operational efficiency, greatly affecting the practical application of person reidentification. To address the above issue, we propose an efficient and lightweight multiscale network for person reidentification. Specifically, we construct a baseline network for lightweight person reidentification called the multiscale efficient network (MSENet). It comprises three primary stages, with each stage employing the multiscale efficient block, consisting of varying numbers of lightweight efficient convolution blocks. The proposed network efficiently achieves pedestrian image retrieval while maintaining low model complexity. Subsequently, we propose a lightweight pyramid feature fusion module to aggregate multilayer features of the MSENet, enhancing feature diversity and obtaining robust features. Finally, we design a contour branch that focuses on the overall feature extraction of pedestrian images, effectively reducing the interference of background information. Extensive experiments conducted on three popular datasets have demonstrated that the proposed method has excellent recognition accuracy and low model complexity.

Occluded person re-identification (re-id) is a task in pedestrian retrieval where occluded person images are matched with holistic person images. Most methods leverage semantic cues from external models to align the availability of visible parts in the feature space. However, presenting visible parts while discarding occluded parts can lead to the loss of semantics in the occluded regions, and in severely crowded regions of occlusion, it will introduce inaccurate features that pollute the overall person features. Thus, constructing person features for occluded regions based on the features of its holistic parts has the potential to address the above issues. In this work, we propose a pose-guided node and trajectory construction transformer (PNTCT). The part feature extraction module extracts parts feature of the person and incorporates pose information to activate key visible local features. However, this is not sufficient to completely separate occluded regions. To further distinguish visible and occluded parts, the skeleton graph module adopts a graph topology to represent local features as graph nodes, enhancing the network’s sensitivity to local features by constructing a skeleton feature graph, which is further utilized to weaken the occlusion noise. The node and trajectory construction module (NTC) mines the relationships between skeleton nodes and aggregates the information of the person’s skeleton to construct a novel skeleton graph. The features of the occluded regions can be reconstructed via the features of the corresponding nodes in the novel skeleton graph. Extensive experiments and analyses confirm the effectiveness and superiority of our PNTCT method.

Various daily behaviors usually exert pressure on the contact surface, such as lying, walking, and sitting. Obviously, the pressure data from the contact surface contain some important biological information for an individual. Recently, a computer vision task, i.e., pose estimation from contact pressure (PECP), has received more and more attention from researchers. Although several deep learning-based methods have been put forward in this field, they cannot achieve accurate prediction using the limited pressure information. To address this issue, we present a multi-task-based PECP model. Specifically, the autoencoder is introduced into our model for reconstructing input pressure data (i.e., the additional task), which can help our model generate high-quality features for the pressure data. Moreover, both the mean squared error and the spectral angle distance are adopted to construct the final loss function, whose aim is to eliminate the Euclidean distance and angle differences between the prediction and ground truth. Extensive experiments on the public dataset show that our method outperforms existing methods significantly in pose prediction from contact pressure.

Contrastive learning for unpaired image-to-image translation (CUT) aims to learn a mapping from source to target domain with an unpaired dataset, which combines contrastive loss to maximize the mutual information between real and generated images. However, the existing CUT-based methods exhibit unsatisfactory visual quality due to the wrong locating of objects and backgrounds, particularly where it incorrectly transforms the background to match the object pattern in layout-changing datasets. To alleviate the issue, we present background-focused contrastive learning for unpaired image-to-image translation (BFCUT) to improve the background’s consistency between real and its generated images. Specifically, we first generate heat maps to explicitly locate the objects and backgrounds for subsequent contrastive loss and global background similarity loss. Then, the representative queries of objects and backgrounds rather than randomly sampling queries are selected for contrastive loss to promote reality of objects and maintenance of backgrounds. Meanwhile, global semantic vectors with less object information are extracted with the help of heat maps, and we further align the vectors of real images and their corresponding generated images to promote the maintenance of the backgrounds in global background similarity loss. Our BFCUT alleviates the wrong translation of backgrounds and generates more realistic images. Extensive experiments on three datasets demonstrate better quantitative results and qualitative visual effects.

The Joint Video Experts Team has recently finalized the versatile video coding (VVC) standard, which incorporates various advanced encoding tools. These tools ensure great enhancements in the coding efficiency, leading to a bitrate reduction up to 50% when compared to the previous standard, high-efficiency video coding. However, this enhancement comes at the expense of high computational complexity. Within this context, we address the new quadtree (QT) with nested multitype tree partition block in VVC for all-intra configuration. In fact, we propose a fast intra-coding unit (CU) partition algorithm using various convolution neural network (CNN) classifiers to directly predict the partition mode, skip unnecessary split modes, and early exit the partitioning process. The proposed approach first predicts the QT depth at a CU of size 64×64 by the corresponding CNN classifier. Then four CNN classifiers are applied to predict the partition decision tree at a CU of size 32×32 using multithreshold values and ignore the rate-distortion optimization process to speed up the partition coding time. Thus the developed method is implemented on the reference software VTM 16.2 and tested for different video sequences. The experimental results confirm that the proposed solution achieves an encoding time reduction of about 46% in average, reaching up to 67.3% with an acceptable increase in bitrate and an unsignificant decrease in quality.

As the elderly population in need of assisted living arrangements continues to grow, the imperative to ensure their safety is paramount. Though effective, traditional surveillance methods, notably RGB cameras, raise significant privacy concerns. This paper highlights the advantages of a surveillance system addressing these issues by utilizing skeleton joint sequences extracted from depth data. The focus on non-intrusive parameters aims to mitigate ethical and privacy concerns. Moreover, the proposed work prioritizes resource efficiency, acknowledging the often limited computing resources in assisted living environments. We strive for a method that can run efficiently even in the most resource-constrained environments. Performance evaluation and a prototypical implementation of our method on a resource-constraint device confirm the efficacy and suitability of the proposed method in real-world applications.

Storage space and security concerns on multimedia images have emerged as a global issue in recent years. Image encryption algorithm via compressed sensing (CS) is an effective method for data security and reducing storage space. However, the existing CS-based image encryption still faces problems, such as weak resistance to attacks and extensive data storage. We design a high-compression image encryption algorithm that combines fractal and semi-tensor product compressed sensing. First, a measurement matrix required for CS is generated using fractal blocks combined with the semi-tensor product method, which enhances security while reducing the size of the measurement matrix. Then, the measurements obtained from the sampling are used to define the product features of their mean and standard deviation. Exclusion criteria are set, and fractal codes are obtained through matched searching. Finally, the fractal code undergoes scrambling and diffusion, providing triple-layer protection and further improving the security of the secret image. In comparison to conventional methods, our proposed method has greatly improved the compression efficiency through compressed sampling and has the advantages of better concealment and enhanced robustness. Experiments show that we substantiate the effectiveness and superior performance of our method, all while upholding image quality and security.

The complexity of clouds, particularly in terms of texture detail at high resolutions, has not been well explored by most existing cloud detection networks. We introduce the high-resolution cloud detection network (HR-cloud-Net), which utilizes a hierarchical high-resolution integration approach. HR-cloud-Net integrates a high-resolution representation module, layer-wise cascaded feature fusion module, and multiresolution pyramid pooling module to effectively capture complex cloud features. This architecture preserves detailed cloud texture information while facilitating feature exchange across different resolutions, thereby enhancing the overall performance in cloud detection. Additionally, an approach is introduced wherein a student view, trained on noisy augmented images, is supervised by a teacher view processing normal images. This setup enables the student to learn from cleaner supervisions provided by the teacher, leading to an improved performance. Extensive evaluations on three optical satellite image cloud detection datasets validate the superior performance of HR-cloud-Net compared with existing methods.

Neuromorphic vision sensors (NVS) with the features of small data redundancy and transmission latency are widely implemented in Internet of Things applications. Previous studies have developed various object detection algorithms based on NVS’s unique event data format. However, most of these methods are only adaptive for scenarios with stationary backgrounds. Under dynamic background conditions, NVS can also acquire the events of non-target objects due to its mechanism of detecting pixel intensity changes. As a result, the performance of existing detection methods is greatly degraded. To address this shortcoming, we introduce an extra refinement process to the conventional histogram-based (HIST) detection method. For the proposed regions from HIST, we apply a practical decision condition to categorize them as either object-dominant or background-dominant cases. Then, the object-dominant regions undergo a second-time HIST-based region proposal for precise localization, while background-dominant regions employ an upper outline determination strategy for target object identification. Finally, the refined results are tracked using a simplified Kalman filter approach. Evaluated in an outdoor drone surveillance with an event camera, the proposed scheme demonstrates superior performance in both intersection over union and F1 score metrics compared to other methods.

Conducting video quality assessment (VQA) for user-generated content (UGC) videos and achieving consistency with subjective quality assessment are highly challenging tasks. We propose a no-reference video quality assessment (NR-VQA) method for UGC scenarios by considering characteristics of human visual perception. To distinguish between varying levels of human attention within different regions of a single frame, we devise a dual-branch network. This network extracts spatial features containing positional information of moving objects from frame-level images. In addition, we employ the temporal pyramid pooling module to effectively integrate temporal features of different scales, enabling the extraction of inter-frame temporal information. To mitigate the time-lag effect in the human visual system, we introduce the temporal pyramid attention module. This module evaluates the significance of individual video frames and simulates the varying attention levels exhibited by humans towards frames. We conducted experiments on the KoNViD-1k, LIVE-VQC, CVD2014, and YouTube-UGC databases. The experimental results demonstrate the superior performance of our proposed method compared to recent NR-VQA techniques in terms of both objective assessment and consistency with subjective assessment.

Clouds are an important factor in predicting future weather changes. Cloud image classification is one of the basic issues in the field of ground-based cloud meteorological observation. Deep CNN mainly focuses on the local receptive field, and the processing of global information may be relatively weak. In ground-based cloud image classification, if there is a complex background, it will help to better model the long-range dependence of the image if the relationship between different locations in the image can be globally captured. A ground-based cloud image classification method is proposed based on the fusion of local features and global features (LG_CloudNet). The ground-based cloud image classification method integrates the global feature extraction module (GF_M) and the local feature extraction module (LF_M), using the attention mechanism to weight and merge features, respectively. The LG_CloudNet model enables richer and comprehensive feature representation at lower computational complexity. In order to ensure the learning and generalization capabilities of the model during training, AdamW (Adam weight decay) is combined with learning rate warm-up and stochastic gradient descent with warm restarts methods to adjust the learning rate. The experimental results demonstrate that the proposed method achieves favorable ground-based cloud image classification outcomes and exhibits robust performance in classifying cloud images. In the datasets of GCD, CCSN, and ZNCL, the classification accuracy is 94.94%, 95.77%, and 98.87%, respectively.

Capturing high-quality photos in an underwater atmosphere is complicated, as light attenuation, color distortion, and reduced contrast pose significant challenges. However, one fact usually ignored is the non-uniform texture degradation in distorted images. The loss of comprehensive textures in underwater images poses obstacles in object detection and recognition. To address this problem, we have introduced an image enhancement model called scene adaptive color compensation and multi-weight fusion for extracting fine textural details under diverse environments and enhancing the overall quality of the underwater imagery. Our method blends three input images derived from the adaptive color-compensating and color-corrected version of the degraded image. The first two input images are used to adjust the low contrast and dehazing of the image respectively. Similarly, the third input image is used to extract the fine texture details based on different scales and orientations of the image. Finally, the input images with their associated weight maps are normalized and fused through multi-weight fusion. The proposed model is tested on a distinct set of underwater imagery with varying levels of degradation and frequently outperformed state-of-the-art methods, producing significant improvements in texture visibility, reducing color distortion, and enhancing the overall quality of the submerged images.

A multitude of confidential and personal digital images are commonly stored and transmitted by devices with limited resources. These devices necessitate the implementation of uncomplicated yet highly efficient encryption techniques to safeguard the images. The challenge of designing encryption algorithms for multiple digital images that are simple, secure, and highly efficient is significant. This challenge arises due to the large quantity of images involved and the considerable size and strong inter-pixel associations exhibited by these digital images. We propose a method for efficiently, simply, and securely encrypting multiple images simultaneously using chaotic one-dimensional (1D) maps. Initially, each grayscale image is consolidated into a single, substantial image. Through transpose columnar transposition and bit-XOR diffusion procedures, each block undergoes parallel permutation and diffusion. The incorporation of parallel permutation and diffusion functions accelerates and enhances the performance of the method. In contrast to existing multi-image encryption methods, the proposed approach consistently employs a single 1D chaotic map, rendering the algorithm both software and hardware efficient while maintaining simplicity. The encryption technique adheres to general requirements for simplicity and high efficiency. Security analysis and simulation results demonstrate that the proposed method is straightforward, highly efficient, and effectively enhances the security of cipher images.

Existing low-light image enhancement (LLIE) technologies have difficulty balancing image quality and computational efficiency. In addition, they amplify the noise and artifacts of the original image when enhancing deep dark images. Therefore, this study proposes a multi-scale adaptive low-light image enhancement method based on deep learning. Specifically, feature extraction and noise reduction modules are designed. First, a more effective low-light enhancement effect is achieved by extracting the details of the dark area of an image. Depth extraction of the details of dark areas is realized through the design of a residual attention mechanism and nonlocal neural network in the UNet model to obtain a visual-attention map of the dark area. Second, the designed noise network obtains the real noise map of the low-light image. Subsequently, the enhanced network uses the dark area visual-attention and noise maps in conjunction with the original low-light image as inputs to adaptively realize LLIE. The LLIE results using the proposed network achieve excellent performance in terms of color, tone, contrast, and detail. Finally, quantitative and visual experiments on multiple test benchmark datasets demonstrate that the proposed method is superior to current state-of-the-art methods in terms of dark area details, image quality enhancement, and image noise reduction. The results of this study can help to address the real world challenges of low-light image quality, such as low contrast, poor visibility, and high noise levels.

Texture features and stability have attracted much attention in the field of biometric recognition. The inner-knuckle print is unique and not easy to forge, so it is widely used in personal identity authentication, criminal detection, and other fields. In recent years, the rapid development of deep learning technology has brought new opportunities for internal-knuckle recognition. We propose a deep inner-knuckle print recognition method named LSKNet network. By establishing a lightweight Siamese network model and combining it with a robust cost function, we can realize efficient and accurate recognition of the inner-knuckle print. Compared to traditional methods and other deep learning methods, the network has lower model complexity and computational resource requirements, which enables it to run under lower hardware configurations. In addition, this paper also uses all the knuckle prints of four fingers for concatenated fusion recognition. Experimental results demonstrate that this method has achieved satisfactory results in the task of internal-knuckle print recognition.

Recent advancements in biometric recognition focus on vein pattern–based person authentication systems. We present a multimodal biometric system using dorsal and finger vein images. By combining Siamese neural networks (SNNs) with hybrid firefly–particle swarm optimization (FF-PSO), we optimize finger and dorsal vein identification and classification. Using FF-PSO to tune SNN parameters is an innovative hybrid optimization approach designed to address the complexities of vein pattern recognition. The proposed system is tested with two public databases: the SDUMLA-HMT finger vein dataset and the Dr. Badawi hand vein dataset. The efficacy of the method is assessed using performance measures such as recall, accuracy, precision, F1 score, false acceptance rate, false rejection rate, and equal error rate. The experimental findings demonstrate that the proposed system achieves an accuracy of 99.5% with the fine-tune SNN and FF-PSO techniques and preprocessing module. The proposed system is also compared with various existing state-of-the-art techniques.

To effectively address the challenges of large motions, complex backgrounds and large occlusions in videos, we introduce an end-to-end method for video frame interpolation based on recurrent residual convolution and depthwise over-parameterized convolution in this paper. Specifically, we devise a U-Net architecture utilizing recurrent residual convolution to enhance the quality of interpolated frame. First, the recurrent residual U-Net feature extractor is employed to extract features from input frames, yielding the kernel for each pixel. Subsequently, an adaptive collaboration of flows is utilized to warp the input frames, which are then fed into the frame synthesis network to generate initial interpolated frames. Finally, the proposed network incorporates depthwise over-parameterized convolution to further enhance the quality of interpolated frame. Experimental results on various datasets demonstrate the superiority of our method over state-of-the-art techniques in both objective and subjective evaluations.

Current scene text detection methods mainly employ RGB domain information for text localization, and their performance has not been fully exploited in many challenging scenes. Considering that the RGB features of text and background in complex environments are subtle and more discernible in the frequency domain, we consider that the frequency-domain information can effectively complement the RGB-domain features, collectively enhancing text detection capabilities. To this end, we propose a network with complementary frequency-domain semantic and color features, called the bistream structure, to facilitate text detection in scenes characterized by a wide variety of complex patterns. Our approach utilizes a frequency perception module (FPM) that converts features extracted by the backbone into the frequency domain to enhance the ability to distinguish the text from the complex background, thereby achieving coarse localization of texts. This innovation utilizes frequency-domain features to efficiently reveal text structures obscured by background noise in the RGB domain, resulting in a sharper differentiation between text and background elements in challenging scenarios. Moreover, we propose a complementary correction module that guides the fusion of multi-level RGB features through the coarse localization results, progressively refining the segmentation results to achieve the correction of the frequency domain features. Extensive experiments on the Total-Text, CTW1500, and MSRA-TD500 datasets demonstrate that our method achieves outstanding performance in scene text detection.

Understanding the perimeter objects and environment changes in railway scenes is crucial for ensuring the safety of train operation. Semantic segmentation is the basis of intelligent perception and scene understanding. Railway scene categories are complex and effective features are challenging to extract. This work proposes a semantic segmentation network DeepLab-Rail based on classic yet effective encoder-decoder structure. It contains a lightweight feature extraction backbone embedded with channel attention (CA) mechanism to keep computational complexity low. To enrich the receptive fields of convolutional modules, we design a parallel and cascade convolution module called compound-atrous spatial pyramid pooling and a combination of dilated convolution ratio is selected through experiments to obtain multi-scale features. To fully use the shallow features and the high-level features, efficient CA mechanism is introduced and also the mixed loss function is designed for the problem of unbalanced label categories of the dataset. Finally, the experimental results on the RailSem19 railway dataset show that the mean intersection over union reaches 65.52% and the PA reaches 88.48%. The segmentation performance of railway confusing facilities, such as signal lights and catenary pillars, has been significantly improved and surpasses other advanced methods to our best knowledge.

High-dimensional data samples tend to contain highly correlated features and are quite fragile to various noises and outliers in practical applications. For subspace clustering models, it has appeared to be inadequate to adopt conventional norm-based distance measurements to resist feature contaminations by undetermined types of noises. As a consequence, the learned low-dimensional representation is not always reliable and discriminative, which inevitably impedes the clustering performance. To remedy the deficiencies, we propose a robust subspace clustering model via auto-weighted and dual-structural representation (AWDSR) learning. Specifically, a feature-weighted reconstruction term is first introduced to the self-representation framework to automatically reinforce important features by measuring the self-representational reconstruction loss. As such, various types of noise features in data samples could be adaptively assigned with relatively small weights to reduce the residual elements of the reconstruction term. Moreover, an adaptive dual-structural constraint is simultaneously considered to guarantee discriminative block-diagonal representation. Then, an efficient alternative optimization method with guaranteed convergence and relatively low complexity is developed to optimize the challenging objective function. Finally, we carry out extensive experiments to compare the AWDSR model with other state of the arts on one synthetic data and six real-world databases. Experimental results fully demonstrate the effectiveness and superiority of the proposed approach in terms of accuracy and normalized mutual information metrics.

Automatic floor plan image analysis is becoming popular in the construction industry. An architectural floor plan provides the layout of a building floor and includes objects such as walls, doors, windows, and stairs. Detection of the walls in a floor plan image is important as the walls typically define the main layout of the floor and individual rooms. In existing literature, the walls are typically detected as a single class object. However, in construction type floor plans, the walls are represented by different drawings (e.g., solid-wall, dot-wall, diagonal-wall, hollow-wall, and gray-wall) based on the raw materials used for construction. Detection of multiclass walls would be desirable for applications such as materials cost estimation by builders and building information modeling. A convolutional neural network, namely WallNetv2, is proposed for semantic segmentation of multiclass walls in a floor plan image. WallNetv2 consists of an encoder, a channel contextual module, a spatial contextual module, and a decoder. The encoder extracts the hierarchical features from the input floor plan image. The channel and spatial contextual modules capture the relationship of the high-level features among channels and pixels, respectively. The decoder further processes the learned features and recovers the spatial information gradually with connections to the low-level features. The experimental results show that the proposed WallNetv2 achieves a mean IoU of 70%, which is superior to the state-of-the-art techniques.

A challenge in video tracking is effectively utilizing temporal correlation to navigate the complexity of a tracking scene without imposing additional computational requirements. Different from trackers that rely on online update templates, our methodology introduces a concise and efficient Siamese tracker, which assimilates temporal information via an offline strategy, named SiamGPF. Given the potential for a target to display varied appearances over time, a dependence solely on static template information proves inadequate for addressing extreme deformations and occlusions. Accordingly, we integrate the historical state of the target as pseudo-priori information into our framework, using global and local cross-correlation to fuse features accurately. Furthermore, we introduce a state fusion module to enhance the interplay of information across disparate temporal points of the target. In the inference phase, we implement a cost-effective prior frame dynamic selector, adept at eliminating low-quality target information, thereby substantially augmenting the robustness of the tracker. Our tracker exhibits competitive performance across prominent tracking benchmark datasets (OTB100, LaSOT, LaSOTExt, UAV123, NFS, TC128, GOT-10k, and VOT2016), with the official GOT-10k server evaluation results demonstrating an impressive operational speed of 212 FPS.

Nowadays, we are surrounded by various types of data from different modalities, such as text, images, audio, and video. The existence of this multimodal data provides us with rich information, but it also brings new challenges: how do we effectively utilize this data for accurate classification? This is the main problem faced by multimodal classification tasks. Multimodal classification is an important task that aims to classify data from different modalities. However, due to the different characteristics and structures of data from different modalities, effectively fusing and utilizing them for classification is a challenging problem. To address this issue, we propose a cross-attention contextual transformer with modality-collaborative learning for multimodal classification (CACT-MCL-MMC) to better integrate information from different modalities. On the one hand, existing multimodal fusion methods ignore the intra- and inter-modality relationships, and there is unnoticed information in the modalities, resulting in unsatisfactory classification performance. To address the problem of insufficient interaction of modality information in existing algorithms, we use a cross-attention contextual transformer to capture the contextual relationships within and among modalities to improve the representativeness of the model. On the other hand, due to differences in the quality of information among different modalities, some modalities may have misleading or ambiguous information. Treating each modality equally may result in modality perceptual noise, which reduces the performance of multimodal classification. Therefore, we use modality-collaborative to filter misleading information, alleviate the quality difference of information among modalities, align modality information with high-quality and effective modalities, enhance unimodal information, and obtain more ideal multimodal fusion information to improve the model’s discriminative ability. Our comparative experimental results on two benchmark datasets for image-text classification, CrisisMMD and UPMC Food-101, show that our proposed model outperforms other classification methods and even state-of-the-art (SOTA) multimodal classification methods. Meanwhile, the effectiveness of the cross-attention module, multimodal contextual attention network, and modality-collaborative learning was verified through ablation experiments. In addition, conducting hyper-parameter validation experiments showed that different fusion calculation methods resulted in differences in experimental results. The most effective feature tensor calculation method was found. We also conducted qualitative experiments. Compared with the original model, our proposed model can identify the expected results in the vast majority of cases. The codes are available at https://github.com/KobeBryant8-24-MVP/CACT-MCL-MMC. The CrisisMMD is available at https://dataverse.mpisws.org/dataverse/icwsm18, and the UPMC-Food-101 is available at https://visiir.isir.upmc.fr/.

Accurate measurement and identification of the number and length of yarn hairiness is crucial for spinning process optimization and product quality control. However, the existing methods have problems, such as low detection accuracy and efficiency, and incomplete detection. In order to overcome the above defects, an image acquisition device based on a multi-camera system is established to accurately obtain multiple perspectives of hairiness images. An automatic threshold segmentation method based on the local bimodal is proposed based on image difference, convolution kernel enhancement, and histogram equalization. Then, the clear and unbroken yarn hairiness segmentation images are obtained according to the hairiness edge extraction method. Finally, a perspective maximization model is proposed to realize the calculation of the hairiness H value and the number of hairiness in interval. Six kinds of cotton ring-spun yarn with different linear densities are tested using the proposed method, YG133B/M instrument, manual method, and single perspective method. The results show that the proposed multi-camera method can realize the index measurement of the yarn hairiness.

Based on the enhanced YOLOv5, a deep learning defect detection technique is presented to deal with the problem of inadequate effectiveness in manually detecting problems on the surface of filter screens. In the last layer of the backbone network, the method combines the squeeze-and-excitation attention mechanism module, the method assigns weights to image locations based on the channel domain perspective to obtain more feature information. It also compares the results with a simple, parameter-free attention model (SimAM), which is an attention mechanism without the channel domain, and the results are higher than SimAM 0.7%. In addition, the neck network replaces the basic PANet structure with the bi-directional feature pyramid network module, which introduces multi-scale feature fusion. The experimental results show that the improved YOLOv5 algorithm has an average defect detection accuracy of 97.7% on the dataset, which is 11.3%, 12.8%, 2%, 7.8%, 5.1%, and 1.3% higher than YOLOv3, faster R-CNN, YOLOv5, SSD, YOLOv7, and YOLOv8, respectively. It can quickly and accurately identify various defects on the surface of the filter, which has an outstanding contribution to the filter manufacturing industry.

Vision transformer (ViT) is widely used to handle artificial intelligence tasks, making significant advances in a variety of computer vision tasks. However, due to the secondary interaction between tokens, the ViT model is inefficient, which greatly limits the application of the ViT model in real scenarios. In recent years, people have noticed that not all tokens contribute equally to the final prediction of the model, so token compression methods have been proposed, which are mainly divided into token pruning and token merging. Yet, we believe that neither pruning only to reduce non-critical tokens nor merging to reduce similar tokens are optimal strategies for token compression. To overcome this challenge, this work proposes a token compression framework: joint merging and pruning (JMP), which adaptively selects a better token compression strategy based on the similarity between critical tokens and non-critical tokens in each sample. JMP effectively reduces computational complexity while maintaining model performance and does not require the introduction of additional trainable parameters, achieving a good balance between efficiency and performance. Taking DeiT-S as an example, JMP reduces floating point operations by 35% and increases throughput by more than 45% while only decreasing accuracy by 0.2% on ImageNet.

Ensuring safety on the road is crucial, and detecting driving distractions plays a vital role in achieving this goal. Accurate identification of distracted driving behaviors facilitates prompt intervention, thereby contributing to a reduction in accidents. We introduce an advanced fully convolutional one-stage (FCOS) object detection algorithm tailored for driving distraction detection that leverages the knowledge distillation framework. Our proposed methodology enhances the conventional FCOS algorithm through the integration of the selective kernel split-attention module. This module bolsters the performance of the backbone network, ResNet, leading to a substantial improvement in the accuracy of the FCOS target detection algorithm. In addition, we incorporate a knowledge distillation framework equipped with a novel local and global knowledge distillation loss function. This framework facilitates the student network to achieve accuracy levels comparable to that of the teacher network while maintaining a reduced parameter count. The outcomes of our approach are promising, achieving a remarkable accuracy of 92.25% with a compact model size of 31.85 million parameters. This advancement paves the way for more efficient and accurate distracted driving detection systems, ultimately contributing to enhanced road safety.

Convolutional neural networks (CNNs) with a powerful feature extraction ability have raised the performance of salient object detection (SOD) to a unique level, and how to effectively decode the rich features from CNN is the key to improving the performance of the SOD model. Some previous works ignored the differences between the high-level and low-level features and neglected the information loss during feature processing, making them fail in some challenging scenes. To solve this problem, we propose a discriminative feature extraction and details preservation network (D2Net) for SOD. According to the different characteristics of high-level and low-level features, we design a residual optimization module for filtering complex background noise in shallow features and a pyramid feature extraction module to eliminate the information loss caused by atrous convolution in high-level features. Furthermore, we design a features aggregation module to aggregate the elaborately processed high-level and low-level features, which fully considers the performance of different level features and preserves the delicate boundary of salient object. The comparisons with 17 existing state-of-the-art SOD methods on five popular datasets demonstrate the superiority of the proposed D2Net, and the effectiveness of each proposed module is verified through numerous ablation experiments.

Tracking rigid objects in three-dimensional (3D) space and 6DoF pose estimating are essential tasks in the field of computer vision. In general, the region-based 3D tracking methods have emerged as the optimal solution for weakly textured objects tracking within intricate scenes in recent years. However, tracking robustness in situations such as partial occlusion and similarly colored backgrounds is relatively poor. To address this issue, an improved region-based tracking method is proposed for achieving accurate 3D object tracking in the presence of partial occlusion and similarly colored backgrounds. First, a regional cost function based on the correspondence line is adopted, and a step function is proposed to alleviate the misclassification of sampling points in scenes. Afterward, in order to reduce the influence of similarly colored background and partial occlusion on the tracking performance, a weight function that fuses color and distance information of the object contour is proposed. Finally, the transformation matrix of the inter-frame motion obtained by the above region-based tracking method is used to initialize the model point cloud, and an improved point cloud registration method is adopted to achieve accurate registration between the model point cloud and the object point cloud to further realize accurate object tracking. The experiments are conducted on the region-based object tracking (RBOT) dataset and the real scenes, respectively. The results demonstrate that the proposed method outperforms the state-of-the-art region-based 3D object tracking method. On the RBOT dataset, the average tracking success rate is improved by 0.5% across five image sequences. In addition, in real scenes with similarly colored backgrounds and partial occlusion, the average tracking accuracy is improved by 0.28 and 0.26 mm, respectively.

Images shared on social networks often contain a large amount of private information. Bad actors can use deep learning technology to analyze private information from these images, thus causing user privacy leakage. To protect the privacy of users, reversible adversarial examples (RAEs) are proposed, and they may keep malignant models from accessing the image data while ensuring that the authorized model can recover the source data. However, existing RAEs have shortcomings in imperceptibility and attack capability. We utilize frequency domain information to generate RAEs. To improve the attack capability, the RAEs are generated by discarding the discriminant information of the original class and adding specific perturbation information. For imperceptibility, we propose to embed the perturbation in the wavelet domain of the image. Also, we design low-frequency constraints to distribute the perturbations in the high-frequency region and to ensure the similarity between the original examples and RAEs. In addition, the momentum pre-processing method is proposed to ensure that the direction of the gradient is consistent in each iteration by pre-converging the gradient before the formal iteration, thus accelerating the convergence speed of the gradient, which can be applied to the generation process of RAEs to speed up the generation of RAEs. Experimental results on the ImageNet, Caltech-256, and CIFAR-10 datasets show that the proposed method exhibits the best attack capability and visual quality compared with existing RAE generation schemes. The attack success rate and peak signal-to-noise ratio exceed 99% and 42 dB, respectively. In addition, the generated RAEs demonstrate good transferability and robustness.

We propose an approach leveraging the power of self-organizing maps (SOMs) in conjunction with a multiscale local image fitting (LIF) level-set function to enhance the capabilities of the region-based active contour model (ACM). In addition, we employ the lattice Boltzmann method (LBM) to ensure efficient convergence during the segmentation process. The SOM learns the underlying patterns and structures of both the background region and the object of interest region in an image, allowing for more accurate and robust segmentation results. Our multiscale LIF level-set approach influences image-specific fitting criteria into the energy functional, considering the features extracted by the SOM. Finally, the LBM is utilized to solve the level set equation and evolve the contour, allowing for a faster contour evolution. To evaluate the effectiveness of our approach, we performed our experiments on the challenging Pascal Visual Object Classes Challenge 2012 dataset. This dataset consists of images containing objects with diverse characteristics, such as illumination variations, shadows, occlusions, scale changes, and cluttered backgrounds. Our experimental results highlight the efficiency and robustness of our proposed method in achieving accurate segmentation. In terms of accuracy, our approach outperforms state-of-the-art learning-based ACMs, reaching a precision value of up to 93%. Moreover, our approach also demonstrates improvements in terms of computation time, leading to a reduction in computational time of 76% compared with the state-of-the-art methods. By integrating SOMs and the LBM, we enhance the efficiency of the segmentation process. This enables us to achieve accurate segmentation within reasonable time frames, making our method practical for real-world applications. Furthermore, we conducted experiments on medical imagery and thermal imagery, which yielded precise results.

Under unfavorable conditions, fusion images of infrared and visible images often lack edge contrast and details. To address this issue, we propose an edge-oriented unrolling network, which comprises a feature extraction network and a feature fusion network. In our approach, after respective enhancement processes, the original infrared/visible image pair with their enhancement version is combined as the input to get more prior information acquisition. First, the feature extraction network consists of four independent iterative edge-oriented unrolling feature extraction networks based on the edge-oriented deep unrolling residual module (EURM), in which the convolutions in the EURM modules are replaced with edge-oriented convolution blocks to enhance the edge features. Then, the convolutional feature fusion network with differential structure is proposed to obtain the final fusion result, through utilizing the concatenate operation to map multidimensional features. In addition, the loss function in the fusion network is optimized to balance multiple features with significant differences in order to achieve better visual effect. Experimental results on multiple datasets demonstrate that the proposed method produces competitive fusion images as evaluated subjectively and objectively, with balanced luminance, sharper edge, and better details.

Due to the rapid advancement of hyperspectral remote sensing technology, classification methods based on hyperspectral images (HSIs) have gained increasing significance in the processes of target identification, mineral mapping, and environmental management. This importance arises from the fact that HSIs offer a more comprehensive understanding of a target’s composition. However, addressing the challenges posed by the high dimensionality and redundancy of HSI sets, coupled with potential class imbalances in hyperspectral datasets, remains a complex task. Both convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have demonstrated promising results in HSI classification in recent years. Nonetheless, CNNs struggle to attain high accuracy with limited sample sizes, whereas GCNs demand substantial computational resources. Oversmoothing remains a persistent challenge with conventional GCNs. In response to these issues, an approach known as the graph attention neural network for remote target classification (GATN-RTC) has been proposed. GATN-RTC employs a spectral filter and an autoregressive moving average filter to classify distant targets, addressing datasets both with and without labeled samples. To evaluate the performance of GATN-RTC, we conducted a comparative analysis against state-of-the-art methodologies using key performance metrics, such as overall accuracy (OA), per-class accuracy, and the Cohen’s Kappa statistic (KC). The findings reveal that GATN-RTC outperforms existing approaches, achieving improvements of 5.95% in OA, 5.33% in per-class accuracy, and 8.28% in the Cohen’s KC for the Salinas dataset. Furthermore, it demonstrates enhancements of 6.05% and 6.4% in OA, 6.56% and 5.89% in per-class accuracy, and 6.71% and 6.23% in the Cohen’s KC for the Pavia University dataset.

The primary objective of precipitation forecasting is to accurately predict short-term precipitation at a high resolution within a specific area, which is a significant and intricate challenge. Traditional models often struggle to capture the multidimensional characteristics of precipitation clouds in both time and space, leading to imprecise predictions due to their expansion, dissipation, and deformation. Recognizing this limitation, we introduce ConvLSTM-UNet, which leverages spatiotemporal feature extraction from meteorological images. ConvLSTM-UNet is an efficient convolutional neural network (CNN) based on the classical UNet architecture, equipped with ConvLSTM and improved deep separable convolutions. We evaluate our approach on the generic time series dataset Moving MNIST and the regional precipitation dataset of the Netherlands. The experimental results show that the proposed method has better spatiotemporal prediction skills than other tested models, and the mean squared error is reduced by more than 7.2%. In addition, the visualization results of the precipitation forecast show that the approach has a better ability to capture heavy precipitation, and the texture details of the precipitation forecast are closer to the ground truth.

Under limited bandwidth conditions, video transmission often employs lossy compression to reduce the data volume, inevitably introducing compression noise. Quality enhancement of compressed videos can effectively recover the information loss incurred during the compression process. Currently, multi-frame quality enhancement of compressed videos has shown performance advantages compared to single-frame methods, as it utilizes the temporal correlation of videos. Methods based on deformable convolutions obtain spatio-temporal fusion features for reconstruction through multi-frame alignment. However, due to the limited utilization of deep information and sensitivity to alignment accuracy, these methods yield suboptimal results, especially in scenarios with scene changes and intense motion. To overcome these limitations, we propose a dense network-based quality enhancement method to obtain more accurate spatio-temporal fusion features. Specifically, the deep spatial features are first extracted from the to-be-enhanced frames using dense connections, then combined with the aligned features obtained from deformable convolution through the convolution and attention mechanism to make the network more attentive to useful branches in an adaptive way, and finally, the enhanced frames are obtained through the quality enhancement module of the dense connection structure. The experimental results show that when the quantization parameter is 37, the proposed method can improve the average peak signal-to-noise ratio by 0.99 dB in the lowdelay_P configuration.

Deep learning based defect detection methods require a large amount of high-quality defect data. However, the defect samples obtained in practical production are relatively expensive and lack diversity. The data generation method based on generative adversarial networks (GANs) can address the issue of insufficient defect samples at a lower cost. However, the training process of data generation algorithms based on GANs may be affected by various factors, making it challenging to ensure the stability of the quality of the synthesized defect data. Since high-quality defect data determine the performance and representation of the detection model, it is necessary to conduct alternative evaluations on the synthesized defect data. We comprehensively consider the evaluation indicators that affect the generated defect data and propose an alternative evaluation method for comprehensive data on surface defects of industrial products. First, an evaluation index system is constructed based on the attributes of defect data. Then, a substitution evaluation model for surface defect data is built using a multi-level quantitative analytic hierarchy process. Finally, to verify the effectiveness of the evaluation model, we use three advanced defect detection networks and validate the effectiveness of the evaluation model through comparative experiments. We provide an effective solution for screening high-quality defect data generation and improve the performance of downstream task defect detection models.

Geospatial information is essential for development planning, like in the context of land and resource management. Existing research mainly focuses on multi-spectral or panchromatic images with specific sensor details. Incorporating multi-sensory panchromatic images at different scales makes the segmentation problem challenging. In this work, we propose a pixel-based globally trained model with a deep learning network to improve the segmentation results over existing patch-based networks. The proposed model consists of the encoder-decoder mechanism for semantic segmentation. Convolution and pooling layers are used at the encoding phase and transposed convolution and convolution layers are used for the decoding phase. Experiments show about 98.95% correct detection rate and 0.07% false detection rate of our proposed methodology on benchmark images. We prove the effectiveness of the proposed methodology by doing comparisons with previous work.

White blood cells are a core component of the immune system, responsible for protecting the human body from foreign invaders and infectious diseases. A decrease in the white blood cell count can lead to weakened immune function, increasing the risk of infection and illness. However, determining the number of white blood cells usually requires the expertise and effort of radiologists. In recent years, with the development of image processing technology, biomedical systems have widely applied image processing techniques in disease diagnosis. We aim to classify the subtypes of white blood cells using image processing technology. To improve the ability to extract fine information during the feature extraction process, the spatial prior convolutional attention (SPCA) module is proposed. In addition, to enhance the connection between features at distant distances, the Shifted Window (Swin) Transformer network is used as the backbone for feature extraction. The SGTformer network for white blood cell subtype classification is proposed by combining recursive gate convolution and SPCA modules. Our method is validated on the white blood cell dataset, and the experimental results demonstrate an overall accuracy of 99.47% in white blood cell classification, surpassing existing mainstream classification algorithms. It is evident that this method can effectively accomplish the task of white blood cell classification and provide robust support for the health of the immune system.

Circularly coded targets are widely used in 3D measurement, target tracking, augmented reality, and other fields as feature points to be measured. The traditional coded target recognition algorithm is easily affected by illumination changes and excessive shooting angles, and the recognition accuracy is significantly reduced. Therefore, a new coded target recognition algorithm is required to reduce the effects of illumination and angle on the recognition process. The influence of illumination on the recognition of coding targets was analyzed in depth, and the advantages and disadvantages of traditional algorithms are discussed. A new adaptive threshold image segmentation method was designed, which, in contrast to traditional algorithms, incorporates the feature information of coding targets in the determination of the image segmentation threshold. The experimental results show that this method significantly reduces the influence of illumination variations and cluttered backgrounds on image segmentation. Similarly, the influence of different angles on the recognition process of coding targets was studied. The coding target is decoded by radial sampling of the dense point network, which can effectively reduce the influence of angle on the recognition process and improve the recognition accuracy of coding targets and the robustness of the algorithm. In addition, further experiments verified that the proposed detection and recognition algorithm can better extract and identify with high positioning accuracy and decoding success rate. It can achieve accurate positioning even in complex environments and meet the needs of industrial measurements.