This PDF file contains the front matter associated with SPIE Proceedings Volume 12521, including the Title Page, Copyright information, Table of Contents, and Conference Committee listings.

DARPA has made significant investments in artificial intelligence (AI) research since the 1960s. The past year has seen dramatic growth in AI capabilities, primarily from large pre-trained models (LPTMs) like ChatGPT. While LPTMs bring novel, emergent capabilities, they also create the opportunity for failures that could occur quickly and at scale. This talk will cover some of the recent advances in LPTMs, the downsides of current LPTMs, areas where DARPA believes significant investment in AI is still required to support national security, and recent results from DARPA AI programs.

To be determined

Decision metrics for automated target recognition and classification rely upon statistical distributions for the signal of interest and the background noise. This paper describes distribution models for situations in which an acoustic, RF, optical, or seismic signal is randomly scattered by the environment and received at one or two sensors, with the scattering strength varying randomly in space and time. A new distribution, called the compound variance gamma, is introduced, which applies to partially correlated data between two sensors with random scattering strength. This distribution reduces to several previously known scattering distributions as special cases. Calculation of receiver operating characteristic (ROC) curves using the new distribution is also discussed. A second new distribution, involving a product of modified Bessel functions, is also introduced to describe the magnitude of the cross product between a pair of sensors as needed to calculate the ROC curves. It is shown that the randomized scattering strength and correlation between the two sensors significantly impact signal detection.

Automated target detection and recognition (ATDR) algorithms solely based on sensor data have seen great strides in improvement, especially with the on-set of deep learning neural networks, multi-sensor and multimodal fusion techniques. However, ATR applied just on imagery with few-pixels-on-targets in highly-cluttered environments remains a tough problem. Rather than focusing on imagery as the only input to an ATDR process, in this paper, we turn our attention to using contextual and heterogeneous information to help aid in improving ATDR accuracy. We treat scene context as a collection of random variables that can then be cast into a Bayesian framework. Specifically, targets likelihoods given the context are estimated by an ensemble training process. Then statistical inference is applied to update the probability vector of the target estimates. For low-observability cases on the targets, this can dramatically improve the accuracy of the true target type. In this paper, we identify some of these contexts and apply it from the output of an emulated ATDR image-only process and report results.

Deep learning has expedited important breakthroughs in research and commercial applications for next-generation technologies across many domains including Automatic Target Recognition (ATR). The success of these models in a specific application is often attributed to optimized hyperparameters: user-configured values controlling the model’s ability to learn from data. Tuning hyperparameters however remains a difficult and computationally expensive task contributing to deficient ATR model performance compared to set requirements. We present the efficacy of applying our developed hyperparameter optimization method to boost the effectiveness and performance of any given optimization method. Specifically, we use a generalized additive model surrogate homotopy hyperparameter optimization strategy to approximate regions of interest and trace minimal points over regions of the hyperparameter space instead of ineffectively evaluating the entire hyperparameter surface. We integrate our approach into SHADHO (Scalable Hardware-Aware Distributed Hyperparameter Optimization) a hyperparameter optimization framework that computes the relative complexity of each search space and then monitors the performance of the learning task over the trials. We demonstrate how our approach effectively finds optimal hyperparameters for object detection by conducting a model search to optimize multiple object detection algorithms on a subset of the DSIAC ATR Algorithm Development Image Database and finding models that achieve comparable or lower validation loss in fewer iterations than standard techniques and manual tuning practices.

In most state-of-the-art (SoTA) infrared small target detection algorithms, image regions are processed locally. More recently, some transformer-based algorithms have been proposed that account for separate image regions to detect small objects. Besides their success, transformer-based algorithms require more data when compared to classical methods. In these algorithms, massive datasets are used to achieve comparable performance with the SoTA methods for the RGB domain. There is no solid work in the literature about how much data is required to develop a transformer-based small target detection algorithm. By its nature, a small target does not contain discriminative contextual information. Thus, its blob-like shape and the contrast difference between the target and background are the main features exploited by the literature. Analyzing the required amount of data to obtain acceptable accuracy for infrared small target detection is the main motivation of this study.

One of the major obstacles in designing an automatic target recognition (ATR) algorithm, is that there are often labeled images in one domain (i.e., infrared source domain) but no annotated images in the other target domains (i.e., visible, SAR, LIDAR). Therefore, automatically annotating these images is essential to build a robust classifier in the target domain based on the labeled images of the source domain. Transductive transfer learning is an effective way to adapt a network to a new target domain by utilizing a pretrained ATR network in the source domain. We propose an unpaired transductive transfer learning framework where a CycleGAN model and a well-trained ATR classifier in the source domain are used to construct an ATR classifier in the target domain without having any labeled data in the target domain. We employ a CycleGAN model to transfer the mid-wave infrared (MWIR) images to visible (VIS) domain images (or visible to MWIR domain). To train the transductive CycleGAN, we optimize a cost function consisting of the adversarial, identity, cycle-consistency, and categorical cross-entropy loss for both the source and target classifiers. In this paper, we perform a detailed experimental analysis on the challenging DSIAC ATR dataset. The dataset consists of ten classes of vehicles at different poses and distances ranging from 1-5 kilometers on both the MWIR and VIS domains. In our experiment, we assume that the images in the VIS domain are the unlabeled target dataset. We first detect and crop the vehicles from the raw images and then project them into a common distance of 2 kilometers. Our proposed transductive CycleGAN achieves 71.56% accuracy in classifying the visible domain vehicles in the DSIAC ATR dataset.

TBD

This paper considers the problem of aerial view object classification using co-registered electro-optical (EO) and synthetic aperture radar (SAR) images. Both EO and SAR sensors possess different advantages and drawbacks. There have been many research efforts focusing on joint multi-modal machine learning trying to take advantage of both modalities to develop a more performant classifier. These approaches usually assume the images produced by both modalities are consistent, meaning they contain the information of the same target. However, due to the limitation of EO sensor, it is not always true. For example, aerial viewed EO images may suffer from cloud occlusion. In some cases, inclusion of the cloud occluded EO images for inference may limit performace. This paper proposes an approach to detect if the EO-SAR chip pair contains cloud occluded EO image or not. We use the term “class disagreement detection” (CDD) to describe the mechanism to distinguish the normal EO chips from the corrupted EO chips by treating the corrupted EO chips as another class which is different from the class of the target in the corresponding SAR chips. The EOSAR-CDD machine-learning based approach is to encode EO and SAR features in a way that the distances between the features of the same classes are small while the distances between the features of different classes are large. The EOSAR-CCD can be utilized to construct a simple yet effective modality selection-based EO-SAR fusion scheme that outperforms a popular EO-SAR fusion scheme.

Synthetic Aperture Radar (SAR) images are commonly utilized in military applications for automatic target recognition (ATR). Machine learning (ML) methods, such as Convolutional Neural Networks (CNN) and Graph Neural Networks (GNN), are frequently used to identify ground-based objects, including battle tanks, personnel carriers, and missile launchers. Determining the vehicle class, such as the BRDM2 tank, BMP2 tank, BTR60 tank, and BTR70 tank, is crucial, as it can help determine whether the target object is an ally or an enemy. While the ML algorithm provides feedback on the recognized target, the final decision is left to the commanding officers. Therefore, providing detailed information alongside the identified target can significantly impact their actions. This detailed information includes the SAR image features that contributed to the classification, the classification confidence, and the probability of the identified object being classified as a different object type or class. We propose a GNN-based ATR framework that provides the final classified class and outputs the detailed information mentioned above. This is the first study to provide a detailed analysis of the classification class, making final decisions more straightforward. Moreover, our GNN framework achieves an overall accuracy of 99.2% when evaluated on the MSTAR dataset, improving over previous state-of-the-art GNN methods.

Automatic Target Detection (ATD) leverages machine learning to efficiently process datasets that are too large for humans to evaluate quickly enough for practical applications. Technological and natural factors such as the type of sensor, collection conditions, and environment can affect image interpretability. Synthetic Aperture Radar (SAR) sensors are sensitive to different issues from optical sensors. While SAR imagery can be collected at any time of day and in almost any weather conditions, some conditions are uniquely challenging. Properties of targets and the environment can affect the radar signatures. In this experiment, we simulated these effects in quantifiable increments to measure how strongly they impact the performance of a machine learning model when detecting targets. The experiments demonstrate the differences in image interpretability for machine learning vs. human perception.

The desertSim detection dataset consists of more than forty-seven thousand synthetically generated infrared images exhibiting unique characteristics not found in academic datasets typically used for machine learning research. The desertSim set of images provides realistic infrared signatures of armored vehicles under a variety of configurations, engine states, time of day, and clutter conditions. The dataset is publicly available and was created to provide academic researchers a military relevant dataset to support machine learning research. The synthetic infrared image dataset can be used in conjunction with a publicly available real infrared image dataset for experiments having a synthetic data training set and real data test set. Consistency in the nature of the two datasets make them particularly suitable for conducting academic experiments in support of machine learning research.

Deep learning models are currently the models of choice for image classification tasks. But large scale models require large quantities of data. For many tasks, acquiring a sufficient quantity of training data is not feasible. Because of this, an active area of research in machine learning is the field of few sample learning or few shot learning (FSL), with architectures that attempt to build effective models for a low-sample regime. In this paper, we focus on the established few-shot learning algorithm developed by Snell et al.1 We propose an FSL model where the model is produced via traditional encoding with the backend output layer replaced with the prototypical clustering classifier of Snell et al.. We hypothesize that this algorithm’s encoding structure produced by this training may be equivalent to models produced by traditional cross-entropy deep learning optimization. We compare few shot classification performance on unseen classes between models trained using the FSL training paradigm and our hybrid models trained traditionally with softmax, but modified for FSL use. Our empirical results indicate that traditionally trained models can be effectively re-used for few sample classification.

Detection of underwater manmade objects via deep learning solutions is subject to unique challenges particular to sonar imaging, such as sparse target examples on the seafloor, distortions from sonar image construction, and variations of target pose and sediment accumulation. The combined effect of these artifacts in underwater imagery reduces the efficacy of convolutional neural networks, which achieve state-of-the-art performance on photometric imagery. To mitigate these issues, we implement synthetic data generation via factorization of StyleGAN2 and distortion augmentation, and evaluate several object detection architectures. We develop a detection framework to integrate these treatments and present results on a synthetic aperture sonar imagery dataset collected by an uncrewed undersea vehicle.

The rapid adoption of autonomous Unmanned Aerial Vehicles (UAVs) for various real-world applications in both industry and the military is driving the need for efficient UAV surveillance and countering systems, as these vehicles create new threats to the safety of people and assets. These systems typically contain a variety of sensors and effectors, including video sensors that are used for both human confirmation of a potential menacing UAV, and visual servicing of effectors used to counter an aerial threat. In this case, the performance of the system depends on the accuracy of the algorithm chain (classification, localization and threat identification) used for video tracking. In this paper, we study an original approach for temporally stable video tracking of targets. Specifically, we use state-of-the-art algorithms for semantic object detection and then consolidate them with a pose estimation method to enhance the perception performance. This paper compares different approaches on real data.

This paper presents the results of applying several transfer learning front-ends (e.g. Resnet50, Inception, MobileNet) commonly utilized in academia based upon the ImageNet database to perform feature extraction for the DSIAC ATR data set followed by classification layers. This paper describes the performance of a machine learning system (MLS) composed of a feature generating front-end followed by a classification backend trained on electro-optical (EO) and mid-wave infrared (MWIR) imagery from the DSIAC dataset. The baseline MLS architecture achieves over 99 percent accuracy on both the EO and MWIR DSIAC datasets.

For many applications, Neural Network (NN) training remains difficult due to lack of relevant data. Transfer learning is a technique used to combat this challenge by leveraging knowledge gained to solve one problem effectively and “transferring” this knowledge to solve a different ill-posed problem. In this paper we present an optimal transfer learning method for visible to infrared (IR) classification networks. First, principal component analysis (PCA) of collected target and background imagery across the visible and IR domain is performed. This analysis shows clear separation in the visible domain but weak separation of IR features in the last fully connected layers of pretrained VGG11. Our test data shows increased separation in the IR domain at the earlier layers in the features module of VGG11, indicating that improved transfer learning can be accomplished by either retraining from this point, or by generating a new classifier with the earlier NN generated features . In this paper we first gain insights from PCA on intermediate layers of VGG11 to observe statistic separation of our data. From there, we learn new classifiers using the NN extracted features and show increased accuracy when optimal representative layers from VGG11 are used.

The detection and classification of buried objects utilizing long wave infrared (LWIR) imaging is a challenging task. The ability to detect a buried object is reliant on discriminating background noise from surface temperature anomalies induced by the presence of a foreign object below ground surface. The presence of background noise and temperature anomalies in LWIR images containing buried objects is correlated to the ambient environmental conditions. For example, increased solar loading of the soil can lead to increased background noise, while increased volumetric water content of the soil can mask the presence of temperature anomalies due to buried objects. This paper discusses advancements to a proposed environmentally informed two-step automatic target recognition (ATR) algorithm for buried objects and the characterization of environmental phenomenology corresponding to buried objects and background noise. The detection step of the algorithm is based on an edge detection approach and is designed to maximize probability of detection while ignoring the false alarm rate. The classification step filters the false alarms from the true alarms utilizing a novel framework that combines the environmental conditions with the LWIR imagery. The environmentally informed classification algorithm concurrently reasons from a set of environmental conditions recorded by sensors coupled with a region of interest detected in the first step. The classification algorithm combines a CNNbased image machine learning algorithm with a fully connected neural network to extract features on the coupled environmental and image data to ultimately produce a classification. The performance of the algorithm is compared to common machine learning ATRs.

Recent developments in the military domain introduce the need to detect and track hypersonic glide vehicles in Earth’s atmosphere. The Multispectral Object Sensing by Artificial Intelligence-processed Cameras (MOSAIC) experiment is part of the small-satellite ATHENE-1 of the Universit¨at der Bundeswehr M¨unchen. The primary scientific objective of MOSAIC is to demonstrate reliable detection, identification and tracking of hypersonic glide vehicles using primarily a cooled infrared camera and complementary a visual camera. To cope with a large amount of data from both high-resolution cameras in real-time, state-of-the-art computer vision on-board processing methods are used for detection and tracking. The secondary scientific objective is to investigate the efficiency and reliability of Artificial Intelligence (AI) based image processing algorithms and data compression for space applications. This is of particular importance given the high volumes and rates of data. The application of such algorithms requires a reliable and resource-efficient On-Board Computer (OBC) that can withstand the harsh space environment. The approach outlined in this paper envisions a dedicated OBC to manage the AI-based experiments of the satellite, called the Artificial Intelligence capable On-Board Computer (AI-OBC). The AI-OBC includes multiple hardware-based AI accelerators to meet the computational requirements and ensure real-time processing for object detection and tracking. This paper describes the structure of the data processing pipeline and includes the AI-OBC architecture with its connections to both the cameras and the platform’s OBC. Further, the study discusses the training and validation steps of the intended use-cases.

Quadratic Correlation Filters (QCFs) have been shown to be useful at separating target areas of interest from background clutter for two-class discrimination. However, extension to multi-class discrimination is not straightforward, as QCFs primarily maximize the distance of the response of a filtered target area from clutter. Several attempts have been made to extend QCFs to multi-class discrimination using support vector machines and convolutional neural networks. In addition, detection and recognition of targets that are considered “unresolved” are still elusive for neural network architectures like YOLO which have minimum target size requirements. This work will show that the localization aspect of a QCF neural network layer paired with feature extraction layers of a purely convolutional neural network provides a robust solution to this problem. We will compare the recognition accuracy to YOLO outputs, since this is the current state-of-the-art for target localization and recognition for autonomous vehicles.

Recent advancements in image sensor technology and deep learning have led to increased expectations for improving object detection performance in EO/IR systems. However, the large-scale datasets made available for deep learning model training exhibit significant differences in data distribution compared to the multispectral images acquired from the actual target domain, EO/IR systems. Because of this, when a pretrained deep-learning model is applied to real-world issues for multispectral object detection, it does not work well for inputs with a different distribution from a training dataset. Therefore, a dataset for additional fine-tuning is required. However, labeled datasets required for retraining can be challenging to acquire in practice because of security issues and high acquisition costs. These domain differences and dataset issues are crucial challenges concerning the use of AI in military applications. We herein propose an unsupervised domain adaptation method for object detection in multispectral images with domain discrepancies with a pretrained model on a large public dataset. We train an encoder network without direct labeling to generate feature maps similar to the backbone applied to existing detectors. The proposed approach has the advantage of being able to directly utilize pretrained object detection models from large-scale public datasets by simply training the encoder network alone.

Many tasks previously performed by human analysts watching videos (live or recorded) are now being done by machine learning and artificial intelligence-based tools. Image enhancement techniques have been developed over decades to improve the quality of video for human operators. However, little work has been done to determine if this same approach can improve the efficacy of automated tools. In this paper, we briefly look at the improvement image enhancement can provide to common tasks such as detection, classification, and identification.

The automatic identification system, or AIS, transmits a vessel’s position, identity, and speed information so that other vessels are aware of the transmitting vessel and can avoid collisions. However, AIS has multiple limitations, including the quality, availability, and validity of data for non-cooperative vessels. We investigate methods of supporting the availability of AIS operation through an emergency response of cooperative vessels, thereby creating an opportunistic resource utilization network, or Oppnet. An Oppnet invites and integrates heterogeneous devices and systems available in the given environment, wherein the integrated devices/systems become the Oppnet's helpers. Helpers are resources that deliver support, communication, computation, and control to find missing cooperative vessels. We design and build this Oppnet scenario using the Common Open Research Emulator (CORE), an open-source network emulation tool funded by a U.S. Naval Research Laboratory research project. CORE provides a graphic user interface (GUI) tool for creating and tracking virtual network status and performance as well as a Python API to programmatically access, control, and monitor instances. During the simulation, the Oppnet starts by inviting and integrating heterogeneous nodes (devices or systems) available within its reach, becoming helpers. The Oppnet then assigns tasks to all helpers to detect the missing cooperative vessel, “Nemo”. In each iteration, the simulation runs for a specified time until Oppnet’s helpers find “Nemo”. Otherwise, “Nemo” is lost. Initial results of simulation show that the finding “Nemo” rate increase when Oppnet helpers increase; however, the timespan of finding “Nemo” remains nearly identical due to simulation conditions and parameters.

The detection and recognition of targets within imagery and video analysis is vital for military and commercial applications. The development of infrared sensor devices for tactical aviation systems imagery has increased the performance of target detection. Due to the advancements of infrared sensors capabilities, their use for field operations such as visual operations (visops) or reconnaissance missions that take place in a variety of operational environments have become paramount. Many techniques implemented stretch back to 1970, but were limited due to computational power. The AI industry has recently been able to bridge the gap between traditional signal processing tools and machine learning. Current state of the art target detection and recognition algorithms are too bloated to be applied for on ground or aerial mission reconnaissance. Therefore, this paper proposes Edge IR Vision Transformer (EIR-ViT), a novel algorithm for automatic target detection utilizing infrared images that is lightweight and operates on the edge for easier deployability.

We introduce a new and efficient solution to the Positive and Unlabeled (PU) problem which is tailored specifically for a deep learning framework. We demonstrate the merit of this method using image classification. When only positive and unlabeled images are available for training, our custom loss function, paired with a simple linear transform of the output, results in an inductive classifier where no estimate of the class prior is required. This algorithm, known as the aaPU (Adaptive Asymmetric Positive Unlabeled) algorithm, provides near supervised classification accuracy with very low levels of labeled data on several image benchmark sets. aaPU demonstrates significant performance improvements over current state-of-the-art positive unlabeled learning algorithms.

Researching gun muzzle flash detection can be costly and time-consuming, as data collection requires specialized equipment to be set up at various ranges and angles. This process is further complicated by the need to hire licensed weapon handlers for each weapon class, and by the scarcity of shooting ranges. To address this, we propose a novel approach that uses Generative Adversarial Networks (GANs) to speed up the research process. Specifically, we train a deep convolutional GAN (DCGAN) to generate synthetic muzzle flash waveforms, which can then be used to augment limited training data for deep-learning classifier models. We evaluate the performance of the DCGAN using a lightweight deep-learning model based on ResNet and explore the possibility of re-purposing the trained discriminator as a classifier.