This PDF file contains front matter associated with SPIE Proceedings Volume 7346, including the Title Page, Copyright information, Table of Contents, Introduction (if any), and the Conference Committee listing.

Protecting critical infrastructure systems, such as electrical power grids, has become a primary concern for many governments and organizations across a variety of stakeholder perspectives. Critical infrastructures involve multidimensional, highly complex collections of technologies, processes, and people, and as such, are vulnerable to potentially catastrophic failures on many levels. Moreover, cross-infrastructure dependencies can give rise to cascading effects with escalating impact across multiple infrastructures. Critical infrastructure protection involves both safeguarding against potential disaster scenarios and effective response in the aftermath of infrastructure failure. Our research is developing innovative approaches to modeling critical infrastructures in order to support decision-making during reconstitution efforts in response to infrastructure disruptions. By modeling the impact of infrastructure elements, both within and across infrastructures, we can recommend focus areas for reconstitution resources across different stakeholders in the context of their current goals. An interactive geovisualization interface provides a natural context for this infrastructure analysis support. This paper presents an overview of our approach and the GIS modeling environment under development for decision support in critical infrastructure reconstitution.

This paper presents key components of the Law Enforcement Information Framework (LEIF), an information system that provides communications, situational awareness, and visual analytics tools in a service-oriented architecture supporting web-based desktop and handheld device users. LEIF simplifies interfaces and visualizations of wellestablished visual analytic techniques to improve usability. Advanced analytics capability is maintained by enhancing the underlying processing to support the new interface. LEIF development is driven by real-world user feedback gathered through deployments at three operational law enforcement organizations in the U.S. The system incorporates a robust information ingest pipeline supporting a wide variety of information formats. LEIF also insulates interface and analytical components from information sources making it easier to adapt the framework for many different data repositories.

Infrastructure management (and its associated processes) is complex to understand, perform and thus, hard to make efficient and effective informed decisions. The management involves a multi-faceted operation that requires the most robust data fusion, visualization and decision making. In order to protect and build sustainable critical assets, we present our on-going multi-disciplinary large-scale project that establishes the Integrated Remote Sensing and Visualization (IRSV) system with a focus on supporting bridge structure inspection and management. This project involves specific expertise from civil engineers, computer scientists, geographers, and real-world practitioners from industry, local and federal government agencies. IRSV is being designed to accommodate the essential needs from the following aspects: 1) Better understanding and enforcement of complex inspection process that can bridge the gap between evidence gathering and decision making through the implementation of ontological knowledge engineering system; 2) Aggregation, representation and fusion of complex multi-layered heterogeneous data (i.e. infrared imaging, aerial photos and ground-mounted LIDAR etc.) with domain application knowledge to support machine understandable recommendation system; 3) Robust visualization techniques with large-scale analytical and interactive visualizations that support users' decision making; and 4) Integration of these needs through the flexible Service-oriented Architecture (SOA) framework to compose and provide services on-demand. IRSV is expected to serve as a management and data visualization tool for construction deliverable assurance and infrastructure monitoring both periodically (annually, monthly, even daily if needed) as well as after extreme events.

Motivated by the problem of uncovering networks of illicit actors in complex urban environments, we present a prototype system for intuitive navigation of vehicle track data via interacting map and network views. Our system combines 3D geospatial visualization, social network display and interactive track search software, and it provides a multi-touch interface for operators to navigate urban scenes and investigate potentially suspicious vehicle activity. We describe a case study to highlight the system's capabilities using ground truth vehicle data collected during a 2007 urban exercise. This data is most naturally viewed as tracks in space and time. But as cluttered track displays obscure potentially important actor relationships, our system provides a social network picture whose condensed format is easier to interpret. Through coordinated space-time/vehicle network searches, we demonstrate how analysts can uncover "Red" activities of tactical significance.

Infrastructure safety affects millions of U.S citizens in many ways. Among all the infrastructures, the bridge plays a significant role in providing substantial economy and public safety. Nearly 600,000 bridges across the U.S are mandated to be inspected every twenty-four months. Although these inspections could generate great amount of rich data for bridge engineers to make critical maintenance decisions, processing these data has become challenging due to the low efficiency from those traditional bridge management systems. In collaboration with North Carolina Department of Transportation (NCDOT) and other regional DOT collaborators, we present our knowledge integrated visual analytics bridge management system. Our system aims to provide bridge engineers a highly interactive data exploration environment as well as knowledge pools for corresponding bridge information. By integrating the knowledge structure with visualization system, our system could provide comprehensive understandings of the bridge assets and enables bridge engineers to investigate potential bridge safety issues and make maintenance decisions.

Monitoring the surveillance of large sea areas normally involves the analysis of huge quantities of heterogeneous data from multiple sources (radars, cameras, automatic identification systems, reports, etc.). The rapid identification of anomalous behavior or any threat activity in the data is an important objective for enabling homeland security. While it is worth acknowledging that many existing mining applications support identification of anomalous behavior, autonomous anomaly detection systems are rarely used in the real world. There are two main reasons: (1) the detection of anomalous behavior is normally not a well-defined and structured problem and therefore, automatic data mining approaches do not work well and (2) the difficulties that these systems have regarding the representation and employment of the prior knowledge that the users bring to their tasks. In order to overcome these limitations, we believe that human involvement in the entire discovery process is crucial. Using a visual analytics process model as a framework, we present VISAD: an interactive, visual knowledge discovery tool for supporting the detection and identification of anomalous behavior in maritime traffic data. VISAD supports the insertion of human expert knowledge in (1) the preparation of the system, (2) the establishment of the normal picture and (3) in the actual detection of rare events. For each of these three modules, VISAD implements different layers of data mining, visualization and interaction techniques. Thus, the detection procedure becomes transparent to the user, which increases his/her confidence and trust in the system and overall, in the whole discovery process.

Content-based video retrieval from archived image/video is a very attractive capability of modern intelligent video surveillance systems. This paper presents an innovative Semantic-Based Video Indexing and Retrieval (SBVIR) software toolkit to help users of intelligent video surveillance to easily and rapidly search the content of large video archives to conduct video-based forensic and image intelligence. Tailored for maritime environment, SBVIR is suited for surveillance applications in harbor, sea shores, or around ships. The system comprises two major modules: a video analytic module that performs automatic target detection, tracking, classification, activities recognition, and a retrieval module that performs data indexing, and information retrieval. SBVIR is capable of detecting and tracking objects from multiple cameras robustly in condition of dynamic water background and illumination changes. The system provides hierarchical target classification among a large ontology of watercraft classes, and is capable of recognizing a variety of boat activities. Video retrieval is achieved with both query-by-keyword and query-by-example. Users can query video content using semantic concepts selected from a large dictionary of objects and activities, display the history linked to a given target/activity, and search for anomalies. The user can interact with the system and provide feedbacks to tune the system for improved accuracy and relevance of retrieved data. SBVIR has been tested for real maritime surveillance scenarios and shown to be able to generate highly-semantic metadata tags that can be used during the retrieval to provide user with relevant and accurate data in real-time.

Maritime assets such as ports, harbors, and vessels are vulnerable to a variety of near-shore threats such as small-boat attacks. Currently, such vulnerabilities are addressed predominantly by watchstanders and manual video surveillance, which is manpower intensive. Automatic maritime video surveillance techniques are being introduced to reduce manpower costs, but they have limited functionality and performance. For example, they only detect simple events such as perimeter breaches and cannot predict emerging threats. They also generate too many false alerts and cannot explain their reasoning. To overcome these limitations, we are developing the Maritime Activity Analysis Workbench (MAAW), which will be a mixed-initiative real-time maritime video surveillance tool that uses an integrated supervised machine learning approach to label independent and coordinated maritime activities. It uses the same information to predict anomalous behavior and explain its reasoning; this is an important capability for watchstander training and for collecting performance feedback. In this paper, we describe MAAW's functional architecture, which includes the following pipeline of components: (1) a video acquisition and preprocessing component that detects and tracks vessels in video images, (2) a vessel categorization and activity labeling component that uses standard and relational supervised machine learning methods to label maritime activities, and (3) an ontology-guided vessel and maritime activity annotator to enable subject matter experts (e.g., watchstanders) to provide feedback and supervision to the system. We report our findings from a preliminary system evaluation on river traffic video.

The goal of visual analytical tools is to support the analytical reasoning process, maximizing human perceptual, understanding and reasoning capabilities in complex and dynamic situations. Visual analytics software must be built upon an understanding of the reasoning process, since it must provide appropriate interactions that allow a true discourse with the information. In order to deepen our understanding of the human analytical process and guide developers in the creation of more efficient anomaly detection systems, this paper investigates how is the human analytical process of detecting and identifying anomalous behavior in maritime traffic data. The main focus of this work is to capture the entire analysis process that an analyst goes through, from the raw data to the detection and identification of anomalous behavior. Three different sources are used in this study: a literature survey of the science of analytical reasoning, requirements specified by experts from organizations with interest in port security and user field studies conducted in different marine surveillance control centers. Furthermore, this study elaborates on how to support the human analytical process using data mining, visualization and interaction methods. The contribution of this paper is twofold: (1) within visual analytics, contribute to the science of analytical reasoning with practical understanding of users tasks in order to develop a taxonomy of interactions that support the analytical reasoning process and (2) within anomaly detection, facilitate the design of future anomaly detector systems when fully automatic approaches are not viable and human participation is needed.

In the information age today, we are experiencing an explosion of data and information from a variety of sources unlike anything that the world has seen before. While technology has advanced to keep up with the collection and storage of data, what we lack now is the ability to analyze and understand the meaning behind the data. Traditionally, data mining and data management techniques require the data to be uniform such that a single process can search for knowledge within the data. However, in analysis of complex tasks where knowledge and information need to be pieced together from different sources of data, a new paradigm is required. In this paper, we present a framework of using visual analytical approaches to integrate multiple heterogeneous processes that can each analyze a specific type of data. Under this framework, stand-alone software solutions can focus on specific aspects of the problem based on domain-specific techniques. The framework serves as a visual repository for all the information and knowledge discovered by each individual process, and allows the user to interactively perform sense-making analysis to form a cohesive and comprehensive understanding of the problem at hand. We demonstrate the effectiveness of this framework by applying it to inspecting bridge conditions that utilizes data sources from 2D imagery, 3D LiDAR, and multi-dimensional data based on bridge reports.

Aviation disaster prevention has always been of interest to homeland security, especially after the recent use of aircrafts as weapons by terrorists. With better understanding of the deficiency of different types of aircraft and their corresponding effects on the craft's safety, better maintenance and response plans can be devised to prevent disasters from occurring. In this paper, we present a visual analytical technique to examine the Federal Aviation Agency's Accident/Incident Database, which contains more than 90,000 incidents across 53 dimensions over the last 30 years, for identifying trends of relationships between dimensions over time. Our technique is based on the integration of the ThemeRiver technique directly within a parallel coordinates framework, and simultaneously presents both a "forward flow view" and a "backward flow view" between each dimension. The forward flow view shows the trends over time of each of the elements-of-interest in the first dimension, while the backward flow view illustrates how the elements in the second dimension contribute to the overall trends seen in the first dimension. Through the use of our technique, we were able to identify characteristics of aircrafts and suggest plausible explanations to their common failures.

In time critical visual analytic environments collaboration between multiple expert users allows for rapid knowledge discovery and facilitates the sharing of insight. New collaborative display technologies, such as multi-touch tables, have shown great promise as the medium for such collaborations to take place. However, under such new technologies, traditional selection techniques, having been developed for mouse and keyboard interfaces, become inconvenient, inefficient, and in some cases, obsolete. We present selection techniques for multi-touch environments that allow for the natural and efficient selection of complex regions-of-interest within a hierarchical geospatial environment, as well as methods for refining and organizing these selections. The intuitive nature of the touch-based interaction permits new users to quickly grasp complex controls, while the consideration for collaboration coordinates the actions of multiple users simultaneously within the same environment. As an example, we apply our simple gestures and actions mimicking real-world tactile behaviors to increase the usefulness and efficacy of an existing urban growth simulation in a traditional GIS-like environment. However, our techniques are general enough to be applied across a wide range of geospatial analytical applications for both domestic security and military use.

Autonomous and network centric smart cameras for use in homeland security and other human activities monitoring applications require a multi-layer approach for real time image processing. We propose a novel method to achieve behavior digitization and preemptive course of action (COA) analysis by converting temporal and spatial pixel subframes into a form that can be encoded into equation based Data Models. Output from these Data Models is fused with evidence and sensor data in the COA decision cascade, which recommends COAs that yield evidence. Evidence from the decision cascade continues to be amassed until the hypothesized threat forms a strong enough conviction to initiate alert responses and external intercepting events. This paper outlines our proposed methodology and approach.