Redefining automatic target recognition (ATR) performance standards

Donald Waagen; Charles Hester; Ben Schmid; Margaret Phillips; M. Shane Thompson; Steven Vanstone; Kelly Risko

doi:10.1117/12.886308

19 May 2011 Redefining automatic target recognition (ATR) performance standards

Donald Waagen, Charles Hester, Ben Schmid, Margaret Phillips, M. Shane Thompson, Steven Vanstone, Kelly Risko

Proceedings Volume 8049, Automatic Target Recognition XXI; 80490Z (2011) https://doi.org/10.1117/12.886308
Event: SPIE Defense, Security, and Sensing, 2011, Orlando, Florida, United States

Abstract

Present descriptors for Automatic Target Recognition (ATR) performance are inadequate for use in comparing algorithms that are purported to be a solution to the problem. The use of receiver operator characteristic curves (ROCs) is a defacto standard, but they do not communicate several key performance measures, including (i) intrinsic separation between classes in the input space, (ii) the efficacy of the mapping induced by the algorithm, (iii) the complexity of the algorithmic mapping, and (iv) a measure of the generalization of the proposed solution. Previous work by Sims et. al.^2,5 has addressed the distortion of the evaluation sets to indicate an algorithm's capability (or lack thereof) for generalization and handling of unspecified cases. This paper addresses the rethinking of the summary statistics used for understanding the performance of a solution. We propose new approaches for solution characterization, allowing algorithm performance comparison in an equitable and insightful manner. This paper proffers some examples and suggests directions for new work from the community in this field.

Citation Download Citation

Donald Waagen, Charles Hester, Ben Schmid, Margaret Phillips, M. Shane Thompson, Steven Vanstone, and Kelly Risko "Redefining automatic target recognition (ATR) performance standards", Proc. SPIE 8049, Automatic Target Recognition XXI, 80490Z (19 May 2011); https://doi.org/10.1117/12.886308

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available