Comparing the performance of two observers using a novel utility-based performance metric for ROC analysis

Darrin C. Edwards; Charles E. Metz

doi:10.1117/12.810778

12 March 2009 Comparing the performance of two observers using a novel utility-based performance metric for ROC analysis

Darrin C. Edwards, Charles E. Metz

Author Affiliations +

Proceedings Volume 7263, Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment; 72630W (2009) https://doi.org/10.1117/12.810778
Event: SPIE Medical Imaging, 2009, Lake Buena Vista (Orlando Area), Florida, United States

Abstract

We previously introduced a utility-based ROC performance metric, the "surface-averaged expected cost" (SAEC), to address difficulties which arise in generalizing the well-known area under the ROC curve (AUC) to classification tasks with more than two classes. In a two-class classification task, the SAEC can be shown explicitly to be twice the area above the conventional ROC curve (1-AUC) divided by the arclength along the ROC curve. In the present work, we show that in tasks comparing the performance of two observers whose behavior is described by the proper binormal model, our proposed performance metric is consistent with AUC in the qualitative sense of deciding which of the two observers is "better," and by how wide a margin.

Citation Download Citation

Darrin C. Edwards and Charles E. Metz "Comparing the performance of two observers using a novel utility-based performance metric for ROC analysis", Proc. SPIE 7263, Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment, 72630W (12 March 2009); https://doi.org/10.1117/12.810778

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available