Regular Articles

Comparative study of localization metrics for the evaluation of image interpretation systems

[+] Author Affiliations
Baptiste Hemery

École Nationale Supérieure d’Ingéneurs de Caen,Université de Caen Basse-Normandie, CNRS, Laboratoire GREYC, 6 Boulevard du Maréchal Juin, 14000 Caen, France

Helene Laurent

ENSI de Bourges, Université d’Orléans, Institut PRISME, 88 Boulevard Lahitolle, 18020 Bourges, France

Bruno Emile

ENSI de Bourges, Université d’Orléans, Institut PRISME, 88 Boulevard Lahitolle, 18020 Bourges, France

Christophe Rosenberger

École Nationale Supérieure d’Ingéneurs de Caen, Université de Caen Basse-Normandie, CNRS, Laboratoire GREYC, 6 Boulevard du Maréchal Juin, 14000 Caen, France

J. Electron. Imaging. 19(2), 023017 (June 16, 2010). doi:10.1117/1.3446803
History: Received October 26, 2009; Revised April 23, 2010; Accepted April 28, 2010; Published June 16, 2010; Online June 16, 2010
Text Size: A A A

Image interpretation is particularly important in many real applications (video monitoring, biometrics, etc.). Due to the proliferation of image interpretation systems in the literature, their evaluation still remains a crucial stake. Among all the tasks in this field, the quality of object localization is often evaluated through an evaluation metric. We propose to review these techniques and study their reliability. We first propose a generic definition of a localization algorithm. Then, different state of the art techniques to evaluate image interpretation results are detailed. Secondly, we focus on metrics that enable us to evaluate localization results. We propose a general methodology to analyze the behavior of an evaluation metric, considered here as a black box (its definition is not even supposed to be known). We define the properties that these metrics should fulfill. We then perform a comparative study of 33 localization metrics from the state of the art. Experimental results conducted on a large and significant image database permit us to determine metrics that should be used in the future for the evaluation of object localization results.

Figures in this Article
© 2010 SPIE and IS&T

Citation

Baptiste Hemery ; Helene Laurent ; Bruno Emile and Christophe Rosenberger
"Comparative study of localization metrics for the evaluation of image interpretation systems", J. Electron. Imaging. 19(2), 023017 (June 16, 2010). ; http://dx.doi.org/10.1117/1.3446803


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.