Divergence as a measure of visual recognition: bias and errors caused by small samples

Manhot Lau; Takashi Okagaki

doi:10.1117/12.50368

1 November 1991 Divergence as a measure of visual recognition: bias and errors caused by small samples

Manhot Lau, Takashi Okagaki

Proceedings Volume 1606, Visual Communications and Image Processing '91: Image Processing; (1991) https://doi.org/10.1117/12.50368
Event: Visual Communications, '91, 1991, Boston, MA, United States

Abstract

Total divergence has been shown to be a useful measure of performance of visual recognition processes. Measurement and comparison of divergences using samples require statistical processing and sample size becomes an important factor in practical applications when sample sizes are small. We recently completed a numerical study on the applicability of total divergence to a medical Pap test process in small samples. This study suggested that in order to keep measurement biases within +/- 0.5 decits, 100 - 200 samples of cytology- histology pairs were required in the best classifications of 3, 4, and 5 category-states. At these sample sizes, measurement errors (standard errors) were also contained within +/- 0.5 decits. This study also confirmed that previously reported, over-estimated propagated errors in small samples were in fact over-estimation, and that their use for testing a null hypothesis was valid. The number of samples with indefinable statistics due to a zero denominator can be as high as 30% when the sample sizes were 500 for 3, 4, and 5 category-state classifications. Biases due to small samples were positive for most category-states except for the optimum 3 category-states, in which bias changed to negative (bias inversion), and observed errors of total divergence paradoxically decreased as N decreased (error-sample paradox) for a small sample size (N < 700).

Citation Download Citation

Manhot Lau and Takashi Okagaki "Divergence as a measure of visual recognition: bias and errors caused by small samples", Proc. SPIE 1606, Visual Communications and Image Processing '91: Image Processing, (1 November 1991); https://doi.org/10.1117/12.50368

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Error analysis

Visualization

Image processing

Monte Carlo methods

Classification systems

Visual communications

Image classification

Show All Keywords

Keywords/Phrases

Search In:

Publication Years