Expert viewing protocol performance study: the case of subjective evaluation of HDR coding

Vittorio Baroncini; Giacomo Baroncini; Pankaj Topiwala

doi:10.1117/12.2240257

27 September 2016 Expert viewing protocol performance study: the case of subjective evaluation of HDR coding

Vittorio Baroncini, Giacomo Baroncini, Pankaj Topiwala

Proceedings Volume 9971, Applications of Digital Image Processing XXXIX; 99710I (2016) https://doi.org/10.1117/12.2240257
Event: SPIE Optical Engineering + Applications, 2016, San Diego, California, United States

Abstract

This paper tries to examine the results of a subjective evaluation experiment, made by means of the new Expert Viewing Protocol, recently approved by ITU-R Study Group 6 [1]. The EVP subjective test was designed and performed to compare different HDR coding technologies during an MPEG meeting (San Diego, CA, February 2016) [2]. Thanks to the wide and enthusiastic participation of the MPEG experts to the subjective evaluation experiment, it was possible to collect data from a total of sixteen viewers; this allowed to perform a sort of “validation” of the performance of the EVP. The ITU-R Recommendation states that tests with nine viewers is sufficient to get acceptable results from an EVP experiment. In our case, having data from 16 viewers, it was possible to compute the MOS and the Confidence Interval data as if it were a standard subjective assessment experiment (which typically requires more viewers). This allowed a sort of “validation” of the results obtained using results from 9 experts only vs. the results obtained using the data from the 16 viewers. The analysis of the raw data showed a rather good conversion of the EVP results towards the results obtained using the full viewers’ data set. The results of the EVP evaluation of MPEG HDR content was described in details in a previous paper [3], to which we defer for details on the EVP protocol procedure and rules. This paper instead tries to answer to a demand for further clarification on the “context” and “limitations of use” of the EVP when performed in alternative to a formal subjective experiment trial.

Conference Presentation

Citation Download Citation

Vittorio Baroncini, Giacomo Baroncini, and Pankaj Topiwala "Expert viewing protocol performance study: the case of subjective evaluation of HDR coding", Proc. SPIE 9971, Applications of Digital Image Processing XXXIX, 99710I (27 September 2016); https://doi.org/10.1117/12.2240257

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available