Face and lip tracking in unconstrained imagery for improved automatic speech recognition

Brandon Crow; Jane Xiaozheng Zhang

doi:10.1117/12.817092

19 January 2009 Face and lip tracking in unconstrained imagery for improved automatic speech recognition

Brandon Crow, Jane Xiaozheng Zhang

Proceedings Volume 7257, Visual Communications and Image Processing 2009; 72571Y (2009) https://doi.org/10.1117/12.817092
Event: IS&T/SPIE Electronic Imaging, 2009, San Jose, California, United States

Abstract

When combined with acoustical speech information, visual speech information (lip movement) significantly improves Automatic Speech Recognition (ASR) in acoustically noisy environments. Previous research has demonstrated that visual modality is a viable tool for identifying speech. However, the visual information has yet to become utilized in mainstream ASR systems due to the difficulty in accurately tracking lips in real-world conditions. This paper presents our current progress in tracking face and lips in visually challenging environments. Findings suggest the mean shift algorithm performs poorly for small regions, in this case the lips, but it achieves near 80% accuracy for facial tracking.

Citation Download Citation

Brandon Crow and Jane Xiaozheng Zhang "Face and lip tracking in unconstrained imagery for improved automatic speech recognition", Proc. SPIE 7257, Visual Communications and Image Processing 2009, 72571Y (19 January 2009); https://doi.org/10.1117/12.817092

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available