Regular Articles

Video analysis using spatiotemporal descriptor and kernel extreme learning machine for lip reading

[+] Author Affiliations
Longbin Lu, Xinman Zhang, Dongpeng Shang

Xi’an Jiaotong University, School of Electronics and Information Engineering, MOE Key Lab for Intelligent Networks and Network Security, No. 28, Xianning West Road, Xi’an 710049, China

Xuebin Xu

Guangdong Xi'an Jiaotong University Academy, No. 3, Daliangdesheng East Road, Foshan, Guangdong 528000, China

J. Electron. Imaging. 24(5), 053023 (Oct 06, 2015). doi:10.1117/1.JEI.24.5.053023
History: Received May 7, 2015; Accepted September 1, 2015
Text Size: A A A

Abstract.  Lip-reading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. We aim to solve two important issues regarding lip reading: (1) how to extract discriminative lip motion features and (2) how to establish a classifier that can provide promising recognition accuracy for lip reading. For the first issue, a projection local spatiotemporal descriptor, which considers the lip appearance and motion information at the same time, is utilized to provide an efficient representation of a video sequence. For the second issue, a kernel extreme learning machine (KELM) based on the single-hidden-layer feedforward neural network is presented to distinguish all kinds of utterances. In general, this method has fast learning speed and great robustness to nonlinear data. Furthermore, quantum-behaved particle swarm optimization with binary encoding is introduced to select the appropriate feature subset and parameters for KELM training. Experiments conducted on the AVLetters and OuluVS databases show that the proposed lip-reading method achieves a superior recognition accuracy compared with two previous methods.

Figures in this Article
© 2015 SPIE and IS&T

Citation

Longbin Lu ; Xinman Zhang ; Xuebin Xu and Dongpeng Shang
"Video analysis using spatiotemporal descriptor and kernel extreme learning machine for lip reading", J. Electron. Imaging. 24(5), 053023 (Oct 06, 2015). ; http://dx.doi.org/10.1117/1.JEI.24.5.053023


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.