Human fatigue expression recognition through image-based dynamic multi-information and bimodal deep learning

Lei Zhao; Zengcai Wang; Xiaojin Wang; Yazhou Qi; Qing Liu; Guoxin Zhang

doi:10.1117/1.JEI.25.5.053024

6 October 2016 Human fatigue expression recognition through image-based dynamic multi-information and bimodal deep learning

Lei Zhao, Zengcai Wang, Xiaojin Wang, Yazhou Qi, Qing Liu, Guoxin Zhang

Author Affiliations +

Journal of Electronic Imaging, Vol. 25, Issue 5, 053024 (October 2016). https://doi.org/10.1117/1.JEI.25.5.053024

Abstract

Human fatigue is an important cause of traffic accidents. To improve the safety of transportation, we propose, in this paper, a framework for fatigue expression recognition using image-based facial dynamic multi-information and a bimodal deep neural network. First, the landmark of face region and the texture of eye region, which complement each other in fatigue expression recognition, are extracted from facial image sequences captured by a single camera. Then, two stacked autoencoder neural networks are trained for landmark and texture, respectively. Finally, the two trained neural networks are combined by learning a joint layer on top of them to construct a bimodal deep neural network. The model can be used to extract a unified representation that fuses landmark and texture modalities together and classify fatigue expressions accurately. The proposed system is tested on a human fatigue dataset obtained from an actual driving environment. The experimental results demonstrate that the proposed method performs stably and robustly, and that the average accuracy achieves 96.2%.

Citation Download Citation

Lei Zhao, Zengcai Wang, Xiaojin Wang, Yazhou Qi, Qing Liu, and Guoxin Zhang "Human fatigue expression recognition through image-based dynamic multi-information and bimodal deep learning," Journal of Electronic Imaging 25(5), 053024 (6 October 2016). https://doi.org/10.1117/1.JEI.25.5.053024

Published: 6 October 2016

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available