Regular Articles

Multiview fusion for activity recognition using deep neural networks

[+] Author Affiliations
Rahul Kavi, Vinod Kulathumani, Fnu Rohit

West Virginia University, Department of Computer Science and Electrical Engineering, 109 Research Way, Morgantown, West Virginia 26506-6109, United States

Vlad Kecojevic

West Virginia University, Department of Mining Engineering, 395 Evansdale Drive, Morgantown, West Virginia 26506-6070, United States

J. Electron. Imaging. 25(4), 043010 (Jul 18, 2016). doi:10.1117/1.JEI.25.4.043010
History: Received February 17, 2016; Accepted June 27, 2016
Text Size: A A A

Abstract.  Convolutional neural networks (ConvNets) coupled with long short term memory (LSTM) networks have been recently shown to be effective for video classification as they combine the automatic feature extraction capabilities of a neural network with additional memory in the temporal domain. This paper shows how multiview fusion can be applied to such a ConvNet LSTM architecture. Two different fusion techniques are presented. The system is first evaluated in the context of a driver activity recognition system using data collected in a multicamera driving simulator. These results show significant improvement in accuracy with multiview fusion and also show that deep learning performs better than a traditional approach using spatiotemporal features even without requiring any background subtraction. The system is also validated on another publicly available multiview action recognition dataset that has 12 action classes and 8 camera views.

Figures in this Article
© 2016 SPIE and IS&T

Citation

Rahul Kavi ; Vinod Kulathumani ; Fnu Rohit and Vlad Kecojevic
"Multiview fusion for activity recognition using deep neural networks", J. Electron. Imaging. 25(4), 043010 (Jul 18, 2016). ; http://dx.doi.org/10.1117/1.JEI.25.4.043010


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.