Spatio-temporal dual-attention network for view-invariant human action recognition

Kumie Gedamu; Getinet Yilma; Maregu Assefa; Melese Ayalew

doi:10.1117/12.2643446

12 October 2022 Spatio-temporal dual-attention network for view-invariant human action recognition

Kumie Gedamu, Getinet Yilma, Maregu Assefa, Melese Ayalew

Proceedings Volume 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022); 123420Q (2022) https://doi.org/10.1117/12.2643446
Event: Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 2022, Wuhan, China

Abstract

Due to the action occlusion and information loss caused by the view changes, view-invariant human action recognition is challenging in plenty of real-world applications. One possible solution to this problem is minimizing representation discrepancy in different views while learning discriminative feature representation for view-invariant action recognition. To solve the problem, we propose a Spatio-temporal Dual-Attention Network (SDA-Net) for view-invariant human action recognition. The SDA-Net is composed of a spatial/temporal self-attention and spatial/temporal cross-attention modules. The spatial/temporal self-attention module captures global long-range dependencies of action features. The cross-attention module is designed to learn view-invariant co-occurrence attention maps and generates discriminative features for a semantic representation of actions in different views. We exhaustively evaluate our approach on the NTU- 60, NTU-120, and UESTC datasets with multi-type evaluations, i.e., Cross-Subject, Cross-View, Cross-Set, and Arbitrary-view. Extensive experiment results demonstrate that our approach exceeds the state-of-the-art approaches with a significant margin in view-invariant human action recognition.

Citation Download Citation

Kumie Gedamu, Getinet Yilma, Maregu Assefa, and Melese Ayalew "Spatio-temporal dual-attention network for view-invariant human action recognition", Proc. SPIE 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 123420Q (12 October 2022); https://doi.org/10.1117/12.2643446

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Cameras

Convolution

Video

RGB color model

Data modeling

Feature extraction

Matrices

Show All Keywords

Keywords/Phrases

Search In:

Publication Years