CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12718 > Article
Paper
1 June 2023 A context-aware model with a pre-trained context encoder for dense video captioning
Weilun Wu, Yang Gao
Author Affiliations +
Proceedings Volume 12718, International Conference on Cyber Security, Artificial Intelligence, and Digital Economy (CSAIDE 2023); 127181J (2023) https://doi.org/10.1117/12.2681586
Event: International Conference on Cyber Security, Artificial Intelligence, and Digital Economy (CSAIDE 2023), 2023, Nanjing, China
Abstract
The task of dense video captioning is to generate descriptions for multiple events in an untrimmed video. Previous works generate descriptions for each event, and then concatenate the individual sentence descriptions into a final paragraph, often leading to inconsistency and redundancy problems. To overcome these limitations, we propose a context-aware model that leverages context event information from both past and future events to provide conditional dependence on the current event description. Moreover, context event information is encoded by a strong pre-trained context encoder and is incorporated into the captioning module through a gate-attention mechanism. Experimental results on both YouCookII and ActivityNet datasets demonstrate that our model outperforms the existing context-aware and pre-trained model by a margin.
© (2023) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Weilun Wu and Yang Gao "A context-aware model with a pre-trained context encoder for dense video captioning", Proc. SPIE 12718, International Conference on Cyber Security, Artificial Intelligence, and Digital Economy (CSAIDE 2023), 127181J (1 June 2023); https://doi.org/10.1117/12.2681586
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 10 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Video
Transformers
Data modeling
Performance modeling
Ablation
Semantic video
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top