Video summarization
Reinforcement learning-based methods mainly achieve video summarization by maximizing reward functions. Zhou et al. proposed the first work in this direction, defining video summarization as a decision-making process and training the summary generator through diversity-representativeness rewards. 27 On this basis, Yaliniz et al. further considered the consistency of generated summaries. 28 Gonuguntla et al. used segment networks within the basic framework of the first work in this field to better extract spatial and temporal information of video frames, training the summary generator via reward functions. 29 Yuan and Zhang enriches features by extracting shot-level information while setting three shot-based reward functions for generator training. 30 Zang et al. employs a Unet network as an encoder-decoder based on diversity-representativeness rewards to further enrich features. 31
3 Datasets, evaluation metrics, and method performance comparisons 3.1 Datasets
This section provides a detailed introduction to four datasets in the field of video summarization, where Cosum and MPII Movie Description Dataset are domain-specific datasets, while TVsum and SumMe are general-purpose datasets. The SumMe dataset consists of 25 first-person view videos from YouTube, featuring diverse contents and themes such as sports, travel, and daily activities. 32 Each video ranges from 1 to 6 minutes in length. For each video, user-generated summaries and their keyframes are provided. The summaries are manually annotated by users who mark the parts they consider most representative of the video content. The TVSum dataset comprises 50 third-person perspective video clips sourced from YouTube, each ranging from 3 to 10 minutes in length. 33 These 27 Zhou, K. et al. (2018) ‘Deep reinforcement learning for unsupervised video summarization with diversity- representativeness reward,’ AAAI Conference on Artificial Intelligence (available at https://ojs.aaai.org/index.php/AAAI/article/view/12255). 28 Yaliniz, G. & Ikizler-Cinbis, N. (2021) ‘Using independently recurrent networks for reinforcement learning based unsupervised video summarization’, Multimedia Tools and Applications 80: 17827–17847. 29 Gonuguntla, N. et al. (2019) ‘Enhanced Deep Video Summarization Network’, British Machine Vision
Conference , 2019 (available at https://keele-repository.worktribe.com/output/414881/enhanced-deep-video-
summarization-network). 30 Yuan, Y. & Zhang, J. (2023) ‘Unsupervised video summarization via deep reinforcement learning with shot- level semantics’, IEEE Transactions on Circuits and Systems for Video Technology 33.1: 445–456. 31 Zang, S. et al. ‘Unsupervised video summarization using deep non-local video summarization networks’, Neurocomputing 519: 26–35. 32 Gygli, M.et al. 2014) ‘Creating Summaries from User Videos,’ ECCV, 2014, pp. 505–520 (available at https://link.springer.com/chapter/10.1007/978-3-319-10584-0_33). 33 Song, Y. et al. (2015) ‘TVSum: Summarizing web videos using titles,’ IEEE Conference on Computer Vision
and Pattern Recognition, 2015, pp. 5179–5187 (available at
https://www.researchgate.net/publication/308861592_TVSum_Summarizing_web_videos_using_titles).
102
Made with FlippingBook - PDF hosting