Video summarization
K-nearest neighbour methods. 22 Additionally, combining prior information from specific domains with clustering methods can yield better results. Two other studies [23][24] further integrated colour information as features for clustering, achieving remarkable performance. 23 The algorithm flowchart is shown in Figure 5. GAN-based methods primarily learn summarization by distinguishing between the original video and summary segments generated by the generator. The model typically consists of a generator and a discriminator: the generator estimates frame importance and produces summaries, while the discriminator judges the similarity between the generated summary and the original video. When the discriminator fails to distinguish between the two videos, it indicates that the generator has learned to construct summary segments representing the entire video content. In this context, Mahasseni et al. first integrated an LSTM-based keyframe selector
with a variational auto- encoder and a trainable discriminator, learning video summarization through an adversarial training process. 24 Subsequently, Apostolidis et al. proposed a label-based method to train the adversarial component of
Figure 6: A schematic diagram of a video
Figure 7: Diagram of a video
summarization model based on GAN
summarization model based on
network
reinforcement learning
the network. 25 Jung et al. extended the attention n mechanism on the basic framework to evaluate frame dependencies across different temporal granularities during keyframe selection. 26 The model schematic of GAN-based methods is shown in Figure 6.
22 Hadi, Y. et al. (2006) ‘Video summarization by k-medoid clustering,’ in ACM Symposium on Applied
Computing, 2006, pp. 1400–1401 (available at
https://www.researchgate.net/publication/221002204_Video_summarization_by_k-medoid_clustering). 23 Mundur, P. et al. (2006) ‘Keyframe-based video summarization using Delaunay clustering’, International
Journal on Digital Libraries 6.2: 219–232; de Avila, S. et al. (2011) ‘VSUMM: A mechanism designed to produce
static video summaries and a novel evaluation method’, Pattern Recognition Letters 32.1: 56–68. 24 Mahasseni, B. et al. (2017) ‘Unsupervised Video Summarization with Adversarial LSTM Networks,’ IEEE
Conference on Computer Vision and Pattern Recognition , 2017, pp. 2982–2991 (available at
https://web.engr.oregonstate.edu/~sinisa/research/publications/cvpr17_summarization.pdf). 25 Apostolidis, E. et al. (2020) ‘Unsupervised video summarization via attention-driven adversarial learning,’ International Conference on Multimedia Modeling, 2020, pp. 492–504 (available at https://link.springer.com/chapter/10.1007/978-3-030-37731-1_40). 26 Jung, Y. et al. (2019) ‘Discriminative feature learning for unsupervised video summarization,’ AAAI
Conference on Artificial Intelligence (available at https://ojs.aaai.org/index.php/AAAI/article/view/4872).
101
Made with FlippingBook - PDF hosting