{"entry_id": "0164e41c-25db-44d9-84d0-56eacd22dc78", "infos": [{"txt": "LVC:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.114, 0.134, 0.17, 0.148]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 1, "global_sentence_id": 0, "edu_l1_label": "BOT"}, {"txt": " A Lightweight Compression Framework for Enhancing VLMs in Long", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.17, 0.134, 0.883, 0.15]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 1, "global_sentence_id": 1, "edu_l1_label": "BOT"}, {"txt": "Video Understanding", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.392, 0.157, 0.606, 0.174]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 2, "global_sentence_id": 2, "edu_l1_label": "EDU_O"}, {"txt": "Ziyi Wang1,2*Haoran$\\text {Wu}^{1*}$Yiming Rong1,2*Deyang Jiang1,2,Yixin Zhang1,2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 100000, "bbox": [[0.196, 0.203, 0.804, 0.223]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 3, "global_sentence_id": 3, "edu_l1_label": "EDU_O"}, {"txt": "Yunlong Zhao1,2,Shuang", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 200000, "bbox": [[0.348, 0.225, 0.477, 0.23], [0.478, 0.233, 0.545, 0.237]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 4, "global_sentence_id": 4, "edu_l1_label": "EDU_O"}, {"txt": "1 The Key Laboratory of Cognition and Decision Intelligence for Complex Systems,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.17, 0.241, 0.83, 0.255]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 5, "global_sentence_id": 5, "edu_l1_label": "EDU_O"}, {"txt": "Institute of Automation, Chinese Academy of Sciences", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 400000, "bbox": [[0.284, 0.26, 0.714, 0.27]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 6, "global_sentence_id": 6, "edu_l1_label": "EDU_O"}, {"txt": "2 School of Artificial Intelligence, University of Chinese Academy of Sciences", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.19, 0.275, 0.807, 0.289]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 7, "global_sentence_id": 7, "edu_l1_label": "EDU_O"}, {"txt": "{wangziyi2022, wuhaoran2018, rongyiming2022,jiangdeyang2022}@ia.ac.cn,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 600000, "bbox": [[0.192, 0.296, 0.81, 0.307]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 8, "global_sentence_id": 8, "edu_l1_label": "EDU_O"}, {"txt": "{zhangyixin2024, zhaoyunlong2020,shuang.xu, xubo}@ia.ac.cn", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.237, 0.314, 0.752, 0.324]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 9, "global_sentence_id": 9, "edu_l1_label": "EDU_O"}, {"txt": "SZ07ady6 IAO.s2] TASC890.+OsZ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.029, 0.272, 0.052, 0.646]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 10, "global_sentence_id": 10, "edu_l1_label": "EDU_O"}, {"txt": ":AIXTe", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.052, 0.272, 0.052, 0.701]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 10, "global_sentence_id": 11, "edu_l1_label": "EDU_O"}, {"txt": "Abstract", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.252, 0.364, 0.324, 0.376]]}]}, "tags": ["title"], "label": "abstract", "web_segment_id": 11, "global_sentence_id": 12, "edu_l1_label": "BOS"}, {"txt": "Long video understanding is a complex task that requires both spatial detail and temporal awareness.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 13, "edu_l1_label": "IOS"}, {"txt": "While Vision-Language Models (VLMs) obtain frame-level understand-ing capabilities through multi-frame input, they suffer from information loss due to the sparse sampling strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 14, "edu_l1_label": "IOS"}, {"txt": "In contrast,Video Large Language Models (Video-LLMs) cap-ture temporal relationships within visual features but are limited by the scarcity of high-quality video-text datasets.To transfer long video understanding capabilities to VLMs with minimal data and computational cost,we propose Lightweight Video Compression (LVC), a novel method fea-turing the Query-Attention Video Compression mechanism,which effectively tackles the sparse sampling problem in VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 15, "edu_l1_label": "IOS"}, {"txt": "By training only the alignment layer with 10k short video-text pairs, LVC significantly enhances the temporal reasoning abilities of VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 16, "edu_l1_label": "IOS"}, {"txt": "Extensive experiments show that LVC provides consistent performance improvements across various models, including the InternVL2 series and Phi-3.5-Vision.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 17, "edu_l1_label": "IOS"}, {"txt": "Notably, the InternVL2-40B-LVC achieves scores of 68.2 and 65.9 on the long video understanding benchmarks MLVU and Video-MME, respectively, with rel-ative improvements of$14.6\\%$and $7.7\\%$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 18, "edu_l1_label": "IOS"}, {"txt": "The enhanced mod-els and code will be publicly available soon.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.092, 0.397, 0.484, 0.74]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 19, "edu_l1_label": "IOS"}, {"txt": "1.Introduction", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.096, 0.775, 0.22, 0.787]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 13, "global_sentence_id": 20, "edu_l1_label": "BOS"}, {"txt": "The rapid advancement of large language models (LLMs)has driven a paradigm shift in video understanding re-search, transitioning from conventional vision-centric ap-proaches [4,11,41,43] to LLM-based frameworks that leverage cross-modal alignment capabilities.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.095, 0.802, 0.482, 0.812], [0.095, 0.818, 0.481, 0.826], [0.094, 0.835, 0.481, 0.842], [0.095, 0.85, 0.48, 0.856], [0.095, 0.863, 0.395, 0.871]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 21, "edu_l1_label": "IOS"}, {"txt": "This LLM-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.395, 0.863, 0.481, 0.872]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 22, "edu_l1_label": "IOS"}, {"txt": "*Corresponding author.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.112, 0.89, 0.237, 0.898]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 15, "global_sentence_id": 23, "edu_l1_label": "EDU_O"}, {"txt": "driven revolution manifests in two predominant architec-tures:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.517, 0.367, 0.904, 0.375], [0.517, 0.383, 0.556, 0.391]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 24, "edu_l1_label": "IOS"}, {"txt": " Video-LLMs [3,16,23] pre-trained on video-text-paired data and VLMs [8,17,44] centered on image-text alignment [19,25].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.556, 0.383, 0.904, 0.39], [0.518, 0.4, 0.904, 0.406], [0.517, 0.414, 0.645, 0.42]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 25, "edu_l1_label": "IOS"}, {"txt": "Video-LLMs establish video-text alignment [15,35],projecting sequential frame features into LLM's text space.While Video-LLMs demonstrate superior temporal aware-ness through video pre-training, their practical implementa-tion faces two key bottlenecks:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.538, 0.43, 0.905, 0.439], [0.518, 0.448, 0.905, 0.453], [0.519, 0.46, 0.904, 0.468], [0.518, 0.478, 0.905, 0.482], [0.517, 0.492, 0.721, 0.499]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 26, "edu_l1_label": "IOS"}, {"txt": " limited availability of high-quality video-text datasets [31, 32] and expensive com-putational costs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.721, 0.492, 0.904, 0.499], [0.518, 0.508, 0.904, 0.511], [0.518, 0.523, 0.632, 0.529]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 27, "edu_l1_label": "IOS"}, {"txt": "In contrast, by processing temporally sampled frames as discrete visual inputs, VLMs demon-strate surprising competence in short video understanding tasks [5,20,50].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.632, 0.523, 0.904, 0.532], [0.518, 0.539, 0.904, 0.542], [0.518, 0.552, 0.903, 0.562], [0.518, 0.567, 0.634, 0.574]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 28, "edu_l1_label": "IOS"}, {"txt": "Their strong spatial reasoning abilities,derived from large image-text datasets [10,26,33,37,38],enable them to match or even surpass specialized video ar-chitectures in tasks like temporal grounding [13,15,35]and video captioning [6,27].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.634, 0.567, 0.906, 0.575], [0.518, 0.581, 0.905, 0.59], [0.517, 0.598, 0.904, 0.604], [0.517, 0.612, 0.903, 0.62], [0.518, 0.628, 0.68, 0.635]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 29, "edu_l1_label": "IOS"}, {"txt": "However, the sparse sampling strategy faces challenges when dealing with long videos.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.537, 0.644, 0.904, 0.653], [0.518, 0.662, 0.726, 0.668]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 30, "edu_l1_label": "IOS"}, {"txt": "The limited context length of LLMs restricts the number of sampled frames, caus-ing significant information loss and impairing the ability to capture fine-grained details in the video.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.726, 0.662, 0.904, 0.669], [0.517, 0.677, 0.903, 0.682], [0.518, 0.691, 0.904, 0.698], [0.518, 0.706, 0.792, 0.713]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 31, "edu_l1_label": "IOS"}, {"txt": "To address this,Video-LLMs have introduced memory [14,40] or compres-sion modules [35,48].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.792, 0.706, 0.905, 0.714], [0.518, 0.72, 0.904, 0.725], [0.518, 0.737, 0.668, 0.743]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 32, "edu_l1_label": "IOS"}, {"txt": "However, most of these approaches rely heavily on large-scale pre-training or fine-tuning, fail-ing to effectively utilize the existing capabilities of VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.668, 0.737, 0.904, 0.744], [0.517, 0.752, 0.904, 0.756], [0.517, 0.765, 0.9, 0.774]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 33, "edu_l1_label": "IOS"}, {"txt": "Our key insight is that current VLMs already possess frame-level understanding capabilities required for video tasks, but the sparse sampling strategy fails to provide suf-ficient perceptual information and temporal modeling ca-pabilities.Short video segments, containing enough motion cues, establish a temporal baseline that enables the model to acquire temporal awareness.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.537, 0.783, 0.904, 0.792], [0.517, 0.799, 0.9, 0.807], [0.518, 0.814, 0.905, 0.819], [0.518, 0.828, 0.904, 0.836], [0.518, 0.846, 0.904, 0.852], [0.518, 0.861, 0.904, 0.867], [0.518, 0.876, 0.705, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 19, "global_sentence_id": 34, "edu_l1_label": "IOS"}, {"txt": "This temporal knowledge can then be transferred to long video understanding, providing a", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.705, 0.876, 0.904, 0.883], [0.517, 0.889, 0.904, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 19, "global_sentence_id": 35, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.494, 0.925, 0.506, 0.939]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 20, "global_sentence_id": 36, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.118, 0.086, 0.905, 0.331]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 1, "global_sentence_id": 37, "edu_l1_label": "EDU_O"}, {"txt": "Figure 1. Comparison examples of InternVL2-40B and InternVL2-40B-LVC. Red text indicates errors in InternVL2-40B caused by its sparse sampling strategy failing to capture temporal details, while green text highlights corrections made by InternVL2-40B-LVC.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.092, 0.347, 0.906, 0.372]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 2, "global_sentence_id": 38, "edu_l1_label": "EDU_O"}, {"txt": "robust solution without the need for large-scale pre-training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.094, 0.405, 0.482, 0.411]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 39, "edu_l1_label": "IOS"}, {"txt": "Building on this insight, we propose Lightweight Video Compression(LVC), which introduces the novel parameter-free Query-Attention Video Compression mechanism.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.115, 0.42, 0.481, 0.429], [0.095, 0.436, 0.48, 0.445], [0.095, 0.451, 0.45, 0.46]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 40, "edu_l1_label": "IOS"}, {"txt": "This mechanism transforms discrete sampled features into con-tinuous dynamic representations.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.45, 0.451, 0.48, 0.46], [0.104, 0.469, 0.481, 0.475], [0.095, 0.482, 0.323, 0.49]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 41, "edu_l1_label": "IOS"}, {"txt": "Guided by the input query,LVC extracts key information from long videos, pro-viding a more complete video representation while sup-pressing irrelevant details.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.323, 0.482, 0.481, 0.491], [0.095, 0.499, 0.482, 0.503], [0.095, 0.514, 0.481, 0.521], [0.095, 0.528, 0.27, 0.535]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 42, "edu_l1_label": "IOS"}, {"txt": "By training only the alignment layer of VLMs (approximately 100M parameters), LVC successfully evolves VLMs into video understanding mod-els, fully leveraging the capabilities of VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.27, 0.528, 0.48, 0.535], [0.095, 0.541, 0.48, 0.551], [0.095, 0.559, 0.48, 0.565], [0.094, 0.573, 0.407, 0.58]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 43, "edu_l1_label": "IOS"}, {"txt": "The LVC-enhanced VLMs achieve consistent relative improvements on the long video understanding benchmarks, MLVU and Video-MME.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.407, 0.573, 0.481, 0.578], [0.095, 0.589, 0.48, 0.595], [0.095, 0.604, 0.48, 0.611], [0.095, 0.617, 0.185, 0.626]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 44, "edu_l1_label": "IOS"}, {"txt": "Compared to Video-LLMs,which rely on millions of image-text pairs and hundreds of thousands of video-text pairs for pre-training [15,39] and even more [7],LVC only requires 10k video pairs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.185, 0.617, 0.48, 0.626], [0.095, 0.635, 0.481, 0.641], [0.095, 0.65, 0.482, 0.657], [0.095, 0.662, 0.33, 0.671]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 45, "edu_l1_label": "IOS"}, {"txt": "Moreover, InternVL2-8B-LVC can be trained on a single H100-80G GPU in 5 hours, offering a low-cost and efficient path to high-performance video understanding models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.33, 0.662, 0.482, 0.669], [0.095, 0.679, 0.48, 0.687], [0.095, 0.693, 0.48, 0.701], [0.095, 0.71, 0.379, 0.716]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 46, "edu_l1_label": "IOS"}, {"txt": "In conclusion,our main contributions are as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.379, 0.71, 0.481, 0.717], [0.094, 0.725, 0.342, 0.73]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 47, "edu_l1_label": "IOS"}, {"txt": "·Parameter-free Query-Attention Video Compres-sion.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.113, 0.742, 0.481, 0.749], [0.127, 0.758, 0.161, 0.765]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 48, "edu_l1_label": "IOS"}, {"txt": "We propose a query-guided video dynamic rep-resentation method that enhances the temporal aware-ness of VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.161, 0.758, 0.48, 0.765], [0.127, 0.773, 0.481, 0.78], [0.127, 0.789, 0.226, 0.795]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 49, "edu_l1_label": "IOS"}, {"txt": "It is a plug-and-play approach compat-ible with any VLMs that process videos as sequences of image frames.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.226, 0.789, 0.481, 0.792], [0.127, 0.802, 0.48, 0.811], [0.127, 0.819, 0.238, 0.826]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 50, "edu_l1_label": "IOS"}, {"txt": "·A lightweight framework for long video under-standing.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.113, 0.845, 0.481, 0.852], [0.127, 0.859, 0.195, 0.867]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 51, "edu_l1_label": "IOS"}, {"txt": "We propose a lightweight framework for converting VLMs into video models.By training only the alignment layer with only 10k short videos,the", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.195, 0.859, 0.481, 0.867], [0.127, 0.876, 0.48, 0.884], [0.127, 0.889, 0.48, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 52, "edu_l1_label": "IOS"}, {"txt": "frame-level image-text understanding capabilities of VLMs are fully leveraged for long video understand-ing tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.55, 0.402, 0.904, 0.411], [0.55, 0.417, 0.905, 0.424], [0.55, 0.433, 0.614, 0.442]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 53, "edu_l1_label": "IOS"}, {"txt": "·LVC-enhanced models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.536, 0.455, 0.72, 0.461]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 54, "edu_l1_label": "IOS"}, {"txt": "We introduce InternVL2-8B/26B/40B-LVC and Phi-3.5-Vision-LVC (4B), con-sistently enhancing VLMs' performance on zero-shot long video understanding benchmarks MLVU and Video-MME.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.72, 0.455, 0.905, 0.458], [0.55, 0.468, 0.904, 0.476], [0.549, 0.484, 0.904, 0.491], [0.549, 0.499, 0.902, 0.506], [0.55, 0.515, 0.673, 0.521]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 55, "edu_l1_label": "IOS"}, {"txt": "Specifically, InternVL2-40B-LVC achieve scores of 68.2 and 65.9 on these bench-marks, with relative improvements of 14.6% and 7.7%.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.673, 0.515, 0.902, 0.521], [0.55, 0.53, 0.904, 0.536], [0.551, 0.545, 0.903, 0.551], [0.55, 0.557, 0.592, 0.567]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 56, "edu_l1_label": "IOS"}, {"txt": "Notably, InternVL2-40B-LVC surpass GPT-40on MLVU.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.592, 0.557, 0.904, 0.567], [0.55, 0.575, 0.621, 0.582]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 57, "edu_l1_label": "IOS"}, {"txt": "2.Related Work", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.517, 0.6, 0.653, 0.611]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 9, "global_sentence_id": 58, "edu_l1_label": "BOS"}, {"txt": "2.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.518, 0.625, 0.547, 0.635]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 10, "global_sentence_id": 59, "edu_l1_label": "IOS"}, {"txt": "Vision-Language Models", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.547, 0.625, 0.739, 0.635]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 10, "global_sentence_id": 60, "edu_l1_label": "IOS"}, {"txt": "VLMs aim to perform languaged-based tasks using in-formation from the visual modality.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.518, 0.647, 0.904, 0.656], [0.517, 0.662, 0.769, 0.67]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 61, "edu_l1_label": "IOS"}, {"txt": "Early VLMs [34]adopt contrastive learning to align vision and language em-beddings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.769, 0.662, 0.903, 0.672], [0.518, 0.68, 0.904, 0.683], [0.518, 0.693, 0.584, 0.701]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 62, "edu_l1_label": "IOS"}, {"txt": "With the rapid development of LLMs, VLMs have evolved to utilize LLMs' capabilities to tackle com-plex vision-language tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.584, 0.693, 0.904, 0.701], [0.518, 0.708, 0.904, 0.715], [0.518, 0.725, 0.702, 0.731]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 63, "edu_l1_label": "IOS"}, {"txt": "These models mainly consist of three main components: a vision encoder, a modality alignment module, and an LLM backbone such as [9,42].Modality alignment strategies range from simple linear pro-jection [25,44,56] to more complex approaches such as a Q-Former module with learnable queries [19] or cross-attention mechanisms [2,45].These models typically un-dergo large-scale pre-training and instruction tuning on ex-tensive image-text corpora [21,25],yielding strong per-formance across a variety of vision-language benchmarks.Furthermore, [8,44] unifies single-image, multi-image,and video data, achieving significant improvements in multi-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.702, 0.725, 0.904, 0.731], [0.518, 0.74, 0.904, 0.748], [0.517, 0.755, 0.906, 0.761], [0.525, 0.768, 0.904, 0.774], [0.516, 0.785, 0.904, 0.792], [0.517, 0.801, 0.905, 0.805], [0.518, 0.815, 0.904, 0.821], [0.517, 0.828, 0.904, 0.836], [0.518, 0.845, 0.903, 0.852], [0.517, 0.859, 0.906, 0.867], [0.517, 0.873, 0.903, 0.883], [0.517, 0.891, 0.905, 0.895]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 64, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.493, 0.925, 0.507, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 12, "global_sentence_id": 65, "edu_l1_label": "EDU_O"}, {"txt": "modal tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.095, 0.097, 0.177, 0.104]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 66, "edu_l1_label": "IOS"}, {"txt": "2.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.095, 0.119, 0.124, 0.13]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 2, "global_sentence_id": 67, "edu_l1_label": "IOS"}, {"txt": "Video Large Language Models", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.124, 0.119, 0.36, 0.131]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 2, "global_sentence_id": 68, "edu_l1_label": "IOS"}, {"txt": "Since LLMs rapidly become a research hotspot, many works connect LLMs with visual encoders, leveraging LLMs' language understanding and generation capabilities for video tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.095, 0.143, 0.48, 0.154], [0.095, 0.161, 0.48, 0.169], [0.095, 0.173, 0.48, 0.182], [0.095, 0.189, 0.196, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 69, "edu_l1_label": "IOS"}, {"txt": "Early works, such as [20], use a video trans-former as the encoder, followed by a Q-Former to compress video features.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.196, 0.189, 0.482, 0.194], [0.095, 0.204, 0.48, 0.213], [0.095, 0.221, 0.198, 0.227]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 70, "edu_l1_label": "IOS"}, {"txt": "Later, Video-LLMs tend to utilize a pre-trained visual encoder to process a fixed number of sampled frames.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.198, 0.221, 0.48, 0.227], [0.095, 0.235, 0.48, 0.242], [0.095, 0.249, 0.146, 0.257]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 71, "edu_l1_label": "IOS"}, {"txt": "[31, 32] employ a frozen vision transformer, and [23] leverages [55] for pre-aligning visual features.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.146, 0.249, 0.48, 0.258], [0.095, 0.263, 0.444, 0.272]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 72, "edu_l1_label": "IOS"}, {"txt": "Sub-sequently, Video-LLMs incorporate compression modules to process more input frames within the LLM's context length.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.444, 0.263, 0.48, 0.272], [0.095, 0.282, 0.48, 0.288], [0.095, 0.295, 0.482, 0.302], [0.095, 0.31, 0.141, 0.318]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 73, "edu_l1_label": "IOS"}, {"txt": "[3,51] use pooling to compress visual tokens,[35]adopts a two-layer Q-Former, and [16] applies spatiotem-poral clustering.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.141, 0.31, 0.48, 0.32], [0.095, 0.326, 0.48, 0.332], [0.095, 0.342, 0.205, 0.348]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 74, "edu_l1_label": "IOS"}, {"txt": "As compression module designs evolve,[39] introduces visual summarization tokens, while [30] im-plements a scale-then-compress strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.205, 0.342, 0.483, 0.349], [0.095, 0.354, 0.48, 0.364], [0.095, 0.372, 0.364, 0.379]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 75, "edu_l1_label": "IOS"}, {"txt": "These approaches rely on large-scale video datasets and multi-stage training of compression modules, alignment layers, and LLMs, trading training and data costs for performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.364, 0.372, 0.48, 0.379], [0.094, 0.387, 0.48, 0.394], [0.095, 0.402, 0.48, 0.41], [0.095, 0.416, 0.355, 0.423]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 76, "edu_l1_label": "IOS"}, {"txt": "2.3.Long Video Understanding", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.095, 0.441, 0.333, 0.453]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 4, "global_sentence_id": 77, "edu_l1_label": "IOS"}, {"txt": "Long video understanding faces challenges as high infor-mation density conflicts with LLMs' limited context win-dows.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.094, 0.463, 0.482, 0.469], [0.099, 0.481, 0.481, 0.485], [0.095, 0.494, 0.136, 0.502]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 78, "edu_l1_label": "IOS"}, {"txt": "One approach is hierarchical processing.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.136, 0.494, 0.416, 0.503]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 79, "edu_l1_label": "IOS"}, {"txt": "[24] uses scene detection for video clipping and then employs GPT-4V to understand clip-level video.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.416, 0.494, 0.48, 0.503], [0.095, 0.511, 0.482, 0.515], [0.095, 0.524, 0.312, 0.532]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 80, "edu_l1_label": "IOS"}, {"txt": "[46,49] exemplify agent-based frameworks where LLMs strategically orchestrate vi-sion models through dynamic invocation and output syn-thesis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.312, 0.524, 0.482, 0.53], [0.095, 0.539, 0.48, 0.547], [0.094, 0.556, 0.481, 0.562], [0.095, 0.569, 0.139, 0.578]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 81, "edu_l1_label": "IOS"}, {"txt": "[36,47] facilitate LLM reasoning on long videos through their adaptive tree-based video representation.An-other way is focusing on long-video representation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.139, 0.569, 0.481, 0.578], [0.095, 0.585, 0.48, 0.593], [0.1, 0.602, 0.43, 0.609]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 82, "edu_l1_label": "IOS"}, {"txt": "[14,40]employ the memory mechanism to iteratively store and re-trieve video frame features.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.43, 0.602, 0.48, 0.61], [0.095, 0.616, 0.48, 0.622], [0.095, 0.631, 0.281, 0.638]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 83, "edu_l1_label": "IOS"}, {"txt": "[22] uses context attention to represent each frame with two distinct tokens, and [48] ap-plies a token merging module.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.281, 0.631, 0.48, 0.638], [0.095, 0.646, 0.482, 0.65], [0.095, 0.662, 0.295, 0.668]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 84, "edu_l1_label": "IOS"}, {"txt": "[28] progressively increases resolution and input frames for long-video adaptation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.295, 0.662, 0.48, 0.669], [0.094, 0.677, 0.455, 0.684]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 85, "edu_l1_label": "IOS"}, {"txt": "Hi-erarchical methods are often complex in design and fail to fully utilize video representation information, while data-driven Video-LLMs rely on high-quality video-text pairs for training, which are difficult to acquire at scale.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.455, 0.677, 0.482, 0.681], [0.095, 0.693, 0.48, 0.699], [0.095, 0.705, 0.48, 0.713], [0.095, 0.72, 0.48, 0.729], [0.095, 0.737, 0.4, 0.744]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 86, "edu_l1_label": "IOS"}, {"txt": "3.Methods", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.094, 0.765, 0.187, 0.775]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 6, "global_sentence_id": 87, "edu_l1_label": "BOS"}, {"txt": "3.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.094, 0.79, 0.122, 0.801]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 88, "edu_l1_label": "IOS"}, {"txt": "Problem Formulation of Video Understanding", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.122, 0.79, 0.48, 0.802]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 89, "edu_l1_label": "IOS"}, {"txt": "General video understanding tasks can be formulated as:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.093, 0.812, 0.484, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 90, "edu_l1_label": "IOS"}, {"txt": "given a video $v$ and a textual query $q$, the goal is to pre-dict a textual answer $y$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.093, 0.812, 0.484, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 91, "edu_l1_label": "IOS"}, {"txt": "VLMs process the video into $N$ sampled frames$f_{1},f_{2},\\cdots ,f_{N}$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.093, 0.812, 0.484, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 92, "edu_l1_label": "IOS"}, {"txt": "The length of visual feature tokens must account for the window size limitation of the LLM backbone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.093, 0.812, 0.484, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 93, "edu_l1_label": "IOS"}, {"txt": "Therefore, in typical VLMs,$N$is generally", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.093, 0.812, 0.484, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 94, "edu_l1_label": "IOS"}, {"txt": "set to smaller values, such as 8 or 16 [8, 25].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.516, 0.094, 0.906, 0.179]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 95, "edu_l1_label": "IOS"}, {"txt": "These sam-pled frames are processed through a Vision Transformer,such as CLIP-ViT [34], to obtain sampled frame features $\\boldsymbol {F}_{1}\\boldsymbol {F}_{2}\\cdots \\boldsymbol {F}_{N}\\boldsymbol {F}_{i}\\in \\mathbb {R}^{tx}$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.516, 0.094, 0.906, 0.179]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 96, "edu_l1_label": "IOS"}, {"txt": "Here, $t$ represents the num-ber of tokens describing an image frame, and $d$ denotes the feature dimension of each token.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.516, 0.094, 0.906, 0.179]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 97, "edu_l1_label": "IOS"}, {"txt": "The sampled frame features$\\boldsymbol {F}_{i},i=1,\\cdots ,N$are sequen-tially concatenated along the token dimension and aligned to the LLM's text space using an alignment layer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.516, 0.182, 0.906, 0.27]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 98, "edu_l1_label": "IOS"}, {"txt": "They are then concatenated with a query embeddedas$\\boldsymbol {Q}\\in \\mathbb {R}^{lxd^{\\prime }}$and fed into the LLM for autoregressive decoding to gener-ate the answer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.516, 0.182, 0.906, 0.27]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 99, "edu_l1_label": "IOS"}, {"txt": "$$\\boldsymbol {V}=\\text {Align}\\left(\\left[\\boldsymbol {F}_{1};\\boldsymbol {F}_{2};\\cdots ;\\boldsymbol {F}_{N}\\right]\\right)\\in \\mathbb {R}^{(Nt)xd^{\\prime }}\\tag{1}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.562, 0.297, 0.906, 0.316]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 11, "global_sentence_id": 100, "edu_l1_label": "IOS"}, {"txt": "$$P(\\boldsymbol {Y}\\vert \\boldsymbol {V},\\boldsymbol {Q})=\\prod _{j=1}^{L}P_{LLM}\\left(\\boldsymbol {Y}_{j}\\vert \\boldsymbol {V},\\boldsymbol {Q},\\boldsymbol {Y}_{i<j}\\right)\\tag{2}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.564, 0.321, 0.906, 0.362]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 12, "global_sentence_id": 101, "edu_l1_label": "IOS"}, {"txt": "Here,$d^{\\prime }$ represents the feature dimension in the LLM's text space,$V$ represents the input visual features, and$\\boldsymbol {Y}_{i<j}$represents the text embeddings generated from previous outputs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.516, 0.374, 0.906, 0.434]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 102, "edu_l1_label": "IOS"}, {"txt": "Video-LLMs densely sample $M$ frames and apply fea-ture transformation to ensure that the visual features con-form to the input constraints of the LLM backbone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.516, 0.434, 0.906, 0.583]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 103, "edu_l1_label": "IOS"}, {"txt": "The number of frames $M$ can be set to values such as 64,128,or 256 [18,29,52],allowing Video-LLMs to capture much more temporal information compared to VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.516, 0.434, 0.906, 0.583]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 104, "edu_l1_label": "IOS"}, {"txt": "However,since the $M$sampled frames undergo feature transformation to align with the input requirements of the LLM backbone,this necessitates joint training of the compression module,alignment layers, and the LLM itself.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.516, 0.434, 0.906, 0.583]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 105, "edu_l1_label": "IOS"}, {"txt": "Instead of training the whole model with long video-text-paired data, it uses a parameter-free compression mecha-nism that transforms the features to closely resemble the structure of real frames-both in shape and token concate-nation (including special characters).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.538, 0.586, 0.905, 0.592], [0.517, 0.604, 0.904, 0.607], [0.518, 0.618, 0.904, 0.624], [0.517, 0.633, 0.904, 0.64], [0.518, 0.648, 0.763, 0.655]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 106, "edu_l1_label": "IOS"}, {"txt": "The compressed fea-tures are referred to as pseudo-image frames, which do not rely on extensive data-driven pre-training or fine-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.763, 0.648, 0.905, 0.653], [0.517, 0.663, 0.904, 0.67], [0.517, 0.678, 0.885, 0.685]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 107, "edu_l1_label": "IOS"}, {"txt": "3.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.517, 0.701, 0.546, 0.71]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 16, "global_sentence_id": 108, "edu_l1_label": "IOS"}, {"txt": "Model Architecture", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.546, 0.701, 0.699, 0.71]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 16, "global_sentence_id": 109, "edu_l1_label": "IOS"}, {"txt": "LVC is designed to extend VLMs for long video under-standing with low training cost.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.517, 0.722, 0.904, 0.73], [0.517, 0.74, 0.748, 0.747]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 110, "edu_l1_label": "IOS"}, {"txt": "As shown in Figure 2(c), the overall architecture integrates multi-frame tem-poral information through a Visual Encoder, a parameter-free compression module, an alignment layer, and the LLM backbone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.748, 0.74, 0.904, 0.746], [0.517, 0.753, 0.904, 0.761], [0.518, 0.771, 0.904, 0.777], [0.518, 0.783, 0.904, 0.792], [0.517, 0.798, 0.627, 0.807]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 111, "edu_l1_label": "IOS"}, {"txt": "LVC introduces denser frame sampling to capture fine-grained temporal information and then com-presses them into \"pseudo-images” through the parameter-free compression module.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.627, 0.798, 0.904, 0.809], [0.518, 0.815, 0.904, 0.819], [0.518, 0.831, 0.904, 0.836], [0.518, 0.843, 0.687, 0.852]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 112, "edu_l1_label": "IOS"}, {"txt": "Unlike the pre-training paradigms of VLMs and Video-LLMs shown in Figure 2(a)(b).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.545, 0.859, 0.904, 0.867], [0.517, 0.873, 0.732, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 113, "edu_l1_label": "IOS"}, {"txt": "The only trainable com-ponent is the alignment layer, which projects compressed", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.732, 0.873, 0.904, 0.882], [0.517, 0.891, 0.9, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 114, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.494, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 19, "global_sentence_id": 115, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.089, 0.096, 0.898, 0.46]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 1, "global_sentence_id": 116, "edu_l1_label": "EDU_O"}, {"txt": "Figure 2. (a)VLMs (b)Video-LLMs (c)The overall architecture of LVC.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.287, 0.475, 0.712, 0.485]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 2, "global_sentence_id": 117, "edu_l1_label": "EDU_O"}, {"txt": "features into the text space of the LLM backbone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.095, 0.516, 0.424, 0.525]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 118, "edu_l1_label": "IOS"}, {"txt": "This de-sign allows the well-pre-trained VLM to rapidly adapt to long video inputs without fine-tuning the LLM backbone,achieving both parameter efficiency and temporal aware-ness capability.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.424, 0.516, 0.481, 0.523], [0.095, 0.533, 0.48, 0.54], [0.095, 0.547, 0.483, 0.556], [0.095, 0.564, 0.481, 0.57], [0.095, 0.579, 0.196, 0.586]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 119, "edu_l1_label": "IOS"}, {"txt": "In terms of computational complexity, our method in-creases only the Visual Encoder's computation, while keep-ing LLM-side processing unchanged, offering a novel solu-tion for long-video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.114, 0.593, 0.482, 0.599], [0.095, 0.611, 0.481, 0.617], [0.095, 0.624, 0.481, 0.632], [0.094, 0.64, 0.319, 0.648]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 120, "edu_l1_label": "IOS"}, {"txt": "3.3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.094, 0.666, 0.123, 0.676]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 121, "edu_l1_label": "IOS"}, {"txt": "Infusing Queries into Video Compressing", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.123, 0.666, 0.442, 0.678]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 122, "edu_l1_label": "IOS"}, {"txt": "Densely sampled frames often contain redundant informa-tion, and directly concatenating real frame features in se-quence would far exceed the LLM input limitation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.095, 0.689, 0.48, 0.699], [0.094, 0.706, 0.482, 0.711], [0.094, 0.723, 0.435, 0.729]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 123, "edu_l1_label": "IOS"}, {"txt": "To ad-dress this, we propose the Query-Attention Video Compres-sion mechanism, which leverages the text modality to retain essential information needed for video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.435, 0.723, 0.482, 0.726], [0.094, 0.736, 0.48, 0.743], [0.094, 0.753, 0.48, 0.759], [0.095, 0.768, 0.447, 0.774]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 124, "edu_l1_label": "IOS"}, {"txt": "Specifically,the densely sampled frames are processed through a Vision Transformer and alignment layer to obtain the visual features $V$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.093, 0.782, 0.484, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 125, "edu_l1_label": "IOS"}, {"txt": "Based on the compression ratio, a window length $w$ is set, and $V$ is sliced as:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.093, 0.782, 0.484, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 126, "edu_l1_label": "IOS"}, {"txt": "$$\\boldsymbol {V}=\\left[\\boldsymbol {V}_{1},\\boldsymbol {V}_{2},\\cdots ,\\boldsymbol {V}_{M^{\\prime }t}\\right],\\boldsymbol {V}_{i}\\in \\mathbb {R}^{wxd^{\\prime }},i=1,\\cdots ,M^{\\prime }t.$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.107, 0.869, 0.467, 0.888]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 8, "global_sentence_id": 127, "edu_l1_label": "IOS"}, {"txt": "(3)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.461, 0.889, 0.481, 0.898]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 9, "global_sentence_id": 128, "edu_l1_label": "IOS"}, {"txt": "$\\boldsymbol {V}_{i}$represents the local video features, where the video is divided into windows to ensure that the temporal infor-mation of the video is preserved.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.516, 0.514, 0.906, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 129, "edu_l1_label": "IOS"}, {"txt": "$M^{\\prime }=M/$represents the number of pseudo-image frames after compression.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.516, 0.514, 0.906, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 130, "edu_l1_label": "IOS"}, {"txt": "The textual query $b$ with length $l$ is passed through a Text Embedding layer to obtain the query features$\\boldsymbol {Q}\\in \\mathbb {R}^{x^{\\prime }}$and then the token dimension is averaged to derive the sentence-level query feature:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.516, 0.576, 0.906, 0.633]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 131, "edu_l1_label": "IOS"}, {"txt": "$$\\overline {\\boldsymbol {Q}}=\\frac {1}{l}\\sum _{j=1}^{l}\\boldsymbol {Q}_{j},\\boldsymbol {Q}_{j}\\in \\mathbb {R}^{d^{\\prime }}\\tag{4}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.622, 0.648, 0.906, 0.688]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 12, "global_sentence_id": 132, "edu_l1_label": "IOS"}, {"txt": "The query feature$\\bar {Q}$ is integrated into $V$ by generating weights,which are then used to modulate the visual fea-tures:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.516, 0.696, 0.906, 0.736]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 133, "edu_l1_label": "IOS"}, {"txt": "$$\\boldsymbol {W}_{i}=\\text {Softmax}\\left(\\frac {\\overline {\\boldsymbol {Q}}\\boldsymbol {V}_{i}^{\\top }}{\\sqrt {d^{\\prime }}}\\right)\\in \\mathbb {R}^{w}\\tag{5}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.578, 0.761, 0.906, 0.8]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 14, "global_sentence_id": 134, "edu_l1_label": "IOS"}, {"txt": "$$\\boldsymbol {V}_{i}^{\\prime }=\\sum _{k=0}^{w}\\left(\\boldsymbol {W}_{i}\\odot \\boldsymbol {V}_{i}\\right)^{(k)}\\in \\mathbb {R}^{d^{\\prime }}\\tag{6}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5400000, "bbox": [[0.583, 0.804, 0.906, 0.843]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 15, "global_sentence_id": 135, "edu_l1_label": "IOS"}, {"txt": "$$\\boldsymbol {V}^{\\prime }=\\left[\\boldsymbol {V}_{1}^{\\prime },\\boldsymbol {V}_{2}^{\\prime },\\cdots ,\\boldsymbol {V}_{M^{\\prime }t}^{\\prime }\\right]\\in \\mathbb {R}^{M^{\\prime }txd^{\\prime }}\\tag{7}$$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.583, 0.843, 0.906, 0.862]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 16, "global_sentence_id": 136, "edu_l1_label": "IOS"}, {"txt": "The index $k$ iterates over the window size $w$, applying the corresponding weight from$\\boldsymbol {W}_{i}$to each token feature in", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.516, 0.873, 0.906, 0.901]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 137, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.493, 0.925, 0.506, 0.936]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 18, "global_sentence_id": 138, "edu_l1_label": "EDU_O"}, {"txt": "$\\boldsymbol {V}_{i}$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.093, 0.092, 0.484, 0.165]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 139, "edu_l1_label": "IOS"}, {"txt": "This process allows the model to adaptively adjust the importance of each token in the window, modulating the frame features based on their relevance to the query, thus improving the model's ability to capture and process tem-poral dependencies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.093, 0.092, 0.484, 0.165]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 140, "edu_l1_label": "IOS"}, {"txt": "Additionally, we observe that the multi-head mecha-nism introduces more multi-modal weight information in the compression process.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.114, 0.17, 0.48, 0.179], [0.094, 0.188, 0.48, 0.194], [0.095, 0.201, 0.266, 0.208]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 141, "edu_l1_label": "IOS"}, {"txt": "The pseudo-code for our com-plete compression mechanism is provided in the supple-mentary material.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.266, 0.201, 0.48, 0.208], [0.094, 0.218, 0.481, 0.223], [0.095, 0.232, 0.212, 0.239]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 142, "edu_l1_label": "IOS"}, {"txt": "4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.093, 0.258, 0.111, 0.268]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 3, "global_sentence_id": 143, "edu_l1_label": "BOS"}, {"txt": "Experiments", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.111, 0.258, 0.221, 0.269]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 3, "global_sentence_id": 144, "edu_l1_label": "IOS"}, {"txt": "4.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.094, 0.282, 0.122, 0.292]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 4, "global_sentence_id": 145, "edu_l1_label": "IOS"}, {"txt": "Dataset and Benchmarks", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.122, 0.282, 0.313, 0.293]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 4, "global_sentence_id": 146, "edu_l1_label": "IOS"}, {"txt": "Video-ChatGPT [32] is a commonly used instruction-tuning dataset for general video understanding tasks, con-taining 13,303 videos and 100,010 video-text pairs, an-notated using a combination of human-assisted and semi-automatic methods.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.095, 0.305, 0.482, 0.312], [0.095, 0.322, 0.48, 0.329], [0.095, 0.337, 0.483, 0.342], [0.095, 0.352, 0.483, 0.357], [0.094, 0.368, 0.227, 0.374]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 147, "edu_l1_label": "IOS"}, {"txt": "The average duration of the videos is 117s (short videos).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.227, 0.368, 0.48, 0.374], [0.095, 0.381, 0.248, 0.39]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 148, "edu_l1_label": "IOS"}, {"txt": "In our experiments, we randomly selected 10,000 pairs as a subset Video-ChatGPT-10K to train the alignment layer of VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.248, 0.381, 0.48, 0.39], [0.095, 0.398, 0.48, 0.404], [0.095, 0.413, 0.327, 0.419]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 149, "edu_l1_label": "IOS"}, {"txt": "Ablation studies on the data show that LVC is not a data-driven method,which sig-nificantly distinguishes it from the pre-training paradigm.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.327, 0.413, 0.48, 0.42], [0.095, 0.426, 0.481, 0.436], [0.095, 0.443, 0.47, 0.449]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 150, "edu_l1_label": "IOS"}, {"txt": "MLVU [54] is a multi-task long video understanding benchmark.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.092, 0.456, 0.484, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 151, "edu_l1_label": "IOS"}, {"txt": "We evaluate LVC on the multiple-choice ques-tions $\\left(MLVU_{M}\\right)$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.092, 0.456, 0.484, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 152, "edu_l1_label": "IOS"}, {"txt": "The average length of videos is about 15minutes, ranging from 3 minutes to 2 hours.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.092, 0.456, 0.484, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 153, "edu_l1_label": "IOS"}, {"txt": "LVC achieved comprehensive improvements across seven sub-tasks,in-cluding action order, action count, topic reasoning,anomaly recognition, plot QA, ego reasoning, and needle QA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.092, 0.456, 0.484, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 154, "edu_l1_label": "IOS"}, {"txt": "Video-MME [12]contains 900 videos and 2700 high-quality annotations of multiple-choice questions,includ-ing short (0-2 minutes), medium (4-15 minutes),and long videos (30-60 minutes).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.114, 0.561, 0.481, 0.57], [0.094, 0.58, 0.483, 0.583], [0.095, 0.593, 0.48, 0.602], [0.095, 0.609, 0.253, 0.616]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 155, "edu_l1_label": "IOS"}, {"txt": "We evaluate the VLMs before and after enhancement with LVC without subtitles, as the sam-pled video frames contain more subtitles compared to the input frames of the VLMs, ensuring a fair comparison.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.253, 0.609, 0.48, 0.616], [0.095, 0.624, 0.481, 0.631], [0.095, 0.64, 0.48, 0.646], [0.095, 0.653, 0.451, 0.662]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 156, "edu_l1_label": "IOS"}, {"txt": "4.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.094, 0.676, 0.124, 0.686]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 157, "edu_l1_label": "IOS"}, {"txt": "Experimental Setup", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.124, 0.676, 0.279, 0.689]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 158, "edu_l1_label": "IOS"}, {"txt": "Table 1. Training parameters and total parameters of InternVL2-8/26/40B-LVC and Phi-3.5-Vision-LVC, along with correspond-ing training time and resources.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.092, 0.709, 0.483, 0.748]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 9, "global_sentence_id": 159, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.1, 0.763, 0.496, 0.84]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 10, "global_sentence_id": 160, "edu_l1_label": "EDU_O"}, {"txt": "We conduct all our experiments on the Video-ChatGPT-10K dataset, which is zero-shot with respect to the two benchmarks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.114, 0.858, 0.48, 0.866], [0.096, 0.874, 0.48, 0.883], [0.095, 0.889, 0.181, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 161, "edu_l1_label": "IOS"}, {"txt": "Considering that the average duration of the", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.181, 0.889, 0.48, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 162, "edu_l1_label": "IOS"}, {"txt": "Video-ChatGPT-10K subset is less than two minutes, we fix the number of sampled frames per video to 64 (approxi-mately 0.5 fps).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.518, 0.094, 0.904, 0.104], [0.518, 0.11, 0.904, 0.117], [0.518, 0.127, 0.623, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 163, "edu_l1_label": "IOS"}, {"txt": "After compression, the number of pseudo-image frames is set to 2, 4, 8, and 16, allowing us to com-prehensively evaluate the performance of different models under varying input frame numbers.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.623, 0.127, 0.904, 0.131], [0.517, 0.14, 0.9, 0.148], [0.518, 0.157, 0.904, 0.164], [0.517, 0.171, 0.752, 0.178]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 164, "edu_l1_label": "IOS"}, {"txt": "We conduct all our experiments by training only the alignment layer of the model, with the specific training parameters and total parameter count detailed in Table 1.From both the data and parameter perspectives, LVC is a lightweight approach.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6800000, "bbox": [[0.538, 0.186, 0.904, 0.195], [0.517, 0.203, 0.904, 0.211], [0.518, 0.219, 0.904, 0.225], [0.518, 0.232, 0.904, 0.241], [0.518, 0.247, 0.664, 0.255]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 165, "edu_l1_label": "IOS"}, {"txt": "Additionally, since it does not fine-tune the LLM, LVC remains a plug-and-play method,en-suring compatibility with various VLM architectures.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6800000, "bbox": [[0.664, 0.247, 0.905, 0.253], [0.517, 0.263, 0.904, 0.27], [0.518, 0.278, 0.865, 0.285]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 166, "edu_l1_label": "IOS"}, {"txt": "4.3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.518, 0.304, 0.546, 0.313]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 14, "global_sentence_id": 167, "edu_l1_label": "IOS"}, {"txt": "Results on InternVL2 Series", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.546, 0.304, 0.766, 0.314]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 14, "global_sentence_id": 168, "edu_l1_label": "IOS"}, {"txt": "Tables 2 and 3 provide a detailed report on the experimen-tal results of enhancing InternVL2 [8] across three model scales using the LVC method.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.518, 0.327, 0.904, 0.336], [0.518, 0.343, 0.904, 0.351], [0.518, 0.359, 0.719, 0.366]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 169, "edu_l1_label": "IOS"}, {"txt": "From these results, we can derive the following conclusions:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.719, 0.359, 0.903, 0.366], [0.517, 0.372, 0.734, 0.381]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 170, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.517, 0.388, 0.908, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 171, "edu_l1_label": "IOS"}, {"txt": "The LVC method enhances the long-video understand-ing capability of the InternVL2 model series at a min-imal cost by incorporating additional visual informa-tion.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.517, 0.388, 0.908, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 172, "edu_l1_label": "IOS"}, {"txt": "On the MLVU benchmark, the three model scales achieve relative improvements of $9.6\\%$,     ,$6.1\\%$and $14.6\\%$, respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.517, 0.388, 0.908, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 173, "edu_l1_label": "IOS"}, {"txt": "On the Video-MME bench-mark, the relative improvements are $2.6\\%$,$5.9\\%$,and $7.7\\%$,respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.517, 0.388, 0.908, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 174, "edu_l1_label": "IOS"}, {"txt": "Notably, the InternVL2-8B-LVC outperforms the InternVL2-40B/76B models, while the InternVL2-40B-LVC surpasses GPT-4o on the MLVU benchmark.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.517, 0.388, 0.908, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 175, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.517, 0.554, 0.534, 0.564]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 176, "edu_l1_label": "IOS"}, {"txt": "The InternVL2-40B-LVC achieved consistent improve-ments across both benchmarks and their respective sub-tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.534, 0.554, 0.905, 0.561], [0.538, 0.573, 0.905, 0.576], [0.538, 0.586, 0.578, 0.593]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 177, "edu_l1_label": "IOS"}, {"txt": "This can be attributed to its larger scale and more high-quality pre-training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.578, 0.586, 0.9, 0.593], [0.538, 0.603, 0.773, 0.609]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 178, "edu_l1_label": "IOS"}, {"txt": "LVC effectively ad-dresses the information sparsity of VLMs in temporal reasoning by incorporating denser visual inputs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.773, 0.603, 0.905, 0.607], [0.538, 0.614, 0.904, 0.624], [0.538, 0.631, 0.851, 0.638]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 179, "edu_l1_label": "IOS"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.517, 0.645, 0.534, 0.654]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 180, "edu_l1_label": "IOS"}, {"txt": "The superior performance of LVC on MLVU compared to Video-MME can likely be attributed to the limited length of training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.534, 0.645, 0.904, 0.654], [0.538, 0.662, 0.904, 0.669], [0.538, 0.676, 0.693, 0.684]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 181, "edu_l1_label": "IOS"}, {"txt": "The number of sampled frames during training is insufficient to represent the hour-long videos in Video-MME.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.693, 0.676, 0.904, 0.684], [0.538, 0.69, 0.903, 0.701], [0.538, 0.708, 0.695, 0.714]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 182, "edu_l1_label": "IOS"}, {"txt": "In contrast, MLVU videos ex-hibit a smaller variance in duration, predominantly rang-ing from 5 to 15 minutes, making them more effectively represented by 64 sampled frames.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.695, 0.708, 0.905, 0.711], [0.538, 0.721, 0.904, 0.73], [0.538, 0.737, 0.901, 0.747], [0.538, 0.754, 0.766, 0.76]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 183, "edu_l1_label": "IOS"}, {"txt": "To the best of our knowledge, LVC is the first approach to enhance VLMs for long video understanding, introduc-ing a novel research direction distinct from Video-LLMs.Its core contribution lies in leveraging the high-quality pre-training of VLMs to improve temporal reasoning without requiring extensive video-specific pre-training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7400000, "bbox": [[0.537, 0.767, 0.904, 0.775], [0.518, 0.784, 0.904, 0.788], [0.518, 0.798, 0.905, 0.806], [0.518, 0.812, 0.904, 0.818], [0.517, 0.83, 0.904, 0.836], [0.517, 0.845, 0.824, 0.851]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 19, "global_sentence_id": 184, "edu_l1_label": "IOS"}, {"txt": "LVC preserves the pseudo-image frame format, includ-ing visual feature dimensions and special tokens.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.537, 0.857, 0.904, 0.867], [0.517, 0.874, 0.859, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 185, "edu_l1_label": "IOS"}, {"txt": "Since the windowing strategy along the token dimension allows", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.859, 0.874, 0.904, 0.883], [0.517, 0.89, 0.904, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 186, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.494, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 21, "global_sentence_id": 187, "edu_l1_label": "EDU_O"}, {"txt": "Table 2. Performance of InternVL2-8/26/40B-LVC on MLVU. TR, AR, NQA, ER, PQA, AC, AO represent the sub-tasks:topic reasoning,anomaly recognition, needle QA,ego reasoning, plot QA, action count, and action order. fps represents the number of frames sampled per second.$\\text {'}\\rightarrow 16\\text {'}$ indicates that the model maintains the same visual feature length before and after LVC enhancement for a fairer comparison.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.092, 0.092, 0.906, 0.13]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 1, "global_sentence_id": 188, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.14, 0.148, 0.859, 0.443]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 189, "edu_l1_label": "EDU_O"}, {"txt": "Table 3. Performance of InternVL2-8/26/40B-LVC on Video-MME.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.092, 0.468, 0.484, 0.493]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 3, "global_sentence_id": 190, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8000000, "bbox": [[0.096, 0.508, 0.496, 0.766]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 4, "global_sentence_id": 191, "edu_l1_label": "EDU_O"}, {"txt": "similar tokens to be aggregated through the query-attention mechanism, the compressed video features remain closely aligned with real image frames.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8100000, "bbox": [[0.094, 0.83, 0.48, 0.836], [0.095, 0.846, 0.48, 0.854], [0.095, 0.86, 0.3, 0.867]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 192, "edu_l1_label": "IOS"}, {"txt": "As a result, VLMs can learn the visual representation of pseudo-image frames solely through the alignment layer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8100000, "bbox": [[0.3, 0.86, 0.48, 0.867], [0.095, 0.874, 0.48, 0.884], [0.094, 0.889, 0.28, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 193, "edu_l1_label": "IOS"}, {"txt": "4.4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8200000, "bbox": [[0.516, 0.472, 0.547, 0.482]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 194, "edu_l1_label": "IOS"}, {"txt": "Analyses on Input Frames", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8200000, "bbox": [[0.547, 0.472, 0.749, 0.481]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 195, "edu_l1_label": "IOS"}, {"txt": "Figure 3 demonstrates the performance improvements of the LVC method in enhancing InternVL2-8B/26B/40B at compression ratios of 32/16/8/4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8300000, "bbox": [[0.525, 0.495, 0.904, 0.504], [0.517, 0.51, 0.904, 0.52], [0.517, 0.527, 0.73, 0.533]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 196, "edu_l1_label": "IOS"}, {"txt": "Based on these results,we can draw the following conclusions:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8300000, "bbox": [[0.73, 0.527, 0.904, 0.534], [0.517, 0.542, 0.753, 0.549]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 197, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.517, 0.555, 0.906, 0.645]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 198, "edu_l1_label": "IOS"}, {"txt": "The LVC method provides consistent perforance im-provements across various compression ratios for differ-ent scales of InternVL2, with more significant improve-ments observed on MLVU.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.517, 0.555, 0.906, 0.645]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 199, "edu_l1_label": "IOS"}, {"txt": "Specifically, InternVL2-40B achieved relative performance improvements of $27.9\\%$and $24.7\\%$ at fram$\\mathrm {m}=2/$,respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.517, 0.555, 0.906, 0.645]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 200, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.516, 0.646, 0.535, 0.655]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 201, "edu_l1_label": "IOS"}, {"txt": "The performance improvement of LVC becomes more significant as the compression ratio increases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.535, 0.646, 0.904, 0.655], [0.538, 0.664, 0.851, 0.67]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 202, "edu_l1_label": "IOS"}, {"txt": "This is because with fewer input frames, the amount of infor-mation available to the VLMs decreases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.851, 0.664, 0.904, 0.67], [0.538, 0.677, 0.903, 0.685], [0.538, 0.694, 0.814, 0.701]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 203, "edu_l1_label": "IOS"}, {"txt": "On the other hand, it also highlights LVC's ability to retain more in-formation even under high compression ratios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.814, 0.694, 0.904, 0.701], [0.538, 0.707, 0.905, 0.713], [0.538, 0.722, 0.842, 0.73]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 204, "edu_l1_label": "IOS"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.515, 0.736, 0.906, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 205, "edu_l1_label": "IOS"}, {"txt": "The performance growth of models enhanced with LVC becomes more gradual as the input frame count in-creases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.515, 0.736, 0.906, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 206, "edu_l1_label": "IOS"}, {"txt": "For example, at fram$\\mathrm {e}=4/8/$16,the perfor-mance differences across InternVL2 models are rela-tively small.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.515, 0.736, 0.906, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 207, "edu_l1_label": "IOS"}, {"txt": "This demonstrates the potential of LVC,as it is capable of providing rich and effective information even with a smallernumber of input frames.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.515, 0.736, 0.906, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 208, "edu_l1_label": "IOS"}, {"txt": "In summary, LVC is a highly promising method for en-hancing VLMs under limited computational resources.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.537, 0.843, 0.904, 0.851], [0.517, 0.859, 0.881, 0.867]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 209, "edu_l1_label": "IOS"}, {"txt": "By efficiently representing pseudo-image frames, it provides the visual information required for long video understand-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.881, 0.859, 0.904, 0.869], [0.517, 0.876, 0.904, 0.882], [0.517, 0.889, 0.904, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 210, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.493, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 12, "global_sentence_id": 211, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.093, 0.09, 0.485, 0.27]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 1, "global_sentence_id": 212, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.099, 0.27, 0.482, 0.448]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 213, "edu_l1_label": "EDU_O"}, {"txt": "Figure 3. Performance comparison of InternVL2-8/26/40B before and after LVC enhancement on MLVU and Video-MME, with in-put frames set to 2/4/8/16. For LVC models, input frames refer to the numbers of pseudo-image frame after compression. Sampled frames are uniformly set to 64.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.092, 0.461, 0.484, 0.528]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 3, "global_sentence_id": 214, "edu_l1_label": "EDU_O"}, {"txt": "ing in VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.094, 0.557, 0.185, 0.566]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 215, "edu_l1_label": "IOS"}, {"txt": "4.5.Ablation Studies", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.094, 0.58, 0.253, 0.59]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 216, "edu_l1_label": "IOS"}, {"txt": "Table 4. Data ablation experiment on InternVL2-8/26/40B with input frames set to 16. Origin represents the model's original per-formance, while Trained and LVC-trained indicate the results of training the model with the same data, without and with LVC, re-spectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.093, 0.611, 0.484, 0.677]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 217, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.097, 0.691, 0.497, 0.806]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 7, "global_sentence_id": 218, "edu_l1_label": "EDU_O"}, {"txt": "Data Ablation:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.095, 0.828, 0.2, 0.836]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 219, "edu_l1_label": "IOS"}, {"txt": " Table 4 presents the results of the data ab-lation study, where we compare the performance of the In-ternVL2 series models trained on Video-ChatGPT-10K with and without LVC.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.2, 0.828, 0.483, 0.834], [0.095, 0.843, 0.481, 0.849], [0.095, 0.86, 0.48, 0.867], [0.095, 0.876, 0.213, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 220, "edu_l1_label": "IOS"}, {"txt": "Based on the comparison between Origin and Trained, we can conclude that the LVC method does", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.213, 0.876, 0.48, 0.883], [0.095, 0.891, 0.48, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 221, "edu_l1_label": "IOS"}, {"txt": "Table 5. Component ablation results of InternVL2-8B. w/o multi-heads indicates the removal of the multi-head mechanism in LVC,while w/o query-attn removes the query influence, making it equivalent to average pooling.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.516, 0.092, 0.906, 0.143]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 9, "global_sentence_id": 222, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9600000, "bbox": [[0.519, 0.158, 0.922, 0.246]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 10, "global_sentence_id": 223, "edu_l1_label": "EDU_O"}, {"txt": "not achieve performance improvement through training on video data, but rather through its enhanced representation mechanism.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9700000, "bbox": [[0.517, 0.278, 0.904, 0.285], [0.517, 0.294, 0.904, 0.301], [0.517, 0.309, 0.598, 0.316]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 224, "edu_l1_label": "IOS"}, {"txt": "The primary role of Video-ChatGPT-10K is to help the enhanced model understand pseudo-image frame represen-tations, as the model itself was not aligned with compressed visual features during its pre-training process.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.538, 0.321, 0.904, 0.331], [0.517, 0.34, 0.904, 0.345], [0.518, 0.354, 0.903, 0.361], [0.518, 0.369, 0.817, 0.376]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 225, "edu_l1_label": "IOS"}, {"txt": "Data Scale Ablation:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.517, 0.384, 0.676, 0.393]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 226, "edu_l1_label": "IOS"}, {"txt": " We add the performance of InternVL2-8B at different steps on the Video-ChatGPT10k dataset, as shown in the Figure 4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.676, 0.384, 0.904, 0.393], [0.518, 0.4, 0.903, 0.408], [0.518, 0.414, 0.751, 0.423]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 227, "edu_l1_label": "IOS"}, {"txt": "We can conclude that LVC achieves quite good performance with just a small-scale dataset (around 1000 steps), indicating fast adaptation speed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.751, 0.414, 0.904, 0.423], [0.517, 0.429, 0.904, 0.438], [0.518, 0.447, 0.904, 0.453], [0.518, 0.462, 0.56, 0.468]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 228, "edu_l1_label": "IOS"}, {"txt": "After 1000 steps, the long video understanding abil-ity grows slowly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.56, 0.462, 0.904, 0.468], [0.518, 0.475, 0.631, 0.483]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 229, "edu_l1_label": "IOS"}, {"txt": "We use 10k data as a balanced choice between perfor-mance and training cost, as well as for easy comparison ex-periments across different models and compression ratios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10000000, "bbox": [[0.538, 0.49, 0.904, 0.499], [0.517, 0.508, 0.905, 0.512], [0.518, 0.522, 0.897, 0.529]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 230, "edu_l1_label": "IOS"}, {"txt": "Component Ablation:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.518, 0.537, 0.672, 0.547]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 231, "edu_l1_label": "IOS"}, {"txt": " Table 5 presents the component ablation study, where we separately remove the Query-Attention Video Compression mechanism itself and the in-ternal multi-head mechanism.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.672, 0.537, 0.904, 0.547], [0.517, 0.554, 0.904, 0.561], [0.517, 0.568, 0.905, 0.574], [0.517, 0.583, 0.717, 0.591]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 232, "edu_l1_label": "IOS"}, {"txt": "The former is equivalent to average pooling.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.717, 0.583, 0.904, 0.592], [0.517, 0.6, 0.628, 0.606]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 233, "edu_l1_label": "IOS"}, {"txt": "Based on the results, we can draw the fol-lowing conclusions:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.628, 0.6, 0.904, 0.605], [0.517, 0.613, 0.647, 0.621]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 234, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.518, 0.629, 0.533, 0.637]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 235, "edu_l1_label": "IOS"}, {"txt": "The query provides multi-modal information and plays a guiding role in the video compression process of LVVC.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.533, 0.629, 0.904, 0.638], [0.538, 0.646, 0.896, 0.652]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 236, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10300000, "bbox": [[0.517, 0.658, 0.534, 0.667]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 237, "edu_l1_label": "IOS"}, {"txt": "The introduction of multi-heads increases the number of weights, from one per window to multiple per window,corresponding to the number of attention heads.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10300000, "bbox": [[0.534, 0.658, 0.904, 0.668], [0.538, 0.677, 0.904, 0.684], [0.538, 0.691, 0.868, 0.698]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 238, "edu_l1_label": "IOS"}, {"txt": "The finer-grained weighting has a positive correlation with performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10300000, "bbox": [[0.868, 0.691, 0.904, 0.698], [0.538, 0.705, 0.904, 0.713], [0.538, 0.722, 0.627, 0.729]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 239, "edu_l1_label": "IOS"}, {"txt": "3.Even with the removal of certain components, models enhanced with LVC still achieve performance improve-ments over the original models in long video understand-ing tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.516, 0.734, 0.904, 0.742], [0.538, 0.751, 0.904, 0.757], [0.538, 0.766, 0.905, 0.771], [0.538, 0.78, 0.606, 0.788]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 240, "edu_l1_label": "IOS"}, {"txt": "This is because the core advantage of LVC lies in compensating for the information loss in VLMs,which rely on sparsely sampled frames as multi-image inputs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.606, 0.78, 0.903, 0.789], [0.538, 0.795, 0.906, 0.804], [0.539, 0.811, 0.904, 0.819], [0.538, 0.826, 0.583, 0.834]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 241, "edu_l1_label": "IOS"}, {"txt": "4.6.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10500000, "bbox": [[0.518, 0.851, 0.546, 0.86]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 19, "global_sentence_id": 242, "edu_l1_label": "IOS"}, {"txt": "Results on Phi-3.5-Vision", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10500000, "bbox": [[0.546, 0.851, 0.74, 0.86]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 19, "global_sentence_id": 243, "edu_l1_label": "IOS"}, {"txt": "We select a smaller-scale model, Phi-3.5-Vision (4B) [1],and evaluate its performance before and after enhancement", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.518, 0.873, 0.905, 0.883], [0.516, 0.89, 0.904, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 244, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.493, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 21, "global_sentence_id": 245, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.095, 0.09, 0.486, 0.27]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 1, "global_sentence_id": 246, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.097, 0.27, 0.484, 0.45]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 247, "edu_l1_label": "EDU_O"}, {"txt": "Figure 4. Performance of InternVL2-8 on MLVU with LVC en-hancement at different training steps.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10800000, "bbox": [[0.092, 0.461, 0.481, 0.486]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 3, "global_sentence_id": 248, "edu_l1_label": "EDU_O"}, {"txt": "with the LVC method.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.094, 0.52, 0.243, 0.527]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 249, "edu_l1_label": "IOS"}, {"txt": "The results are shown in Figure 5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.243, 0.52, 0.467, 0.527]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 250, "edu_l1_label": "IOS"}, {"txt": "However, due to low dataset quality or incompatibility between the data and the model, directly training the model results in some performance degradation across both bench-marks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.119, 0.532, 0.481, 0.544], [0.094, 0.548, 0.48, 0.556], [0.094, 0.565, 0.48, 0.572], [0.095, 0.58, 0.142, 0.587]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 251, "edu_l1_label": "IOS"}, {"txt": "This phenomenon is also observed in InternVL2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.142, 0.58, 0.464, 0.587]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 252, "edu_l1_label": "IOS"}, {"txt": "From the figure, it can be observed that LVC achieves consistent improvements on MLVU, with an average im-provement of 5.4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.114, 0.593, 0.481, 0.602], [0.095, 0.611, 0.481, 0.617], [0.095, 0.626, 0.22, 0.632]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 253, "edu_l1_label": "IOS"}, {"txt": "Similarly, on Video-MME,LVC also demonstrates consistent improvements after excluding data influence, with an average improvement of 2.5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.22, 0.626, 0.48, 0.633], [0.1, 0.639, 0.48, 0.648], [0.095, 0.655, 0.413, 0.662]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 254, "edu_l1_label": "IOS"}, {"txt": "This con-firms the scalability of LVC to other VLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.413, 0.655, 0.481, 0.662], [0.096, 0.669, 0.382, 0.677]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 255, "edu_l1_label": "IOS"}, {"txt": "5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.094, 0.697, 0.112, 0.707]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 256, "edu_l1_label": "BOS"}, {"txt": "Discussion and Future Work", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.112, 0.697, 0.353, 0.708]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 257, "edu_l1_label": "IOS"}, {"txt": "The core idea of our proposed LVC method is to en-able VLMs to process richer, continuous representations of video content.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.094, 0.723, 0.481, 0.732], [0.095, 0.739, 0.481, 0.747], [0.095, 0.755, 0.191, 0.761]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 258, "edu_l1_label": "IOS"}, {"txt": "By introducing the Query-Attention Video Compression mechanism and representing compressed fea-tures as pseudo-image frames, we effectively reduce both data requirements and training parameters compared to con-ventional Video-LLMs, which rely heavily on large-scale pre-training and fine-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.191, 0.755, 0.48, 0.761], [0.095, 0.768, 0.48, 0.777], [0.094, 0.785, 0.48, 0.792], [0.095, 0.798, 0.483, 0.805], [0.095, 0.815, 0.48, 0.822], [0.094, 0.831, 0.28, 0.836]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 259, "edu_l1_label": "IOS"}, {"txt": "In contrast to Video-LLMs, our experimental results provide preliminary evidence for the effectiveness of an alternative research direction-enhancing VLMs through lightweight compression.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11400000, "bbox": [[0.114, 0.843, 0.481, 0.852], [0.095, 0.861, 0.48, 0.867], [0.095, 0.876, 0.48, 0.883], [0.095, 0.889, 0.267, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 260, "edu_l1_label": "IOS"}, {"txt": "Moreover, by introducing ad-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11400000, "bbox": [[0.267, 0.889, 0.48, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 261, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.526, 0.092, 0.904, 0.275]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 10, "global_sentence_id": 262, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.52, 0.273, 0.904, 0.447]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 11, "global_sentence_id": 263, "edu_l1_label": "EDU_O"}, {"txt": "Figure 5. Performance of Phi-3.5-Vision before and after LVC enhancement, along with data ablation results(middle).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11600000, "bbox": [[0.516, 0.462, 0.908, 0.486]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 12, "global_sentence_id": 264, "edu_l1_label": "EDU_O"}, {"txt": "ditional information through denser sampling, LVC can strengthen the temporal awareness of more powerful VLMs without modifying the LLM backbone, thus offering flex-ibility and adaptability.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.517, 0.518, 0.9, 0.526], [0.517, 0.535, 0.904, 0.542], [0.518, 0.549, 0.905, 0.554], [0.518, 0.563, 0.672, 0.573]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 265, "edu_l1_label": "IOS"}, {"txt": "This zero-modification aspect for LLMs allows LVC to be easily adopted in different architec-tures and tasks that require long video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.672, 0.563, 0.904, 0.572], [0.517, 0.578, 0.904, 0.585], [0.517, 0.595, 0.877, 0.602]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 266, "edu_l1_label": "IOS"}, {"txt": "We envision several ways to extend LVC:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.877, 0.595, 0.904, 0.602], [0.517, 0.611, 0.762, 0.616]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 267, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.518, 0.623, 0.533, 0.632]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 268, "edu_l1_label": "IOS"}, {"txt": "Adaptive Sampling and Compression: Dynamically de-termine the sampling rate and compression ratio based on video length and content, optimizing the trade-off be-tween information retention and computational cost.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.533, 0.623, 0.904, 0.629], [0.538, 0.64, 0.904, 0.647], [0.538, 0.657, 0.905, 0.66], [0.538, 0.67, 0.881, 0.677]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 269, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11900000, "bbox": [[0.517, 0.683, 0.534, 0.693]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 270, "edu_l1_label": "IOS"}, {"txt": "Advanced Compression Mechanisms: Investigate more sophisticated strategies such as hierarchical compres-sion or more robust similarity metrics to further improve compression efficiency.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11900000, "bbox": [[0.534, 0.683, 0.904, 0.692], [0.538, 0.701, 0.904, 0.706], [0.538, 0.715, 0.904, 0.722], [0.538, 0.732, 0.691, 0.739]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 271, "edu_l1_label": "IOS"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.517, 0.744, 0.532, 0.753]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 272, "edu_l1_label": "IOS"}, {"txt": "Multiple Alignment Layers: Assign specific alignment layers to handle different video lengths or content types,potentially improving generalization across various sce-narios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.532, 0.744, 0.904, 0.753], [0.538, 0.759, 0.904, 0.769], [0.538, 0.777, 0.905, 0.781], [0.538, 0.792, 0.583, 0.799]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 273, "edu_l1_label": "IOS"}, {"txt": "6.Conclusion", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.517, 0.818, 0.631, 0.828]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 17, "global_sentence_id": 274, "edu_l1_label": "BOS"}, {"txt": "We propose LVC and design an effective QQuery-Attention Video Compression mechanism to compensate for the in-formation loss in VLMs caused by sparse sampling.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.518, 0.843, 0.904, 0.852], [0.518, 0.859, 0.904, 0.867], [0.518, 0.874, 0.858, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 275, "edu_l1_label": "IOS"}, {"txt": "Unlike common Video-LLMs, LVC is lightweight in terms of both", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.858, 0.874, 0.904, 0.883], [0.517, 0.891, 0.904, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 276, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.493, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 19, "global_sentence_id": 277, "edu_l1_label": "EDU_O"}, {"txt": "data and training parameters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.095, 0.095, 0.294, 0.103]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 278, "edu_l1_label": "IOS"}, {"txt": "By training only the align-ment layer, LVC can be easily extended to other VLMs.Extensive experiments demonstrate the superiority of LVC,offering a novel research direction for enhancing VLMs in long video understanding tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.294, 0.095, 0.482, 0.1], [0.095, 0.112, 0.482, 0.118], [0.095, 0.124, 0.483, 0.134], [0.095, 0.142, 0.48, 0.149], [0.095, 0.155, 0.305, 0.164]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 279, "edu_l1_label": "IOS"}, {"txt": "In the future, we will con-tinue to extend LVC to enhance its practical applicability.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.305, 0.155, 0.481, 0.163], [0.095, 0.171, 0.47, 0.178]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 280, "edu_l1_label": "IOS"}, {"txt": "References", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.096, 0.199, 0.186, 0.21]]}]}, "tags": ["title"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 281, "edu_l1_label": "EDU_O"}, {"txt": "[1] Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadal-lah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree,Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.101, 0.225, 0.483, 0.23], [0.127, 0.239, 0.483, 0.247], [0.127, 0.253, 0.443, 0.261]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 282, "edu_l1_label": "EDU_O"}, {"txt": "Phi-3technical report: A highly capable language model locally on your phone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.443, 0.253, 0.481, 0.261], [0.127, 0.268, 0.48, 0.276], [0.127, 0.282, 0.217, 0.288]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 283, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2404.14219,2024.7", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.217, 0.282, 0.467, 0.289]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 284, "edu_l1_label": "EDU_O"}, {"txt": "[2] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc,Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men-sch,Katherine Millican, Malcolm Reynolds, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.102, 0.295, 0.48, 0.304], [0.127, 0.309, 0.48, 0.318], [0.128, 0.324, 0.419, 0.331]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 285, "edu_l1_label": "EDU_O"}, {"txt": "Flamingo:a visual language model for few-shot learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.419, 0.324, 0.483, 0.331], [0.127, 0.339, 0.417, 0.345]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 286, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 35:23716-23736,2022.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.417, 0.339, 0.48, 0.345], [0.129, 0.353, 0.483, 0.36], [0.127, 0.365, 0.173, 0.372]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 287, "edu_l1_label": "EDU_O"}, {"txt": "[3] Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Es-sam Sleiman, Deyao Zhu, Jian Ding, and Mohamed El-hoseiny.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.103, 0.38, 0.483, 0.386], [0.127, 0.396, 0.482, 0.4], [0.127, 0.408, 0.179, 0.416]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 288, "edu_l1_label": "EDU_O"}, {"txt": "Minigpt4-video: Advancing multimodal llms for video understanding with interleaved visual-textual tokens.arXiv preprint arXiv:2404.03413,2024.1,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.179, 0.408, 0.483, 0.416], [0.127, 0.423, 0.483, 0.43], [0.127, 0.438, 0.404, 0.444]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 289, "edu_l1_label": "EDU_O"}, {"txt": "[4] Gedas Bertasius, Heng Wang, and Lorenzo Torresani.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.101, 0.451, 0.462, 0.459]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 290, "edu_l1_label": "EDU_O"}, {"txt": "Is space-time attention all you need for video understanding?In ICML, page 4,2021.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.462, 0.451, 0.481, 0.459], [0.127, 0.467, 0.481, 0.473], [0.127, 0.479, 0.276, 0.487]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 291, "edu_l1_label": "EDU_O"}, {"txt": "[5] Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem,and Juan Carlos Niebles.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.104, 0.494, 0.482, 0.504], [0.127, 0.509, 0.28, 0.516]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 292, "edu_l1_label": "EDU_O"}, {"txt": "Activitynet: A large-scale video benchmark for human activity understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.28, 0.509, 0.481, 0.515], [0.128, 0.521, 0.402, 0.53]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 293, "edu_l1_label": "EDU_O"}, {"txt": "In Proceed-ings of the ieee conference on computer vision and pattern recognition,pages 961-970,2015.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.402, 0.521, 0.48, 0.528], [0.128, 0.537, 0.48, 0.544], [0.127, 0.552, 0.342, 0.557]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 294, "edu_l1_label": "EDU_O"}, {"txt": "[6] Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao,Mingzhen Sun, Xinxin Zhu, and Jing Liu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.101, 0.565, 0.482, 0.574], [0.127, 0.579, 0.411, 0.587]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 295, "edu_l1_label": "EDU_O"}, {"txt": "Vast: A vision-audio-subtitle-text omni-modality foundation model and dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.411, 0.579, 0.48, 0.587], [0.127, 0.593, 0.483, 0.6], [0.127, 0.608, 0.198, 0.614]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 296, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Sys-tems,36:72842-72866,2023.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.198, 0.608, 0.48, 0.614], [0.127, 0.622, 0.313, 0.628]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 297, "edu_l1_label": "EDU_O"}, {"txt": "[7] Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13100000, "bbox": [[0.103, 0.635, 0.48, 0.644], [0.127, 0.649, 0.181, 0.658]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 298, "edu_l1_label": "EDU_O"}, {"txt": "Timemarker: A versatile video-llm for long and short video understanding with superior temporal localiza-tion ability.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13100000, "bbox": [[0.181, 0.649, 0.48, 0.658], [0.127, 0.664, 0.48, 0.671], [0.127, 0.679, 0.197, 0.686]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 299, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2411.18211,2024.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13100000, "bbox": [[0.197, 0.679, 0.446, 0.685]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 300, "edu_l1_label": "EDU_O"}, {"txt": "[8] Zhe Chen, Jiannan Wu,Wenhai Wang,Weijie Su, Guo Chen,Sen Xing,Muyan Zhong,Qinglong Zhang, Xizhou Zhu,Lewei Lu, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13200000, "bbox": [[0.106, 0.693, 0.48, 0.701], [0.127, 0.707, 0.483, 0.715], [0.127, 0.72, 0.221, 0.729]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 301, "edu_l1_label": "EDU_O"}, {"txt": "Internvl: Scaling up vision foundation mod-els and aligning for generic visual-linguistic tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13200000, "bbox": [[0.221, 0.72, 0.48, 0.728], [0.127, 0.735, 0.432, 0.742]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 302, "edu_l1_label": "EDU_O"}, {"txt": "In Pro-ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185-24198,2024.1,2,3,5", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13200000, "bbox": [[0.432, 0.735, 0.48, 0.742], [0.127, 0.75, 0.481, 0.756], [0.127, 0.765, 0.483, 0.771], [0.127, 0.776, 0.134, 0.784]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 303, "edu_l1_label": "EDU_O"}, {"txt": "[9] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang-hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang,Yong-hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P.Xing.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13300000, "bbox": [[0.101, 0.791, 0.481, 0.8], [0.127, 0.806, 0.481, 0.813], [0.127, 0.819, 0.483, 0.827], [0.127, 0.833, 0.165, 0.84]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 304, "edu_l1_label": "EDU_O"}, {"txt": "Vicuna: An open-source chatbot impressing gpt-4with 90%* chatgpt quality,2023.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13300000, "bbox": [[0.165, 0.833, 0.481, 0.84], [0.127, 0.848, 0.334, 0.855]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 305, "edu_l1_label": "EDU_O"}, {"txt": "[10] Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh,Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Ba-tra.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.094, 0.862, 0.483, 0.871], [0.127, 0.875, 0.48, 0.883], [0.127, 0.891, 0.15, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 306, "edu_l1_label": "EDU_O"}, {"txt": "Visual dialog.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.15, 0.891, 0.242, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 307, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE conference", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.242, 0.891, 0.481, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 308, "edu_l1_label": "EDU_O"}, {"txt": "on computer vision and pattern recognition, pages 326-335,2017.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.556, 0.098, 0.906, 0.104], [0.55, 0.109, 0.595, 0.117]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 309, "edu_l1_label": "EDU_O"}, {"txt": "[11] Christoph Feichtenhofer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13500000, "bbox": [[0.516, 0.122, 0.699, 0.131]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 310, "edu_l1_label": "EDU_O"}, {"txt": "X3d: Expanding architectures for efficient video recognition.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13500000, "bbox": [[0.699, 0.122, 0.904, 0.131], [0.549, 0.138, 0.711, 0.145]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 311, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,pages 203-213,2020.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13500000, "bbox": [[0.711, 0.138, 0.904, 0.145], [0.549, 0.153, 0.904, 0.159], [0.55, 0.167, 0.691, 0.172]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 312, "edu_l1_label": "EDU_O"}, {"txt": "[12] Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang,Chenyu Zhou,Yunhang Shen, Mengdan Zhang, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.516, 0.179, 0.904, 0.187], [0.551, 0.194, 0.9, 0.203], [0.55, 0.207, 0.733, 0.22]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 313, "edu_l1_label": "EDU_O"}, {"txt": "Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.733, 0.207, 0.904, 0.215], [0.549, 0.223, 0.904, 0.229], [0.55, 0.237, 0.64, 0.243]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 314, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2405.21075,2024.5", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.64, 0.237, 0.889, 0.243]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 315, "edu_l1_label": "EDU_O"}, {"txt": "[13] Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia.Tall:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.516, 0.248, 0.905, 0.256], [0.551, 0.263, 0.578, 0.275]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 316, "edu_l1_label": "EDU_O"}, {"txt": " Temporal activity localization via language query.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.578, 0.263, 0.887, 0.271]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 317, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE international conference on com-puter vision,pages 5267-5275,2017.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.887, 0.263, 0.904, 0.271], [0.55, 0.276, 0.904, 0.284], [0.55, 0.293, 0.782, 0.299]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 318, "edu_l1_label": "EDU_O"}, {"txt": "[14] Bo He,Hengduo Li,Young Kyun Jang, Menglin Jia,Xue-fei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13800000, "bbox": [[0.516, 0.305, 0.904, 0.312], [0.55, 0.32, 0.904, 0.327], [0.55, 0.333, 0.581, 0.34]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 319, "edu_l1_label": "EDU_O"}, {"txt": "Ma-lmm: Memory-augmented large multimodal model for long-term video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13800000, "bbox": [[0.581, 0.333, 0.904, 0.34], [0.551, 0.347, 0.765, 0.354]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 320, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13504-13514,2024.1,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13800000, "bbox": [[0.765, 0.347, 0.904, 0.354], [0.55, 0.36, 0.904, 0.368], [0.55, 0.373, 0.813, 0.382]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 321, "edu_l1_label": "EDU_O"}, {"txt": "[15] Bin Huang,Xin Wang,Hong Chen,Zihan Song, and Wenwu Zhu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13900000, "bbox": [[0.517, 0.388, 0.904, 0.396], [0.55, 0.402, 0.579, 0.41]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 322, "edu_l1_label": "EDU_O"}, {"txt": "Vtimellm: Empower llm to grasp video moments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13900000, "bbox": [[0.579, 0.402, 0.887, 0.41]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 323, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vi-sion and Pattern Recognition, pages 14271-14280,2024.1,2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13900000, "bbox": [[0.887, 0.402, 0.904, 0.41], [0.55, 0.416, 0.903, 0.424], [0.55, 0.432, 0.906, 0.439], [0.55, 0.444, 0.557, 0.451]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 324, "edu_l1_label": "EDU_O"}, {"txt": "[16] Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao,and Li Yuan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.517, 0.458, 0.905, 0.467], [0.549, 0.473, 0.628, 0.48]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 19, "global_sentence_id": 325, "edu_l1_label": "EDU_O"}, {"txt": "Chat-univi: Unified visual representation em-powers large language models with image and video un-derstanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.628, 0.473, 0.904, 0.48], [0.55, 0.488, 0.904, 0.493], [0.55, 0.5, 0.625, 0.507]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 19, "global_sentence_id": 326, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13700-13710,2024.1,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.625, 0.5, 0.904, 0.508], [0.55, 0.516, 0.904, 0.52], [0.551, 0.527, 0.668, 0.535]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 19, "global_sentence_id": 327, "edu_l1_label": "EDU_O"}, {"txt": "[17] Bo Li,Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li,Hao Zhang,Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi-wei Liu, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14100000, "bbox": [[0.516, 0.54, 0.905, 0.55], [0.556, 0.555, 0.905, 0.561], [0.55, 0.571, 0.641, 0.577]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 20, "global_sentence_id": 328, "edu_l1_label": "EDU_O"}, {"txt": "Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,2024.1,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14100000, "bbox": [[0.641, 0.571, 0.906, 0.581], [0.55, 0.585, 0.813, 0.591]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 20, "global_sentence_id": 329, "edu_l1_label": "EDU_O"}, {"txt": "[18] Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14200000, "bbox": [[0.516, 0.597, 0.904, 0.605], [0.55, 0.611, 0.9, 0.619], [0.55, 0.625, 0.617, 0.633]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 330, "edu_l1_label": "EDU_O"}, {"txt": "Aria: An open multimodal native mixture-of-experts model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14200000, "bbox": [[0.617, 0.625, 0.904, 0.633], [0.55, 0.641, 0.638, 0.647]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 331, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2410.05993,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14200000, "bbox": [[0.638, 0.641, 0.889, 0.646]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 332, "edu_l1_label": "EDU_O"}, {"txt": "[19] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.Blip-2:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.516, 0.653, 0.906, 0.661], [0.551, 0.666, 0.595, 0.675]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 22, "global_sentence_id": 333, "edu_l1_label": "EDU_O"}, {"txt": " Bootstrapping language-image pre-training with frozen image encoders and large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.595, 0.666, 0.903, 0.675], [0.551, 0.681, 0.862, 0.689]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 22, "global_sentence_id": 334, "edu_l1_label": "EDU_O"}, {"txt": "In In-ternational conference on machine learning, pages 19730-19742.PMLR,2023.1,2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.862, 0.681, 0.905, 0.686], [0.55, 0.696, 0.904, 0.702], [0.551, 0.708, 0.701, 0.717]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 22, "global_sentence_id": 335, "edu_l1_label": "EDU_O"}, {"txt": "[20] Kunchang Li, Yali Wang,Yinan He,Yizhuo Li,Yi Wang,Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo,et al.Mvbench:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.52, 0.723, 0.905, 0.732], [0.551, 0.736, 0.906, 0.744], [0.551, 0.75, 0.61, 0.758]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 336, "edu_l1_label": "EDU_O"}, {"txt": " A comprehensive multi-modal video understand-ing benchmark.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.61, 0.75, 0.904, 0.758], [0.551, 0.766, 0.643, 0.772]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 337, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195-22206,2024.1,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.643, 0.766, 0.904, 0.773], [0.55, 0.78, 0.904, 0.784], [0.55, 0.791, 0.669, 0.8]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 338, "edu_l1_label": "EDU_O"}, {"txt": "[21] Qingyun Li,Zhe Chen,Weiyun Wang,Wenhai Wang,Shen-glong Ye,Zhenjiang Jin, Guanzhou Chen, Yinan He, Zhang-wei Gao, Erfei Cui, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14500000, "bbox": [[0.518, 0.806, 0.903, 0.814], [0.55, 0.823, 0.905, 0.826], [0.551, 0.836, 0.696, 0.842]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 339, "edu_l1_label": "EDU_O"}, {"txt": "Omnicorpus: A unified multimodal corpus of 10 billion-level images interleaved with text.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14500000, "bbox": [[0.696, 0.836, 0.905, 0.842], [0.55, 0.85, 0.869, 0.856]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 340, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2406.08418,2024.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14500000, "bbox": [[0.869, 0.85, 0.904, 0.856], [0.55, 0.864, 0.761, 0.869]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 341, "edu_l1_label": "EDU_O"}, {"txt": "[22] Yanwei Li, Chengyao Wang, and Jiaya Jia.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.518, 0.876, 0.827, 0.884]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 342, "edu_l1_label": "EDU_O"}, {"txt": "Llama-vid:An image is worth 2 tokens in large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.827, 0.876, 0.903, 0.883], [0.551, 0.89, 0.884, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 343, "edu_l1_label": "EDU_O"}, {"txt": "In", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.884, 0.89, 0.904, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 344, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14700000, "bbox": [[0.493, 0.925, 0.506, 0.938]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 26, "global_sentence_id": 345, "edu_l1_label": "EDU_O"}, {"txt": "European Conference on Computer Vision, pages 323-340.Springer, 2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.127, 0.095, 0.483, 0.104], [0.127, 0.11, 0.231, 0.117]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 346, "edu_l1_label": "EDU_O"}, {"txt": "[23] Bin Lin, Yang Ye,Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.095, 0.123, 0.48, 0.133], [0.127, 0.138, 0.235, 0.145]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 347, "edu_l1_label": "EDU_O"}, {"txt": "Video-llava: Learning united visual rep-resentation by alignment before projection.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.235, 0.138, 0.48, 0.145], [0.127, 0.153, 0.388, 0.159]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 348, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2311.10122,2023.1,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.388, 0.153, 0.48, 0.159], [0.127, 0.167, 0.317, 0.172]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 349, "edu_l1_label": "EDU_O"}, {"txt": "[24] Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin,Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.094, 0.179, 0.483, 0.188], [0.127, 0.193, 0.48, 0.201], [0.127, 0.207, 0.362, 0.215]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 350, "edu_l1_label": "EDU_O"}, {"txt": "Mm-vid: Advanc-ing video understanding with gpt-4v (ision).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.362, 0.207, 0.48, 0.215], [0.128, 0.222, 0.393, 0.229]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 351, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2310.19773,2023.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.393, 0.222, 0.483, 0.229], [0.127, 0.237, 0.287, 0.241]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 352, "edu_l1_label": "EDU_O"}, {"txt": "[25] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.Visual instruction tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15100000, "bbox": [[0.095, 0.249, 0.481, 0.257], [0.127, 0.263, 0.283, 0.271]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 353, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems,36:34892-34916,2023.1,2,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15100000, "bbox": [[0.283, 0.263, 0.48, 0.271], [0.127, 0.279, 0.427, 0.285]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 354, "edu_l1_label": "EDU_O"}, {"txt": "[26] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee.Improved baselines with visual instruction tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15200000, "bbox": [[0.097, 0.29, 0.483, 0.298], [0.127, 0.304, 0.431, 0.312]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 355, "edu_l1_label": "EDU_O"}, {"txt": "In Pro-ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306,2024.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15200000, "bbox": [[0.431, 0.304, 0.48, 0.312], [0.127, 0.32, 0.481, 0.326], [0.127, 0.334, 0.448, 0.34]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 356, "edu_l1_label": "EDU_O"}, {"txt": "[27] Jing Liu, Sihan Chen, Xingjian He, Longteng Guo, Xinxin Zhu,Weining Wang, and Jinhui Tang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.093, 0.346, 0.48, 0.355], [0.127, 0.361, 0.355, 0.369]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 357, "edu_l1_label": "EDU_O"}, {"txt": "Valor: Vision-audio-language omni-perception pretraining model and dataset.IEEE Transactions on Pattern Analysis and Machine Intel-ligence,2024.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.355, 0.361, 0.48, 0.368], [0.127, 0.374, 0.483, 0.382], [0.127, 0.388, 0.482, 0.395], [0.127, 0.402, 0.221, 0.41]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 358, "edu_l1_label": "EDU_O"}, {"txt": "[28] Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu,Xi-aoqi Ma,Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu.Kangaroo:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15400000, "bbox": [[0.095, 0.416, 0.48, 0.424], [0.127, 0.432, 0.483, 0.438], [0.127, 0.443, 0.193, 0.452]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 359, "edu_l1_label": "EDU_O"}, {"txt": " A powerful video-language model supporting long-context video input.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15400000, "bbox": [[0.193, 0.443, 0.48, 0.453], [0.127, 0.458, 0.279, 0.466]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 360, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2408.15542,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15400000, "bbox": [[0.279, 0.458, 0.483, 0.467], [0.127, 0.472, 0.172, 0.48]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 361, "edu_l1_label": "EDU_O"}, {"txt": "[29] Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Ji-wen Lu, and Yongming Rao.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15500000, "bbox": [[0.094, 0.485, 0.483, 0.491], [0.127, 0.501, 0.315, 0.507]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 362, "edu_l1_label": "EDU_O"}, {"txt": "Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15500000, "bbox": [[0.315, 0.501, 0.48, 0.508], [0.127, 0.516, 0.445, 0.521]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 363, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2409.12961,2024.3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15500000, "bbox": [[0.445, 0.516, 0.482, 0.521], [0.127, 0.53, 0.353, 0.535]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 364, "edu_l1_label": "EDU_O"}, {"txt": "[30] Zhijian Liu, Ligeng Zhu, Baifeng Shi,Zhuoyang Zhang,Yuming Lou, Shang Yang,Haocheng Xi,Shiyi Cao,Yux-ian Gu, Dacheng Li, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.095, 0.541, 0.483, 0.551], [0.128, 0.556, 0.483, 0.561], [0.128, 0.57, 0.29, 0.578]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 365, "edu_l1_label": "EDU_O"}, {"txt": "Nvila: Efficient frontier visual language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.29, 0.57, 0.481, 0.578], [0.127, 0.583, 0.233, 0.591]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 366, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2412.04468,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.233, 0.583, 0.48, 0.59]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 367, "edu_l1_label": "EDU_O"}, {"txt": "[31] Ruipu Luo, Ziwang Zhao, Min Yang, Junwei Dong, Da Li,Pengcheng Lu, Tao Wang, Linmei Hu, Minghui Qiu, and Zhongyu Wei.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.095, 0.597, 0.484, 0.606], [0.127, 0.61, 0.48, 0.619], [0.127, 0.625, 0.216, 0.633]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 368, "edu_l1_label": "EDU_O"}, {"txt": "Valley: Video assistant with large language model enhanced ability.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.216, 0.625, 0.48, 0.633], [0.135, 0.641, 0.274, 0.646]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 369, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2306.07207,2023.1,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.274, 0.641, 0.481, 0.647], [0.127, 0.653, 0.187, 0.661]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 370, "edu_l1_label": "EDU_O"}, {"txt": "[32] Muhammad Maaz, Hanoona Rasheed,Salman Khan, and Fa-had Shahbaz Khan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.095, 0.666, 0.483, 0.673], [0.127, 0.681, 0.246, 0.689]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 371, "edu_l1_label": "EDU_O"}, {"txt": "Video-chatgpt: Towards detailed video understanding via large vision and language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.246, 0.681, 0.481, 0.688], [0.127, 0.696, 0.444, 0.703]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 372, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2306.05424,2023.1,3,5,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.444, 0.696, 0.481, 0.703], [0.127, 0.71, 0.382, 0.717]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 373, "edu_l1_label": "EDU_O"}, {"txt": "[33] Vicente Ordonez, Girish Kulkarni, and Tamara Berg.Im2text:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.094, 0.723, 0.483, 0.731], [0.127, 0.737, 0.18, 0.745]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 374, "edu_l1_label": "EDU_O"}, {"txt": " Describing images using 1 million captioned pho-tographs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.18, 0.737, 0.48, 0.744], [0.128, 0.752, 0.185, 0.758]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 375, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing sys-temns,24,2011.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.185, 0.752, 0.48, 0.758], [0.127, 0.766, 0.228, 0.772]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 376, "edu_l1_label": "EDU_O"}, {"txt": "[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry,Amanda Askell, Pamela Mishkin, Jack Clark, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.095, 0.778, 0.48, 0.787], [0.127, 0.793, 0.484, 0.801], [0.127, 0.807, 0.426, 0.814]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 377, "edu_l1_label": "EDU_O"}, {"txt": "Learning transferable visual models from natural language supervi-sion.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.426, 0.807, 0.48, 0.816], [0.127, 0.821, 0.483, 0.826], [0.127, 0.836, 0.157, 0.842]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 378, "edu_l1_label": "EDU_O"}, {"txt": "In International conference on machine learning,pages 8748-8763.PmLR,2021.2,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.157, 0.836, 0.481, 0.842], [0.127, 0.848, 0.306, 0.857]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 379, "edu_l1_label": "EDU_O"}, {"txt": "[35] Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.095, 0.862, 0.48, 0.87], [0.127, 0.876, 0.16, 0.884]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 380, "edu_l1_label": "EDU_O"}, {"txt": "Timechat: A time-sensitive multimodal large lan-guage model for long video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.16, 0.876, 0.48, 0.883], [0.127, 0.892, 0.387, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 381, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.387, 0.892, 0.483, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 382, "edu_l1_label": "EDU_O"}, {"txt": "of the IEEE/CVF Conference on Computer Vision and Pat-tern Recognition, pages 14313-14323,2024.1,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.55, 0.097, 0.905, 0.102], [0.55, 0.111, 0.855, 0.117]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 383, "edu_l1_label": "EDU_O"}, {"txt": "[36] Kate Sanders, Nathaniel Weir, and Benjamin Van Durme.Tv-trees:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.516, 0.123, 0.906, 0.131], [0.55, 0.138, 0.605, 0.145]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 384, "edu_l1_label": "EDU_O"}, {"txt": " MMultimodal entailment trees for neuro-symbolic video reasoning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.605, 0.138, 0.904, 0.145], [0.55, 0.153, 0.65, 0.159]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 385, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2402.19467,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.65, 0.153, 0.9, 0.16]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 386, "edu_l1_label": "EDU_O"}, {"txt": "[37] Christoph Schuhmann, Romain Beaumont, Richard Vencu,Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts-man, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.517, 0.165, 0.904, 0.174], [0.55, 0.179, 0.904, 0.187], [0.55, 0.193, 0.905, 0.199], [0.55, 0.209, 0.613, 0.22]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 387, "edu_l1_label": "EDU_O"}, {"txt": "Laion-5b: An open large-scale dataset for training next generation image-text models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.613, 0.209, 0.9, 0.216], [0.549, 0.223, 0.761, 0.229]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 388, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural in-formation processing systems, 35:25278-25294,2022.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.761, 0.223, 0.904, 0.229], [0.549, 0.235, 0.881, 0.243]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 389, "edu_l1_label": "EDU_O"}, {"txt": "[38] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.516, 0.248, 0.904, 0.257], [0.55, 0.263, 0.6, 0.271]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 390, "edu_l1_label": "EDU_O"}, {"txt": "Conceptual captions: A cleaned, hypernymed, im-age alt-text dataset for automatic image captioning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.6, 0.263, 0.904, 0.27], [0.55, 0.278, 0.858, 0.285]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 391, "edu_l1_label": "EDU_O"}, {"txt": "In Pro-ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556-2565,2018.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.858, 0.278, 0.904, 0.284], [0.55, 0.293, 0.904, 0.299], [0.551, 0.304, 0.904, 0.312], [0.55, 0.318, 0.668, 0.326]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 18, "global_sentence_id": 392, "edu_l1_label": "EDU_O"}, {"txt": "[39] Yan Shu, Peitian Zhang, Zheng Liu, Minghao Qin, Junjie Zhou, Tiejun Huang, and Bo Zhao.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16500000, "bbox": [[0.516, 0.332, 0.904, 0.34], [0.55, 0.346, 0.77, 0.354]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 19, "global_sentence_id": 393, "edu_l1_label": "EDU_O"}, {"txt": "Video-xl: Extra-long vision language model for hour-scale video understanding.arXiv preprint arXiv:2409.14485,2024.2,3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16500000, "bbox": [[0.77, 0.346, 0.9, 0.355], [0.55, 0.362, 0.906, 0.368], [0.55, 0.376, 0.827, 0.382]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 19, "global_sentence_id": 394, "edu_l1_label": "EDU_O"}, {"txt": "[40] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang,Haoyang Zhou,Feiyang Wu,Haozhe Chi,Xun Guo,Tian Ye,Yanting Zhang, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.517, 0.388, 0.904, 0.398], [0.55, 0.402, 0.903, 0.41], [0.551, 0.416, 0.726, 0.424]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 20, "global_sentence_id": 395, "edu_l1_label": "EDU_O"}, {"txt": "Moviechat: From dense token to sparse memory for long video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.726, 0.416, 0.904, 0.424], [0.55, 0.431, 0.833, 0.438]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 20, "global_sentence_id": 396, "edu_l1_label": "EDU_O"}, {"txt": "In Proceed-ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18221-18232,2024.1,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.833, 0.431, 0.905, 0.436], [0.55, 0.445, 0.904, 0.452], [0.55, 0.458, 0.859, 0.466]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 20, "global_sentence_id": 397, "edu_l1_label": "EDU_O"}, {"txt": "[41] Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy,and Cordelia Schmid.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.517, 0.472, 0.904, 0.481], [0.55, 0.487, 0.684, 0.494]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 398, "edu_l1_label": "EDU_O"}, {"txt": "Videobert: A joint model for video and language representation learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.684, 0.487, 0.904, 0.494], [0.55, 0.501, 0.783, 0.507]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 399, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF international conference on computer vision,pages 7464-7473,2019.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.783, 0.501, 0.906, 0.509], [0.55, 0.513, 0.906, 0.522], [0.551, 0.53, 0.706, 0.535]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 21, "global_sentence_id": 400, "edu_l1_label": "EDU_O"}, {"txt": "[42] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.Llama:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16800000, "bbox": [[0.517, 0.541, 0.904, 0.549], [0.55, 0.556, 0.904, 0.562], [0.55, 0.569, 0.906, 0.577], [0.55, 0.583, 0.596, 0.591]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 22, "global_sentence_id": 401, "edu_l1_label": "EDU_O"}, {"txt": " Open and efficient foundation language models.arXiv preprint arXiv:2302.13971,2023.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16800000, "bbox": [[0.596, 0.583, 0.906, 0.591], [0.549, 0.598, 0.797, 0.605]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 22, "global_sentence_id": 402, "edu_l1_label": "EDU_O"}, {"txt": "[43] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani,and Manohar Paluri.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.517, 0.611, 0.906, 0.62], [0.55, 0.627, 0.673, 0.633]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 403, "edu_l1_label": "EDU_O"}, {"txt": "Learning spatiotemporal features with 3d convolutional networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.673, 0.627, 0.904, 0.633], [0.55, 0.639, 0.708, 0.647]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 404, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE inter-national conference on computer vision, pages 4489-4497,2015.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.708, 0.639, 0.904, 0.646], [0.55, 0.655, 0.905, 0.661], [0.55, 0.666, 0.595, 0.674]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 23, "global_sentence_id": 405, "edu_l1_label": "EDU_O"}, {"txt": "[44] Peng Wang,Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan,Jinze Bai,Keqin Chen, Xuejing Liu,Jialin Wang,Wenbin Ge, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.516, 0.68, 0.906, 0.689], [0.55, 0.694, 0.904, 0.703], [0.55, 0.708, 0.612, 0.716]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 406, "edu_l1_label": "EDU_O"}, {"txt": "Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.612, 0.708, 0.905, 0.716], [0.55, 0.725, 0.809, 0.73]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 407, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2409.12191,2024.1,2,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.809, 0.725, 0.905, 0.73], [0.55, 0.738, 0.739, 0.744]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 24, "global_sentence_id": 408, "edu_l1_label": "EDU_O"}, {"txt": "[45] Weihan Wang, Qingsong Lv,Wenmeng Yu, Wenyi Hong,Ji Qi,Yan Wang,Junhui Ji,Zhuoyi Yang,Lei Zhao,Song XiX-uan, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17100000, "bbox": [[0.517, 0.75, 0.905, 0.758], [0.551, 0.765, 0.904, 0.772], [0.55, 0.78, 0.614, 0.786]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 409, "edu_l1_label": "EDU_O"}, {"txt": "Cogvlm: Visual expert for pretrained language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17100000, "bbox": [[0.614, 0.78, 0.9, 0.787], [0.556, 0.794, 0.599, 0.8]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 410, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Sys-tems,37:121475-121499,2025.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17100000, "bbox": [[0.599, 0.794, 0.904, 0.799], [0.55, 0.807, 0.751, 0.814]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 25, "global_sentence_id": 411, "edu_l1_label": "EDU_O"}, {"txt": "[46] Xiaohan Wang,Yuhui Zhang,Orr Zohar, and Serena Yeung-Levy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.516, 0.819, 0.905, 0.826], [0.55, 0.834, 0.587, 0.842]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 26, "global_sentence_id": 412, "edu_l1_label": "EDU_O"}, {"txt": "Videoagent: Long-form video understanding with large language model as agent.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.587, 0.834, 0.903, 0.842], [0.55, 0.849, 0.736, 0.856]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 26, "global_sentence_id": 413, "edu_l1_label": "EDU_O"}, {"txt": "In European Conference on Computer Vision, pages 58-76.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.736, 0.849, 0.904, 0.856], [0.551, 0.862, 0.736, 0.869]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 26, "global_sentence_id": 414, "edu_l1_label": "EDU_O"}, {"txt": "Springer,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.736, 0.862, 0.84, 0.869]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 26, "global_sentence_id": 415, "edu_l1_label": "EDU_O"}, {"txt": "[47] Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon,Feng Cheng, Gedas Bertasius, and Mohit Bansal.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17300000, "bbox": [[0.517, 0.876, 0.904, 0.884], [0.551, 0.889, 0.905, 0.898]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 27, "global_sentence_id": 416, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17400000, "bbox": [[0.49, 0.924, 0.511, 0.939]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 28, "global_sentence_id": 417, "edu_l1_label": "EDU_O"}, {"txt": "Videotree:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17500000, "bbox": [[0.127, 0.095, 0.191, 0.104]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 418, "edu_l1_label": "EDU_O"}, {"txt": " Adaptive tree-based video representation for llm reasoning on long videos.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17500000, "bbox": [[0.191, 0.095, 0.48, 0.104], [0.127, 0.111, 0.281, 0.117]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 419, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2405.19209,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17500000, "bbox": [[0.281, 0.111, 0.483, 0.118], [0.127, 0.123, 0.173, 0.131]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 420, "edu_l1_label": "EDU_O"}, {"txt": "[48] Yuetian Weng,Mingfei Han,Haoyu He,Xiaojun Chang,and Bohan Zhuang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.094, 0.138, 0.48, 0.146], [0.128, 0.152, 0.221, 0.16]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 421, "edu_l1_label": "EDU_O"}, {"txt": "Longvlm: Efficient long video understand-ing via large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.221, 0.152, 0.48, 0.16], [0.128, 0.167, 0.313, 0.174]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 422, "edu_l1_label": "EDU_O"}, {"txt": "In European Conference on Computer Vision, pages 453-470.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.313, 0.167, 0.48, 0.174], [0.127, 0.18, 0.328, 0.188]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 423, "edu_l1_label": "EDU_O"}, {"txt": "Springer, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.328, 0.18, 0.422, 0.187]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 424, "edu_l1_label": "EDU_O"}, {"txt": "1,3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.422, 0.18, 0.445, 0.188]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 425, "edu_l1_label": "EDU_O"}, {"txt": "[49] Chenfei Wu,Shengming Yin,Weizhen Qi,Xiaodong Wang,Zecheng Tang, and Nan Duan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17700000, "bbox": [[0.094, 0.194, 0.484, 0.204], [0.127, 0.209, 0.323, 0.216]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 426, "edu_l1_label": "EDU_O"}, {"txt": "Visual chatgpt: Talking,drawing and editing with visual foundation models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17700000, "bbox": [[0.323, 0.209, 0.484, 0.217], [0.127, 0.223, 0.438, 0.23]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 427, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2303.04671,2023.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17700000, "bbox": [[0.438, 0.223, 0.48, 0.23], [0.127, 0.239, 0.338, 0.243]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 428, "edu_l1_label": "EDU_O"}, {"txt": "[50] Jun Xu, Tao Mei, Ting Yao, and Yong Rui.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17800000, "bbox": [[0.095, 0.251, 0.383, 0.259]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 429, "edu_l1_label": "EDU_O"}, {"txt": "Msr-vtt: A large video description dataset for bridging video and language.In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288-5296,2016.1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17800000, "bbox": [[0.383, 0.251, 0.48, 0.259], [0.127, 0.267, 0.48, 0.273], [0.127, 0.279, 0.481, 0.287], [0.127, 0.295, 0.403, 0.301]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 430, "edu_l1_label": "EDU_O"}, {"txt": "[51] Lin Xu,Yilin Zhao, Daquan Zhou, Zhijie Lin,See Kiong$Ng,$and Jiashi Feng.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.093, 0.307, 0.484, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 431, "edu_l1_label": "EDU_O"}, {"txt": "Pllava: Parameter-free llava extension from images to videos for video dense captioning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.093, 0.307, 0.484, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 432, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2404.16994,2024.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.093, 0.307, 0.484, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 433, "edu_l1_label": "EDU_O"}, {"txt": "[52] Yuan Yao,Tianyu Yu,Ao Zhang, Chongyi Wang,Junbo Cui,Hongji Zhu, Tianchi Cai, Haoyu Li,Weilin Zhao,Zhihui He,et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.095, 0.364, 0.483, 0.374], [0.127, 0.378, 0.483, 0.388], [0.127, 0.393, 0.158, 0.4]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 434, "edu_l1_label": "EDU_O"}, {"txt": "Minicpm-v: A gpt-4v level mllm on your phone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.158, 0.393, 0.445, 0.4]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 435, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2408.01800,2024.3,6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.445, 0.393, 0.48, 0.4], [0.127, 0.408, 0.353, 0.414]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 436, "edu_l1_label": "EDU_O"}, {"txt": "[53] Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng,Jingkang Yang, Yuanhan Zhang, Ziyue Wang,Haoran Tan,Chunyuan Li, and Ziwei Liu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.094, 0.42, 0.484, 0.431], [0.127, 0.435, 0.483, 0.444], [0.127, 0.449, 0.309, 0.456]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 437, "edu_l1_label": "EDU_O"}, {"txt": "Long context transfer from language to vision.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.309, 0.449, 0.475, 0.457], [0.127, 0.463, 0.242, 0.471]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 438, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2406.16852,2024.6", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.242, 0.463, 0.483, 0.471], [0.127, 0.478, 0.135, 0.484]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 439, "edu_l1_label": "EDU_O"}, {"txt": "[54] Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang,Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18200000, "bbox": [[0.094, 0.491, 0.483, 0.5], [0.128, 0.506, 0.48, 0.514], [0.127, 0.519, 0.156, 0.527]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 440, "edu_l1_label": "EDU_O"}, {"txt": "Mlvu: A comprehensive benchmark for multi-task long video understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18200000, "bbox": [[0.156, 0.519, 0.48, 0.527], [0.127, 0.533, 0.282, 0.541]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 441, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2406.04264,2024.5", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18200000, "bbox": [[0.282, 0.533, 0.483, 0.542], [0.127, 0.546, 0.172, 0.554]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 442, "edu_l1_label": "EDU_O"}, {"txt": "[55] Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang,Yatian Pang,Wenhao Jiang,Junwu Zhang,Zongwei Li,et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18300000, "bbox": [[0.095, 0.561, 0.48, 0.571], [0.129, 0.576, 0.481, 0.584], [0.127, 0.589, 0.177, 0.598]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 443, "edu_l1_label": "EDU_O"}, {"txt": "Languagebind: Extending video-language pretrain-ing to n-modality by language-based semantic alignment.arXiv preprint arXiv:2310.01852,2023.3", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18300000, "bbox": [[0.177, 0.589, 0.48, 0.598], [0.128, 0.605, 0.483, 0.616], [0.127, 0.619, 0.374, 0.626]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 444, "edu_l1_label": "EDU_O"}, {"txt": "[56] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo-hamed Elhoseiny.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18400000, "bbox": [[0.095, 0.632, 0.48, 0.64], [0.127, 0.647, 0.238, 0.655]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 445, "edu_l1_label": "EDU_O"}, {"txt": "Minigpt-4: Enhancing vision-language understanding with advanced large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18400000, "bbox": [[0.238, 0.647, 0.48, 0.655], [0.127, 0.662, 0.444, 0.669]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 446, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2304.10592,2023.2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18400000, "bbox": [[0.444, 0.662, 0.48, 0.668], [0.126, 0.676, 0.339, 0.682]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 447, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18500000, "bbox": [[0.49, 0.924, 0.509, 0.939]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": 11, "global_sentence_id": 448, "edu_l1_label": "EDU_O"}], "type": "PDF"}
# LVC: A Lightweight Compression Framework for Enhancing VLMs in LongVideo Understanding
## Abstract
## 1.Introduction
## 2.Related Work
### 2.1.Vision-Language Models
### 2.2.Video Large Language Models
### 2.3.Long Video Understanding
## 3.Methods
### 3.1.Problem Formulation of Video Understanding
### 3.2.Model Architecture
### 3.3.Infusing Queries into Video Compressing
## 4.Experiments
### 4.1.Dataset and Benchmarks
### 4.2.Experimental Setup
### 4.3.Results on InternVL2 Series
### 4.4.Analyses on Input Frames
#### 1.The LVC method provides consistent perforance im-provements across various compression ratios for differ-ent scales of InternVL2, with more significant improve-ments observed on MLVU.
#### 2.The performance improvement of LVC becomes more significant as the compression ratio increases.
#### 3.The performance growth of models enhanced with LVC becomes more gradual as the input frame count in-creases.
### 4.5.Ablation Studies
#### Data Ablation:
#### Data Scale Ablation:
#### Component Ablation:
### 4.6.Results on Phi-3.5-Vision
## 5.Discussion and Future Work
## 6.Conclusion
