{"entry_id": "627a2449-79be-42fd-958f-c6af16aeb6eb", "infos": [{"txt": "NLP-Based.NET CLR Event Logs Analyzer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.175, 0.09, 0.824, 0.109]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 1, "global_sentence_id": 0, "edu_l1_label": "BOT"}, {"txt": "S707 음H9[aS:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 100000, "bbox": [[0.029, 0.278, 0.053, 0.459]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 2, "global_sentence_id": 1, "edu_l1_label": "EDU_O"}, {"txt": "so] IA6IZtO.70sC:AIXTe", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 100000, "bbox": [[0.053, 0.278, 0.052, 0.702]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 2, "global_sentence_id": 2, "edu_l1_label": "EDU_O"}, {"txt": "Maxim Stavtsev", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 200000, "bbox": [[0.234, 0.132, 0.362, 0.142]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 3, "global_sentence_id": 3, "edu_l1_label": "EDU_O"}, {"txt": "mastavtsev@edu.hse.ru", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.219, 0.15, 0.377, 0.157]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 4, "global_sentence_id": 4, "edu_l1_label": "EDU_O"}, {"txt": "HSE University", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 400000, "bbox": [[0.247, 0.164, 0.35, 0.174]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 5, "global_sentence_id": 5, "edu_l1_label": "EDU_O"}, {"txt": "Moscow, Russia", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 400000, "bbox": [[0.245, 0.179, 0.351, 0.188]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 6, "global_sentence_id": 6, "edu_l1_label": "EDU_O"}, {"txt": "Sergey Shershakov", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.624, 0.131, 0.775, 0.141]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 7, "global_sentence_id": 7, "edu_l1_label": "EDU_O"}, {"txt": "sshershakov@hse.ru", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 600000, "bbox": [[0.631, 0.15, 0.768, 0.157]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 8, "global_sentence_id": 8, "edu_l1_label": "EDU_O"}, {"txt": "HSE University", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.649, 0.163, 0.752, 0.174]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 9, "global_sentence_id": 9, "edu_l1_label": "EDU_O"}, {"txt": "Moscow, Russia", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.648, 0.179, 0.754, 0.187]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 10, "global_sentence_id": 10, "edu_l1_label": "EDU_O"}, {"txt": "Abstract", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.087, 0.201, 0.162, 0.212]]}]}, "tags": ["title"], "label": "abstract", "web_segment_id": 11, "global_sentence_id": 11, "edu_l1_label": "BOS"}, {"txt": "In this paper, we present a tool for analyzing .NET CLR event logs based on a novel method inspired by Natural Lan-guage Processing (NLP) approach.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.087, 0.222, 0.479, 0.23], [0.087, 0.24, 0.483, 0.245], [0.087, 0.256, 0.322, 0.261]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 12, "edu_l1_label": "IOS"}, {"txt": "Our research addresses the growing need for effective monitoring and optimization of software systems through detailed event log analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.322, 0.256, 0.48, 0.261], [0.087, 0.267, 0.48, 0.277], [0.087, 0.283, 0.457, 0.29]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 13, "edu_l1_label": "IOS"}, {"txt": "We utilize a BERT-based architecture with an enhanced tokeniza-tion process customized to event logs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.457, 0.283, 0.48, 0.29], [0.088, 0.299, 0.483, 0.305], [0.088, 0.313, 0.346, 0.321]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 14, "edu_l1_label": "IOS"}, {"txt": "The tool, developed using PYTHON, its libraries, and an SQLITE database, allows both conducting experiments for academic purposes and ef-ficiently solving industry-emerging tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.346, 0.313, 0.48, 0.321], [0.087, 0.33, 0.48, 0.336], [0.087, 0.342, 0.483, 0.348], [0.087, 0.358, 0.367, 0.366]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 15, "edu_l1_label": "IOS"}, {"txt": "Our experiments demonstrate the efficacy of our approach in compressing event sequences, detecting recurring patterns, and identi-fying anomalies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.367, 0.358, 0.48, 0.367], [0.087, 0.372, 0.48, 0.383], [0.087, 0.391, 0.483, 0.394], [0.088, 0.403, 0.203, 0.412]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 16, "edu_l1_label": "IOS"}, {"txt": "The trained model shows promising re-sults,with a high accuracy rate in anomaly detection, which demonstrates the potential of NLP methods to improve the reliability and stability of software systems.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.203, 0.403, 0.484, 0.411], [0.087, 0.421, 0.48, 0.427], [0.087, 0.433, 0.48, 0.442], [0.087, 0.451, 0.379, 0.457]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 12, "global_sentence_id": 17, "edu_l1_label": "IOS"}, {"txt": "Demo video:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.087, 0.463, 0.181, 0.472]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 18, "edu_l1_label": "EDU_O"}, {"txt": "YouTube", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.181, 0.463, 0.238, 0.472]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 19, "edu_l1_label": "EDU_O"}, {"txt": "GitHub:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.087, 0.479, 0.146, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 20, "edu_l1_label": "EDU_O"}, {"txt": "NLP-CLR-LogAnalyzer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.146, 0.479, 0.305, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 14, "global_sentence_id": 21, "edu_l1_label": "EDU_O"}, {"txt": "1 Introduction", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.088, 0.513, 0.23, 0.522]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 15, "global_sentence_id": 22, "edu_l1_label": "BOS"}, {"txt": "Most organizations use various software systems, necessitat-ing effective monitoring and resource allocation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.087, 0.533, 0.483, 0.54], [0.087, 0.549, 0.408, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 23, "edu_l1_label": "IOS"}, {"txt": "Event logs,as primary artifacts of software operations, are crucial to un-derstanding system functions and identifying optimization opportunities.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.408, 0.549, 0.483, 0.557], [0.087, 0.564, 0.483, 0.57], [0.087, 0.578, 0.475, 0.586], [0.087, 0.595, 0.181, 0.601]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 24, "edu_l1_label": "IOS"}, {"txt": "Process mining combines process science and data analysis methods to extract value from such logs, which allows the optimization of processes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.181, 0.595, 0.48, 0.602], [0.087, 0.609, 0.479, 0.617], [0.087, 0.625, 0.326, 0.631]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 25, "edu_l1_label": "IOS"}, {"txt": "This project extends the approach proposed by Stepanov and Mitsyuk [6]by enhanc-ing low-level .NET event log analysis in two ways: 1) pattern detection, to understand system interactions, and 2) anomaly detection, to identify and prevent abnormal behaviors.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.326, 0.625, 0.48, 0.632], [0.087, 0.641, 0.481, 0.646], [0.088, 0.655, 0.48, 0.662], [0.087, 0.669, 0.48, 0.68], [0.087, 0.684, 0.451, 0.691]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 26, "edu_l1_label": "IOS"}, {"txt": "We apply neural network models for automated and scalable analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.108, 0.699, 0.479, 0.707], [0.087, 0.716, 0.207, 0.722]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 27, "edu_l1_label": "IOS"}, {"txt": "Unsupervised learning is utilized due to large, unlabeled datasets typical in software systems.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.207, 0.716, 0.48, 0.722], [0.087, 0.728, 0.44, 0.737]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 28, "edu_l1_label": "IOS"}, {"txt": "Using transformer-based NLP methods, we tokenizeevent traces to identify patterns and anomalies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.44, 0.728, 0.48, 0.739], [0.087, 0.744, 0.48, 0.753], [0.087, 0.759, 0.297, 0.768]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 29, "edu_l1_label": "IOS"}, {"txt": "This work demonstrates the effective application of NLP techniques to .NET CLR event log analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.297, 0.759, 0.48, 0.767], [0.087, 0.776, 0.48, 0.783], [0.087, 0.788, 0.172, 0.797]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 30, "edu_l1_label": "IOS"}, {"txt": "This article is organized as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.105, 0.804, 0.337, 0.812]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 31, "edu_l1_label": "IOS"}, {"txt": " Section 2 provides an overview of existing solutions for event log analysis,Sec-tion 3 describes the algorithms used in the project, Section 4presents the proposed method for event log analysis, includ-ing training the machine learning model and implementa-tion of the proposed algorithms, and finally Section 5 offers a summary of the article.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.337, 0.804, 0.48, 0.813], [0.087, 0.822, 0.481, 0.827], [0.087, 0.835, 0.48, 0.843], [0.087, 0.852, 0.482, 0.854], [0.088, 0.866, 0.483, 0.872], [0.087, 0.88, 0.48, 0.888], [0.087, 0.896, 0.255, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 32, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.611, 0.203, 0.81, 0.357]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 19, "global_sentence_id": 33, "edu_l1_label": "EDU_O"}, {"txt": "Figure 1. An example of a predefined hierarchy used to raise the abstraction level of low-level event logs, where \"root\" is the most abstract event and AssemblyLoader/S-tart_System_Threading is the most specific, detailed event.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.517, 0.372, 0.915, 0.429]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 20, "global_sentence_id": 34, "edu_l1_label": "EDU_O"}, {"txt": "2 Related Work", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.519, 0.482, 0.67, 0.492]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 21, "global_sentence_id": 35, "edu_l1_label": "BOS"}, {"txt": "In [6] authors propose a method for extracting high-level activities from low-level event logs of program execution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 36, "edu_l1_label": "IOS"}, {"txt": "To achieve this, they developed a tool called PROCFILER, which collects events that occur during the execution of programs written in the $C\\#$programming language in the .NET CLR runtime environment and creates a log from them.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 37, "edu_l1_label": "IOS"}, {"txt": "They then apply a predefined hierarchy to raise the abstraction level of the events.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 38, "edu_l1_label": "IOS"}, {"txt": "An example of such a hierarchy is shown in Figure 1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 39, "edu_l1_label": "IOS"}, {"txt": "The root is an artificially added, most abstract event.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 40, "edu_l1_label": "IOS"}, {"txt": "For instance, AssemblyLoader/Start_System_Threading represents a very specific and detailed event.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 41, "edu_l1_label": "IOS"}, {"txt": "This event can be abstracted to a higher level by naming it AssemblyLoad-er/Start.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.516, 0.502, 0.915, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 42, "edu_l1_label": "IOS"}, {"txt": "In our project, we utilized the results of the Procfiler tool,specifically logs with the lowest level of abstraction, meaning they contain events that are the leaves in the hierarchical tree shown in Figure 1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.535, 0.698, 0.915, 0.708], [0.52, 0.717, 0.911, 0.723], [0.519, 0.729, 0.91, 0.737], [0.519, 0.744, 0.673, 0.752]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 23, "global_sentence_id": 43, "edu_l1_label": "IOS"}, {"txt": "For the task of anomaly detection in event logs, supervised learning methods have been applied, treating this task as a binary classification problem [3].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.534, 0.759, 0.91, 0.767], [0.518, 0.773, 0.91, 0.783], [0.519, 0.789, 0.736, 0.797]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 24, "global_sentence_id": 44, "edu_l1_label": "IOS"}, {"txt": "However,this significantly reduces the applicability of such methods in real systems, as it requires a prelabeled dataset, and more importantly, limits the ability to detect previously unseen anomalies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.736, 0.789, 0.911, 0.799], [0.519, 0.807, 0.911, 0.813], [0.52, 0.821, 0.913, 0.828], [0.519, 0.835, 0.846, 0.843]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 24, "global_sentence_id": 45, "edu_l1_label": "IOS"}, {"txt": "There are works that use unsupervised learning approaches [2] based on LSTM, as well as the BERT model [5], which is based on the transformerarchitecture and employs preliminary tokenization of the event log.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.846, 0.835, 0.911, 0.843], [0.519, 0.852, 0.909, 0.858], [0.519, 0.867, 0.91, 0.873], [0.519, 0.882, 0.91, 0.89], [0.519, 0.896, 0.715, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 24, "global_sentence_id": 46, "edu_l1_label": "IOS"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.698, 0.061, 0.913, 0.071]]}]}, "tags": ["header"], "label": "O", "web_segment_id": 1, "global_sentence_id": 47, "edu_l1_label": "EDU_O"}, {"txt": "To the best of our knowledge, no previous study has in-vestigated the applicability of NLP methods for automated pattern and anomaly detection for the domain of low-level event .NET CLR logs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.104, 0.094, 0.484, 0.103], [0.087, 0.112, 0.48, 0.118], [0.087, 0.128, 0.48, 0.133], [0.087, 0.143, 0.23, 0.149]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 48, "edu_l1_label": "IOS"}, {"txt": "3 Algorithms", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.087, 0.169, 0.218, 0.179]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 3, "global_sentence_id": 49, "edu_l1_label": "BOS"}, {"txt": "There are some major limitations to the rapid and efficient analysis by process mining methods.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.087, 0.189, 0.48, 0.198], [0.087, 0.206, 0.338, 0.213]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 50, "edu_l1_label": "IOS"}, {"txt": "Among them are the large volume of event logs and the need to perform a prelim-inary analysis of the input data to understand the structure of interacting process elements.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.338, 0.206, 0.48, 0.213], [0.087, 0.218, 0.48, 0.227], [0.087, 0.235, 0.48, 0.242], [0.087, 0.251, 0.301, 0.257]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 51, "edu_l1_label": "IOS"}, {"txt": "One of the most important hypotheses in this work is that approaches adapted from the field of NLP can remove such limitations.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.301, 0.251, 0.48, 0.258], [0.088, 0.264, 0.48, 0.272], [0.087, 0.279, 0.364, 0.287]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 52, "edu_l1_label": "IOS"}, {"txt": "3.1 Event Log Encoding", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.087, 0.307, 0.271, 0.318]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 53, "edu_l1_label": "IOS"}, {"txt": "Most NLP algorithms are applied to sequential data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.087, 0.325, 0.453, 0.334]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 54, "edu_l1_label": "IOS"}, {"txt": "For the task of event log analysis, we need to represent logs as sequences for further application of the algorithm.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.453, 0.325, 0.48, 0.334], [0.087, 0.341, 0.48, 0.349], [0.087, 0.357, 0.425, 0.364]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 55, "edu_l1_label": "IOS"}, {"txt": "Table 1 shows an example fragment of an event log, where each event has three mandatory attributes:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.085, 0.37, 0.486, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 56, "edu_l1_label": "IOS"}, {"txt": " it is the realiza-tion of some activity from the set of activities$\\mathbb {A}-ac_{i},$ 2timestamp $ts_{i}$i corresponding to the event's start time, and a process identifier during which it was executed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.085, 0.37, 0.486, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 57, "edu_l1_label": "IOS"}, {"txt": "The set A is finite and defined by the set of activities allowed in the .NET runtime environment, their list is presented in the Appendix of the work [6].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.085, 0.37, 0.486, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 58, "edu_l1_label": "IOS"}, {"txt": "Table 1. Example fragment of a low-level event log from the CLR environment.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.086, 0.506, 0.483, 0.531]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 59, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.103, 0.55, 0.463, 0.638]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 9, "global_sentence_id": 60, "edu_l1_label": "EDU_O"}, {"txt": "The considered event log contains events related to the .NET application's execution thread (e.g.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.085, 0.658, 0.483, 0.744]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 61, "edu_l1_label": "IOS"}, {"txt": "ID 31237)and system threads (e.g., ID -1).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.085, 0.658, 0.483, 0.744]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 62, "edu_l1_label": "IOS"}, {"txt": "These events can be combined into one trace by merging events according to their IDs and timestamps, resulting in the trace $\\text {trace}_{1}=$<Method/MemoryAllocatedForJitCode,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.085, 0.658, 0.483, 0.744]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 63, "edu_l1_label": "IOS"}, {"txt": "GC/SampledObjectAllocation,Method/LoadVerbose,Buffer/Returned).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.093, 0.749, 0.436, 0.758], [0.088, 0.764, 0.208, 0.772]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 64, "edu_l1_label": "EDU_O"}, {"txt": "By using this approach, we obtained the final set of event log traces.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.105, 0.779, 0.48, 0.788], [0.087, 0.794, 0.157, 0.803]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 65, "edu_l1_label": "IOS"}, {"txt": "In order to represent traces as textual sequences,each activity from the set of all allowed activities A in this work is encoded with a unique non-control Unicode character1,let U be the subset of these characters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.104, 0.809, 0.475, 0.818], [0.087, 0.827, 0.48, 0.833], [0.088, 0.841, 0.484, 0.85], [0.089, 0.855, 0.351, 0.863]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 66, "edu_l1_label": "IOS"}, {"txt": "Thus,we formed a bijection between the set A and the set U,f:.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.351, 0.855, 0.48, 0.864], [0.088, 0.869, 0.393, 0.882]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 67, "edu_l1_label": "IOS"}, {"txt": "1https://en.wikipedia.org/wiki/List_of_Unicode_characters", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.09, 0.895, 0.4, 0.903]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 68, "edu_l1_label": "EDU_O"}, {"txt": "For example,consider a bijective function $f_{0}\\subset f$, defined by the set of pairs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.517, 0.093, 0.913, 0.121]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 15, "global_sentence_id": 69, "edu_l1_label": "IOS"}, {"txt": "fo={(Method/MemoryAllocatedForJitCode,\"a\"),", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.55, 0.145, 0.882, 0.155]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 16, "global_sentence_id": 70, "edu_l1_label": "EDU_O"}, {"txt": "(Method/LoadVerbose,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.589, 0.165, 0.743, 0.174]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 17, "global_sentence_id": 71, "edu_l1_label": "EDU_O"}, {"txt": "(GC/SampledObjectAllocation,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.589, 0.183, 0.798, 0.193]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 18, "global_sentence_id": 72, "edu_l1_label": "EDU_O"}, {"txt": "(Buffer/Returned,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.59, 0.203, 0.708, 0.212]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 19, "global_sentence_id": 73, "edu_l1_label": "EDU_O"}, {"txt": "Then the trace trace1 can be represented as the sequence $\\text {q}_{}=\\text {\"acb\".", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.517, 0.227, 0.913, 0.257]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 74, "edu_l1_label": "EDU_O"}, {"txt": "}$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.517, 0.227, 0.913, 0.257]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 20, "global_sentence_id": 75, "edu_l1_label": "EDU_O"}, {"txt": "We reduced the task of representing a trace to a sequence of Unicode characters, solvable using NLP tokenization algo-rithms such as BPE, WoRDPIECE, and UNIGRAM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.535, 0.258, 0.91, 0.267], [0.519, 0.275, 0.913, 0.281], [0.519, 0.29, 0.824, 0.297]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 76, "edu_l1_label": "IOS"}, {"txt": "We chose the BPE algorithm because it preserves a dictionary of tokens,merging the most common pairs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.824, 0.29, 0.911, 0.297], [0.519, 0.304, 0.914, 0.313], [0.519, 0.321, 0.743, 0.327]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 77, "edu_l1_label": "IOS"}, {"txt": "After completing the tok-enizer training, we obtained the final set of permitted tokens,denoted as T, where each token represents a subsequence of Unicode characters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.743, 0.321, 0.913, 0.327], [0.519, 0.336, 0.914, 0.344], [0.52, 0.349, 0.913, 0.357], [0.519, 0.364, 0.652, 0.372]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 78, "edu_l1_label": "IOS"}, {"txt": "Any sequence seqi can be represented as toker$_{}=\\left(t_{j}\\right.$ $\\left.t_{j}\\in \\mathbb {T},=1,\\cdots ,\\varphi (\\text {q})\\right),$  where $φ(seq)$ determines the number of tokens.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.516, 0.377, 0.913, 0.465]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 79, "edu_l1_label": "IOS"}, {"txt": "The tokenization process, defined as $\\mathcal {T}:\\text {seq}_{i}\\rightarrow \\text {tokens}_{i}$i, converts $\\text {seq}_{1}$ into the set $\\text {tokens}_{1}=$ $\\left[{}^{\\prime }ac^{\\prime },{}^{\\prime }bd^{\\prime }\\right]$.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.516, 0.377, 0.913, 0.465]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 80, "edu_l1_label": "IOS"}, {"txt": "Tokenization also compresses traces, reducing the input sequence length significantly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.516, 0.377, 0.913, 0.465]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 81, "edu_l1_label": "IOS"}, {"txt": "Each token is encoded with a unique number correspond-ing to a value in the embedding table (numeric vectors),which in turn are trainable parameters of the neural network,the configuration of which we will describe in Section 3.2.Using numeric vectors, we can encode traces and feed them to the neural network input.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.535, 0.47, 0.914, 0.477], [0.519, 0.485, 0.914, 0.494], [0.519, 0.502, 0.915, 0.511], [0.52, 0.515, 0.914, 0.524], [0.519, 0.529, 0.904, 0.539], [0.518, 0.545, 0.708, 0.554]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 23, "global_sentence_id": 82, "edu_l1_label": "IOS"}, {"txt": "3.2 Neural Network Configuration", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.519, 0.574, 0.783, 0.583]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 24, "global_sentence_id": 83, "edu_l1_label": "IOS"}, {"txt": "A key algorithm in deep learning, especially in NLP, is the transformer architecture [7], which consists of an encoder and a decoder.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.519, 0.592, 0.911, 0.601], [0.519, 0.609, 0.911, 0.617], [0.519, 0.626, 0.62, 0.637]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 25, "global_sentence_id": 84, "edu_l1_label": "IOS"}, {"txt": "The encoder processes the input sequence with the attention mechanism and feed-forward layers,pro-ducing vectors that the decoder further transforms into a probability vector.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.62, 0.626, 0.91, 0.632], [0.519, 0.641, 0.913, 0.647], [0.519, 0.653, 0.91, 0.662], [0.519, 0.671, 0.645, 0.677]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 25, "global_sentence_id": 85, "edu_l1_label": "IOS"}, {"txt": "BERT [1] is a transformer model that is based only on the encoder and applies attention to tokens based on their context within a sequence.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.645, 0.671, 0.91, 0.677], [0.519, 0.682, 0.911, 0.692], [0.519, 0.698, 0.797, 0.707]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 25, "global_sentence_id": 86, "edu_l1_label": "IOS"}, {"txt": "One of the ways to train BERT is the Masked Language Modeling (MLM) approach, which involves masking a certain percentage of randomly selected tokens with a special token [MASK].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.535, 0.713, 0.911, 0.723], [0.518, 0.729, 0.91, 0.737], [0.519, 0.747, 0.91, 0.753], [0.52, 0.758, 0.582, 0.767]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 26, "global_sentence_id": 87, "edu_l1_label": "IOS"}, {"txt": "During training, the model aims to minimize the loss function's error by predicting the token hidden behind the [MASK].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.582, 0.758, 0.91, 0.768], [0.519, 0.773, 0.91, 0.782], [0.519, 0.789, 0.608, 0.798]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 26, "global_sentence_id": 88, "edu_l1_label": "IOS"}, {"txt": "We claim that a model trained on unlabeled correct (without anomalies) event logs can detect events that do not match the context of normal behavior.For this reason,in this work, we apply a BERT-based model for anomaly detection.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.608, 0.789, 0.911, 0.797], [0.519, 0.807, 0.912, 0.812], [0.52, 0.819, 0.915, 0.83], [0.519, 0.835, 0.911, 0.845], [0.52, 0.849, 0.586, 0.858]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 26, "global_sentence_id": 89, "edu_l1_label": "IOS"}, {"txt": "After analyzing existing BERT-based model architectures,we selected the SQUEEZEBERT architecture [4] for this work.The authors of the work presenting this architecture show", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.535, 0.865, 0.915, 0.874], [0.519, 0.881, 0.915, 0.888], [0.518, 0.895, 0.904, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 27, "global_sentence_id": 90, "edu_l1_label": "IOS"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.087, 0.061, 0.317, 0.07]]}]}, "tags": ["header"], "label": "O", "web_segment_id": 1, "global_sentence_id": 91, "edu_l1_label": "EDU_O"}, {"txt": "that the majority of the model's parameters and the bulk of the time during its application are concentrated in the feed-forward layers.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.087, 0.095, 0.481, 0.103], [0.087, 0.11, 0.484, 0.115], [0.087, 0.125, 0.189, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 92, "edu_l1_label": "IOS"}, {"txt": "They propose a convolution-based approach borrowed from the field of computer vision to optimize the neural network.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.189, 0.125, 0.475, 0.133], [0.087, 0.14, 0.479, 0.148], [0.087, 0.158, 0.195, 0.164]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 93, "edu_l1_label": "IOS"}, {"txt": "As a result, they manage to reduce the num-ber of parameters to approximately 40 million, compared to 100 million in original BERT, which requires less computa-tional resources, without significant loss of quality.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.195, 0.158, 0.482, 0.164], [0.087, 0.169, 0.48, 0.179], [0.087, 0.185, 0.483, 0.193], [0.087, 0.201, 0.428, 0.208]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 94, "edu_l1_label": "IOS"}, {"txt": "4 Method", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.087, 0.229, 0.185, 0.239]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 3, "global_sentence_id": 95, "edu_l1_label": "BOS"}, {"txt": "4.1 Patterns Detection", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.087, 0.249, 0.261, 0.258]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 4, "global_sentence_id": 96, "edu_l1_label": "IOS"}, {"txt": "Algorithm 1 Algorithm for Extracting Traces", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.087, 0.282, 0.395, 0.292]]}]}, "tags": ["title"], "label": "code_title", "web_segment_id": 5, "global_sentence_id": 97, "edu_l1_label": "EDU_O"}, {"txt": "1:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.092, 0.302, 0.104, 0.308]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 98, "edu_l1_label": "EDU_O"}, {"txt": " function PROCESSXESTRACES(input_filepath)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.104, 0.302, 0.32, 0.31]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 99, "edu_l1_label": "EDU_O"}, {"txt": "2:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.312, 0.301, 0.32]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 7, "global_sentence_id": 100, "edu_l1_label": "EDU_O"}, {"txt": "   log$\\leftarrow$ReadXES(input_filepath)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.312, 0.301, 0.32]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 7, "global_sentence_id": 101, "edu_l1_label": "EDU_O"}, {"txt": "3:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.321, 0.378, 0.33]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 8, "global_sentence_id": 102, "edu_l1_label": "EDU_O"}, {"txt": "   event_log$\\leftarrow$FilterLogForNeededColumns(log)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.321, 0.378, 0.33]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 8, "global_sentence_id": 103, "edu_l1_label": "EDU_O"}, {"txt": "4:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.331, 0.392, 0.34]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 9, "global_sentence_id": 104, "edu_l1_label": "EDU_O"}, {"txt": "   needed_indexes$\\leftarrow$GetNeededIndexes(event_log)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.331, 0.392, 0.34]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 9, "global_sentence_id": 105, "edu_l1_label": "EDU_O"}, {"txt": "5:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.341, 0.43, 0.35]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 10, "global_sentence_id": 106, "edu_l1_label": "EDU_O"}, {"txt": "   outliers$\\leftarrow$IdentifyOutliers(event_log,needed_indexes)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.341, 0.43, 0.35]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 10, "global_sentence_id": 107, "edu_l1_label": "EDU_O"}, {"txt": "6:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.352, 0.451, 0.36]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 108, "edu_l1_label": "EDU_O"}, {"txt": "   event_log$\\leftarrow$FilterLogByIndexes(event_log,needed_indexes)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.352, 0.451, 0.36]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 109, "edu_l1_label": "EDU_O"}, {"txt": "7:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.36, 0.35, 0.371]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 12, "global_sentence_id": 110, "edu_l1_label": "EDU_O"}, {"txt": "   traces_log$\\leftarrow$CreateTracesLog(event_log)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.36, 0.35, 0.371]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 12, "global_sentence_id": 111, "edu_l1_label": "EDU_O"}, {"txt": "8:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.372, 0.494, 0.38]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 13, "global_sentence_id": 112, "edu_l1_label": "EDU_O"}, {"txt": "   final_trace_log$\\leftarrow$IntegrateOutliersIntoTraces(traces_log,outliers)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.372, 0.494, 0.38]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 13, "global_sentence_id": 113, "edu_l1_label": "EDU_O"}, {"txt": "9:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.381, 0.427, 0.39]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 14, "global_sentence_id": 114, "edu_l1_label": "EDU_O"}, {"txt": "   list_of_traces$\\leftarrow$ConvertTracesToList(final_trace_log)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.091, 0.381, 0.427, 0.39]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 14, "global_sentence_id": 115, "edu_l1_label": "EDU_O"}, {"txt": "10:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.087, 0.391, 0.239, 0.4]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 15, "global_sentence_id": 116, "edu_l1_label": "EDU_O"}, {"txt": "   return list_of_traces", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.087, 0.391, 0.239, 0.4]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 15, "global_sentence_id": 117, "edu_l1_label": "EDU_O"}, {"txt": "11:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.088, 0.403, 0.106, 0.408]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 16, "global_sentence_id": 118, "edu_l1_label": "EDU_O"}, {"txt": "end function", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.106, 0.403, 0.172, 0.41]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 16, "global_sentence_id": 119, "edu_l1_label": "EDU_O"}, {"txt": "Algorithm 2 Algorithm for Tokenizing Traces", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.087, 0.452, 0.4, 0.463]]}]}, "tags": ["title"], "label": "code_title", "web_segment_id": 17, "global_sentence_id": 120, "edu_l1_label": "EDU_O"}, {"txt": "1:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.092, 0.472, 0.104, 0.478]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 18, "global_sentence_id": 121, "edu_l1_label": "EDU_O"}, {"txt": " function PROCESSTRACESTOSEQUENCES(traces, LoA)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.104, 0.472, 0.352, 0.48]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 18, "global_sentence_id": 122, "edu_l1_label": "EDU_O"}, {"txt": "2:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.48, 0.353, 0.491]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 19, "global_sentence_id": 123, "edu_l1_label": "EDU_O"}, {"txt": "   accepted_events$\\leftarrow$LoadAcceptedEvents()", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.48, 0.353, 0.491]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 19, "global_sentence_id": 124, "edu_l1_label": "EDU_O"}, {"txt": "3:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.492, 0.412, 0.5]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 20, "global_sentence_id": 125, "edu_l1_label": "EDU_O"}, {"txt": "   event_codes$\\leftarrow$MapEventsToCodes(accepted_events)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.492, 0.412, 0.5]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 20, "global_sentence_id": 126, "edu_l1_label": "EDU_O"}, {"txt": "4:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.501, 0.208, 0.513]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 21, "global_sentence_id": 127, "edu_l1_label": "EDU_O"}, {"txt": "   sequences$\\leftarrow []$", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.501, 0.208, 0.513]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 21, "global_sentence_id": 128, "edu_l1_label": "EDU_O"}, {"txt": "5:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.513, 0.261, 0.519]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 22, "global_sentence_id": 129, "edu_l1_label": "EDU_O"}, {"txt": "   for each trace in traces do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.513, 0.261, 0.519]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 22, "global_sentence_id": 130, "edu_l1_label": "EDU_O"}, {"txt": "6:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.522, 0.453, 0.53]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 23, "global_sentence_id": 131, "edu_l1_label": "EDU_O"}, {"txt": "      sequence$\\leftarrow$ConvertTraceToSequence(trace,event_codes)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.522, 0.453, 0.53]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 23, "global_sentence_id": 132, "edu_l1_label": "EDU_O"}, {"txt": "7:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.533, 0.296, 0.54]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 24, "global_sentence_id": 133, "edu_l1_label": "EDU_O"}, {"txt": "      sequences.append(sequence)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.533, 0.296, 0.54]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 24, "global_sentence_id": 134, "edu_l1_label": "EDU_O"}, {"txt": "8:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.542, 0.164, 0.549]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 25, "global_sentence_id": 135, "edu_l1_label": "EDU_O"}, {"txt": "   end for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.091, 0.542, 0.164, 0.549]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 25, "global_sentence_id": 136, "edu_l1_label": "EDU_O"}, {"txt": "9:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.09, 0.552, 0.427, 0.561]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 26, "global_sentence_id": 137, "edu_l1_label": "EDU_O"}, {"txt": "   processed_traces$\\leftarrow$TokenizeSequences(sequences,LoA)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.09, 0.552, 0.427, 0.561]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 26, "global_sentence_id": 138, "edu_l1_label": "EDU_O"}, {"txt": "10:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.087, 0.563, 0.254, 0.571]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 27, "global_sentence_id": 139, "edu_l1_label": "EDU_O"}, {"txt": "   return processed_traces", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.087, 0.563, 0.254, 0.571]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 27, "global_sentence_id": 140, "edu_l1_label": "EDU_O"}, {"txt": "11:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.088, 0.573, 0.106, 0.578]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 28, "global_sentence_id": 141, "edu_l1_label": "EDU_O"}, {"txt": "end function", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.106, 0.573, 0.172, 0.58]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 28, "global_sentence_id": 142, "edu_l1_label": "EDU_O"}, {"txt": "Tokens in the tokenizer's dictionary reflect frequently occurring interactions, thus considered patterns in this work.For instance, a group of events events encoded by symbols $a$,$b$,c appearing as the token bac in the event trace is a pattern.We trained 13 tokenizers with dictionary sizes from 512 to 20,000 tokens, some with a maximum token length limit,to analyze these patterns at different abstraction levels-higher levels encode larger numbers of events into single tokens.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.086, 0.607, 0.484, 0.737]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 29, "global_sentence_id": 143, "edu_l1_label": "IOS"}, {"txt": "The pattern detection process involves two algorithms.Algorithm 1 describes obtaining a list of traces from raw CLR low-level event logs by extracting necessary columns and combining events by timestamps.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.105, 0.744, 0.484, 0.752], [0.088, 0.759, 0.479, 0.767], [0.087, 0.774, 0.48, 0.783], [0.087, 0.791, 0.342, 0.797]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 30, "global_sentence_id": 144, "edu_l1_label": "IOS"}, {"txt": "Algorithm 2 extracts tokens from event traces at a specified abstraction level (LoA),forming a list of acceptable events and applying a mapping to create sequences.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.342, 0.791, 0.48, 0.797], [0.087, 0.806, 0.483, 0.814], [0.087, 0.819, 0.48, 0.828], [0.087, 0.837, 0.203, 0.843]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 30, "global_sentence_id": 145, "edu_l1_label": "IOS"}, {"txt": "These sequences are then tokenized using the trained tokenizers, resulting in a list of tokens for each trace.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.203, 0.837, 0.48, 0.845], [0.087, 0.849, 0.48, 0.858], [0.087, 0.865, 0.124, 0.873]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 30, "global_sentence_id": 146, "edu_l1_label": "IOS"}, {"txt": "Consider an example of the results of the pattern search algorithms on event logs of $25C\\#$ program runs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.086, 0.879, 0.483, 0.906]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 31, "global_sentence_id": 147, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5400000, "bbox": [[0.569, 0.094, 0.865, 0.179]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 32, "global_sentence_id": 148, "edu_l1_label": "EDU_O"}, {"txt": "Figure 2. Non tokenized log", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.618, 0.194, 0.813, 0.205]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 33, "global_sentence_id": 149, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.567, 0.227, 0.864, 0.312]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 34, "global_sentence_id": 150, "edu_l1_label": "EDU_O"}, {"txt": "Figure 3. Tokenized log with LoA 10", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.59, 0.328, 0.841, 0.342]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 35, "global_sentence_id": 151, "edu_l1_label": "EDU_O"}, {"txt": "We visualized the obtained traces, showing the trace ID on the horizontal axis and the number of events (tokens)on the vertical axis, with each color representing a different token/event.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.535, 0.366, 0.91, 0.374], [0.519, 0.384, 0.913, 0.391], [0.519, 0.399, 0.913, 0.405], [0.519, 0.413, 0.603, 0.42]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 36, "global_sentence_id": 152, "edu_l1_label": "IOS"}, {"txt": "Tokenization not only significantly reduced the average trace length from 8000 events in the non-tokenized log (Figure 2) to 200-300 tokens at an abstraction level of 10(Figure 3), but also produced a list of frequently co-occurring events during program execution, which is the set of to-kens of the encoded trace of program execution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.603, 0.413, 0.91, 0.42], [0.519, 0.429, 0.91, 0.436], [0.519, 0.442, 0.91, 0.451], [0.518, 0.456, 0.91, 0.467], [0.519, 0.474, 0.914, 0.48], [0.519, 0.485, 0.855, 0.495]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 36, "global_sentence_id": 153, "edu_l1_label": "IOS"}, {"txt": "This list can be further analyzed by domain experts to gain deeper insights into the execution process and potentially identify performance bottlenecks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.855, 0.485, 0.912, 0.495], [0.519, 0.504, 0.91, 0.511], [0.519, 0.518, 0.911, 0.528], [0.519, 0.535, 0.689, 0.54]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 36, "global_sentence_id": 154, "edu_l1_label": "IOS"}, {"txt": "The method of pattern detection presented in this work is an alternative to the repeated alphabets method presented in [6].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.534, 0.547, 0.912, 0.556], [0.519, 0.565, 0.91, 0.571], [0.519, 0.579, 0.56, 0.586]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 37, "global_sentence_id": 155, "edu_l1_label": "IOS"}, {"txt": "Thus, we can say that our method is more versatile as it is used not only for pattern search, but also when using the SQUEEZEBERT neural network.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.56, 0.579, 0.911, 0.586], [0.519, 0.594, 0.91, 0.604], [0.519, 0.608, 0.751, 0.616]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 37, "global_sentence_id": 156, "edu_l1_label": "IOS"}, {"txt": "4.2 Anomalies Detection", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.519, 0.634, 0.71, 0.643]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 38, "global_sentence_id": 157, "edu_l1_label": "IOS"}, {"txt": "Anomaly detection is based on the SQUEEZEBERT neural network architecture, and there are two main approaches in the NLP field for using machine learning models.The first approach involves fine-tuning an already trained model for specific tasks, which is usually optimal.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.519, 0.654, 0.911, 0.662], [0.519, 0.671, 0.911, 0.677], [0.519, 0.684, 0.911, 0.692], [0.519, 0.698, 0.912, 0.707], [0.519, 0.713, 0.815, 0.727]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 39, "global_sentence_id": 158, "edu_l1_label": "IOS"}, {"txt": "However,this approach is not feasible in our case as the BERT-based model has not been previously applied to .NET CLR event logs.Therefore, we train the model from scratch using a tokenizer with a maximum abstraction level of 13, a dictionary size of 20,000 tokens, and a maximum token length of 300 charac-ters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.815, 0.713, 0.91, 0.722], [0.519, 0.731, 0.912, 0.737], [0.519, 0.744, 0.914, 0.752], [0.518, 0.759, 0.913, 0.768], [0.519, 0.776, 0.913, 0.782], [0.519, 0.79, 0.913, 0.797], [0.519, 0.805, 0.548, 0.813]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 39, "global_sentence_id": 159, "edu_l1_label": "IOS"}, {"txt": "We used the LAMB optimizer [8] for training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.536, 0.819, 0.849, 0.828]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 40, "global_sentence_id": 160, "edu_l1_label": "IOS"}, {"txt": "The final model consists of 43.6 million parameters and was trained on the same dataset used for tokenizers, employing a tok-enizer with a maximum abstraction level of 13,which has a vocabulary size of 20,000 tokens, and a maximum token length of 300 characters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.849, 0.819, 0.911, 0.828], [0.519, 0.837, 0.91, 0.843], [0.519, 0.852, 0.913, 0.858], [0.52, 0.867, 0.911, 0.873], [0.519, 0.882, 0.91, 0.888], [0.519, 0.894, 0.687, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 40, "global_sentence_id": 161, "edu_l1_label": "IOS"}, {"txt": "The context window size was set", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.687, 0.894, 0.91, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 40, "global_sentence_id": 162, "edu_l1_label": "IOS"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.698, 0.061, 0.913, 0.07]]}]}, "tags": ["header"], "label": "O", "web_segment_id": 1, "global_sentence_id": 163, "edu_l1_label": "EDU_O"}, {"txt": "to 512 tokens, with shorter traces padded using the [PAD]token.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.087, 0.096, 0.48, 0.105], [0.087, 0.11, 0.131, 0.119]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 164, "edu_l1_label": "IOS"}, {"txt": "Training was conducted for 300 epochs in the Google Colab environment on an Nvidia Tesla A100 GPU.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.131, 0.11, 0.48, 0.119], [0.087, 0.125, 0.421, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 165, "edu_l1_label": "IOS"}, {"txt": "Algorithm 3 Algorithm for Anomaly Detection", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.087, 0.148, 0.405, 0.158]]}]}, "tags": ["title"], "label": "code_title", "web_segment_id": 3, "global_sentence_id": 166, "edu_l1_label": "EDU_O"}, {"txt": "1:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.092, 0.169, 0.103, 0.174]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 4, "global_sentence_id": 167, "edu_l1_label": "EDU_O"}, {"txt": " function EVALUATETRACES(traces)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.103, 0.169, 0.27, 0.176]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 4, "global_sentence_id": 168, "edu_l1_label": "EDU_O"}, {"txt": "2:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.179, 0.25, 0.188]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 5, "global_sentence_id": 169, "edu_l1_label": "EDU_O"}, {"txt": "   for each trace in traces do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.179, 0.25, 0.188]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 5, "global_sentence_id": 170, "edu_l1_label": "EDU_O"}, {"txt": "3:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.188, 0.395, 0.196]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 171, "edu_l1_label": "EDU_O"}, {"txt": "      tokens←ProcessTracesToSequences(trace,13)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.188, 0.395, 0.196]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 172, "edu_l1_label": "EDU_O"}, {"txt": "4:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.198, 0.374, 0.206]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 7, "global_sentence_id": 173, "edu_l1_label": "EDU_O"}, {"txt": "      (probs,loss)←EvaluateByTokens(tokens)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.198, 0.374, 0.206]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 7, "global_sentence_id": 174, "edu_l1_label": "EDU_O"}, {"txt": "5:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.209, 0.342, 0.217]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 8, "global_sentence_id": 175, "edu_l1_label": "EDU_O"}, {"txt": "      brier←EvaluateTraceBrier(tokens)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.209, 0.342, 0.217]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 8, "global_sentence_id": 176, "edu_l1_label": "EDU_O"}, {"txt": "6:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.218, 0.45, 0.227]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 9, "global_sentence_id": 177, "edu_l1_label": "EDU_O"}, {"txt": "      count_abnormal←CountNonEmpty([probs,loss,brier])", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.218, 0.45, 0.227]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 9, "global_sentence_id": 178, "edu_l1_label": "EDU_O"}, {"txt": "7:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.228, 0.289, 0.237]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 10, "global_sentence_id": 179, "edu_l1_label": "EDU_O"}, {"txt": "      if count_abnormal≥2 then", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.228, 0.289, 0.237]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 10, "global_sentence_id": 180, "edu_l1_label": "EDU_O"}, {"txt": "8:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.239, 0.336, 0.246]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 181, "edu_l1_label": "EDU_O"}, {"txt": "        Print \"Trace \" + trace +\"is abnormal.\"", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.239, 0.336, 0.246]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 182, "edu_l1_label": "EDU_O"}, {"txt": "9:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.249, 0.163, 0.256]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 12, "global_sentence_id": 183, "edu_l1_label": "EDU_O"}, {"txt": "      else", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.091, 0.249, 0.163, 0.256]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 12, "global_sentence_id": 184, "edu_l1_label": "EDU_O"}, {"txt": "10:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.259, 0.326, 0.266]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 13, "global_sentence_id": 185, "edu_l1_label": "EDU_O"}, {"txt": "        Print \"Trace \"+trace +\"is normal.\"", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.259, 0.326, 0.266]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 13, "global_sentence_id": 186, "edu_l1_label": "EDU_O"}, {"txt": "11:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.268, 0.176, 0.276]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 14, "global_sentence_id": 187, "edu_l1_label": "EDU_O"}, {"txt": "      end if", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.268, 0.176, 0.276]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 14, "global_sentence_id": 188, "edu_l1_label": "EDU_O"}, {"txt": "12:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.278, 0.164, 0.286]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 15, "global_sentence_id": 189, "edu_l1_label": "EDU_O"}, {"txt": "   end for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.087, 0.278, 0.164, 0.286]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 15, "global_sentence_id": 190, "edu_l1_label": "EDU_O"}, {"txt": "13:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.088, 0.29, 0.105, 0.295]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 16, "global_sentence_id": 191, "edu_l1_label": "EDU_O"}, {"txt": " end function", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.105, 0.29, 0.176, 0.296]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 16, "global_sentence_id": 192, "edu_l1_label": "EDU_O"}, {"txt": "The anomaly detection algorithm is described in Algo-rithm 3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.105, 0.319, 0.481, 0.327], [0.087, 0.336, 0.145, 0.343]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 193, "edu_l1_label": "IOS"}, {"txt": "It takes a list of traces as input, which are subse-quently tokenized based on Algorithm 2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.145, 0.336, 0.481, 0.342], [0.087, 0.352, 0.362, 0.357]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 194, "edu_l1_label": "IOS"}, {"txt": "Then, it performs an evaluation using two methods: probability-based and losS-based,as well as Brier score evaluation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.362, 0.352, 0.48, 0.358], [0.087, 0.366, 0.484, 0.372], [0.087, 0.379, 0.351, 0.388]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 195, "edu_l1_label": "IOS"}, {"txt": "The probability and loss evaluation is performed by mask-ing each token in a trace with the [MASK] token, applying the SQUEEZEBERT model, and comparing the model output with the observed value.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.104, 0.394, 0.483, 0.402], [0.087, 0.41, 0.48, 0.419], [0.087, 0.424, 0.479, 0.433], [0.087, 0.441, 0.255, 0.448]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 196, "edu_l1_label": "IOS"}, {"txt": "If the probability of the observed token is less than 0.85, it is considered anomalous.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.255, 0.441, 0.479, 0.448], [0.087, 0.456, 0.418, 0.463]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 197, "edu_l1_label": "IOS"}, {"txt": "Similarly,if the loss function value is greater than 0.05, the token is considered anomalous.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.418, 0.456, 0.484, 0.465], [0.088, 0.47, 0.48, 0.479], [0.087, 0.487, 0.241, 0.493]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 18, "global_sentence_id": 198, "edu_l1_label": "IOS"}, {"txt": "The Brier score helps detect anomalies at group token lev-els, increasing the accuracy of detecting incorrect behavior.By masking 20% of randomly selected tokens and calculating the Brier score for them, we determine if the entire trace is anomalous if the score exceeds 0.5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6800000, "bbox": [[0.105, 0.5, 0.483, 0.508], [0.087, 0.517, 0.483, 0.523], [0.087, 0.53, 0.48, 0.54], [0.087, 0.545, 0.48, 0.554], [0.087, 0.562, 0.321, 0.569]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 19, "global_sentence_id": 199, "edu_l1_label": "IOS"}, {"txt": "We also used a SQLite database to store these three eval-uations for each trace.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.105, 0.575, 0.482, 0.581], [0.087, 0.592, 0.243, 0.598]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 200, "edu_l1_label": "IOS"}, {"txt": "If an identical trace appears again,we retrieve the scores from the database instead of rerun-ning the model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.243, 0.592, 0.483, 0.601], [0.087, 0.607, 0.482, 0.614], [0.087, 0.623, 0.191, 0.629]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 201, "edu_l1_label": "IOS"}, {"txt": "This approach saves time and computational resources.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.191, 0.623, 0.48, 0.629], [0.087, 0.638, 0.154, 0.645]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 20, "global_sentence_id": 202, "edu_l1_label": "IOS"}, {"txt": "Model validation was performed on the basis of 25 syn-thetically generated anomalous traces, as well as 25 traces with normal behavior.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.104, 0.651, 0.481, 0.66], [0.087, 0.666, 0.48, 0.675], [0.087, 0.683, 0.239, 0.689]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 203, "edu_l1_label": "IOS"}, {"txt": "Anomalous traces were obtained by adding five random events at random positions, simulating the expected anomalous behavior in the trace.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.239, 0.683, 0.48, 0.691], [0.087, 0.698, 0.48, 0.707], [0.087, 0.711, 0.385, 0.72]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 204, "edu_l1_label": "IOS"}, {"txt": "The validation results are presented in the confusion matrix in Figure 4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.385, 0.711, 0.475, 0.72], [0.087, 0.728, 0.466, 0.735]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 21, "global_sentence_id": 205, "edu_l1_label": "IOS"}, {"txt": "As observed in Figure 4, the developed model shows sat-isfactory results;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.105, 0.741, 0.484, 0.749], [0.09, 0.758, 0.203, 0.766]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 206, "edu_l1_label": "IOS"}, {"txt": " however, it makes errors in 6 cases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.203, 0.758, 0.448, 0.765]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 207, "edu_l1_label": "IOS"}, {"txt": "This error is less problematic because falsely labeling a normal trace as anomalous is preferable to missing an actual anom-aly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.448, 0.758, 0.48, 0.765], [0.087, 0.774, 0.482, 0.78], [0.087, 0.788, 0.482, 0.795], [0.087, 0.803, 0.113, 0.81]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 208, "edu_l1_label": "IOS"}, {"txt": "Increasing the volume of training data could improve the model's quality, so further training on a larger dataset is recommended.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.113, 0.803, 0.48, 0.81], [0.087, 0.818, 0.48, 0.826], [0.087, 0.834, 0.187, 0.841]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 22, "global_sentence_id": 209, "edu_l1_label": "IOS"}, {"txt": "5 Conclusion", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.087, 0.859, 0.217, 0.869]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 23, "global_sentence_id": 210, "edu_l1_label": "BOS"}, {"txt": "In this study, existing NLP approaches applied to the anal-ysis of event logs were examined.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.087, 0.879, 0.482, 0.884], [0.087, 0.896, 0.321, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 24, "global_sentence_id": 211, "edu_l1_label": "IOS"}, {"txt": "We developed a tool in", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.321, 0.896, 0.48, 0.903]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 24, "global_sentence_id": 212, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7400000, "bbox": [[0.6, 0.092, 0.832, 0.243]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 25, "global_sentence_id": 213, "edu_l1_label": "EDU_O"}, {"txt": "Figure 4. Confusion matrix of model performance", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.545, 0.259, 0.885, 0.271]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 26, "global_sentence_id": 214, "edu_l1_label": "EDU_O"}, {"txt": "PYTHON, which is designed for analyzing patterns and anom-alies in logs of .NET CLR applications.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.519, 0.295, 0.913, 0.304], [0.519, 0.312, 0.786, 0.319]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 27, "global_sentence_id": 215, "edu_l1_label": "IOS"}, {"txt": "Our tool supports multiple levels of abstraction for pattern detection and uti-lizes an SQLITE database to store and reference previously analyzed traces, optimizing performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.786, 0.312, 0.911, 0.319], [0.519, 0.328, 0.913, 0.331], [0.519, 0.341, 0.91, 0.351], [0.519, 0.359, 0.806, 0.364]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 27, "global_sentence_id": 216, "edu_l1_label": "IOS"}, {"txt": "A model based on the BERT architecture, specifically SQUEEZEBERT, was trained from scratch.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.806, 0.359, 0.91, 0.364], [0.519, 0.373, 0.911, 0.379], [0.519, 0.388, 0.657, 0.395]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 27, "global_sentence_id": 217, "edu_l1_label": "IOS"}, {"txt": "Validation results demonstrate that the model performs well in anomaly detection, and it is expected that increasing the volume of data can improve its quality.Thus,we have shown that NLP approaches can be effectively applied to the analysis of event logs in the .NET CLR runtime environment.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.657, 0.388, 0.91, 0.395], [0.519, 0.404, 0.91, 0.41], [0.519, 0.417, 0.914, 0.425], [0.518, 0.432, 0.911, 0.443], [0.519, 0.45, 0.91, 0.456], [0.519, 0.464, 0.609, 0.47]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 27, "global_sentence_id": 218, "edu_l1_label": "IOS"}, {"txt": "References", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.519, 0.489, 0.617, 0.499]]}]}, "tags": ["title"], "label": "reference", "web_segment_id": 28, "global_sentence_id": 219, "edu_l1_label": "EDU_O"}, {"txt": "[1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.2018.Bert:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.519, 0.508, 0.916, 0.516], [0.542, 0.521, 0.6, 0.528]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 29, "global_sentence_id": 220, "edu_l1_label": "EDU_O"}, {"txt": " Pre-training of deep bidirectional transformers for language understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.6, 0.521, 0.908, 0.53], [0.543, 0.535, 0.624, 0.54]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 29, "global_sentence_id": 221, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1810.04805 (2018).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.624, 0.535, 0.83, 0.541]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 29, "global_sentence_id": 222, "edu_l1_label": "EDU_O"}, {"txt": "[2] Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.52, 0.546, 0.833, 0.553]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 30, "global_sentence_id": 223, "edu_l1_label": "EDU_O"}, {"txt": "2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.833, 0.546, 0.863, 0.554]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 30, "global_sentence_id": 224, "edu_l1_label": "EDU_O"}, {"txt": "Deeplog:Anomaly detection and diagnosis from system logs through deep learn-ing.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.863, 0.546, 0.913, 0.553], [0.542, 0.56, 0.913, 0.564], [0.543, 0.573, 0.564, 0.579]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 30, "global_sentence_id": 225, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security.1285-1298.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.564, 0.573, 0.911, 0.579], [0.543, 0.586, 0.737, 0.592]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 30, "global_sentence_id": 226, "edu_l1_label": "EDU_O"}, {"txt": "[3] Shaohan Huang,Yi Liu, Carol Fung, Rong He,Yining Zhao,Hailong Yang,and Zhongzhi Luan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.52, 0.597, 0.911, 0.605], [0.542, 0.61, 0.678, 0.616]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 31, "global_sentence_id": 227, "edu_l1_label": "EDU_O"}, {"txt": "2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.678, 0.61, 0.708, 0.616]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 31, "global_sentence_id": 228, "edu_l1_label": "EDU_O"}, {"txt": "Hitanomaly: Hierarchical transformers for anomaly detection in system log.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.708, 0.61, 0.911, 0.617], [0.542, 0.622, 0.737, 0.628]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 31, "global_sentence_id": 229, "edu_l1_label": "EDU_O"}, {"txt": "IEEE transactions on network and service management 17,4 (2020),2064-2076.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.737, 0.622, 0.91, 0.629], [0.543, 0.636, 0.777, 0.641]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 31, "global_sentence_id": 230, "edu_l1_label": "EDU_O"}, {"txt": "[4] Forrest N Iandola, Albert E Shaw, Ravi Krishna, and Kurt W Keutzer.2020.SqueezeBERT:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.518, 0.646, 0.914, 0.654], [0.542, 0.66, 0.655, 0.666]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 32, "global_sentence_id": 231, "edu_l1_label": "EDU_O"}, {"txt": "What can computer vision teach NLP about effi-cient neural networks?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.655, 0.66, 0.913, 0.664], [0.543, 0.674, 0.664, 0.679]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 32, "global_sentence_id": 232, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2006.11316 (2020).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.664, 0.674, 0.872, 0.679]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 32, "global_sentence_id": 233, "edu_l1_label": "EDU_O"}, {"txt": "[5] Yukyung Lee, Jina Kim, and Pilsung Kang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.518, 0.685, 0.767, 0.691]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 33, "global_sentence_id": 234, "edu_l1_label": "EDU_O"}, {"txt": "2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.767, 0.685, 0.797, 0.691]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 33, "global_sentence_id": 235, "edu_l1_label": "EDU_O"}, {"txt": "Lanobert: System log anomaly detection based on bert masked language model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.797, 0.685, 0.91, 0.693], [0.542, 0.699, 0.846, 0.704]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 33, "global_sentence_id": 236, "edu_l1_label": "EDU_O"}, {"txt": "Applied Soft Computing 146 (2023),110689.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.846, 0.699, 0.911, 0.704], [0.542, 0.71, 0.704, 0.717]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 33, "global_sentence_id": 237, "edu_l1_label": "EDU_O"}, {"txt": "[6] Evgenii V Stepanov and Alexey A Mitsyuk.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.519, 0.722, 0.77, 0.73]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 34, "global_sentence_id": 238, "edu_l1_label": "EDU_O"}, {"txt": "2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.77, 0.722, 0.8, 0.73]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 34, "global_sentence_id": 239, "edu_l1_label": "EDU_O"}, {"txt": "Extracting high-level activities from low-level program execution logs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.8, 0.722, 0.913, 0.73], [0.542, 0.737, 0.803, 0.742]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 34, "global_sentence_id": 240, "edu_l1_label": "EDU_O"}, {"txt": "Automated Software Engineering 31,2(2024),41.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.803, 0.737, 0.91, 0.742], [0.543, 0.748, 0.69, 0.755]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 34, "global_sentence_id": 241, "edu_l1_label": "EDU_O"}, {"txt": "[7] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.52, 0.759, 0.91, 0.768], [0.542, 0.773, 0.848, 0.78]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 35, "global_sentence_id": 242, "edu_l1_label": "EDU_O"}, {"txt": "2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.848, 0.773, 0.877, 0.78]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 35, "global_sentence_id": 243, "edu_l1_label": "EDU_O"}, {"txt": "Atten-tion is all you need.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.877, 0.773, 0.913, 0.778], [0.542, 0.787, 0.648, 0.792]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 35, "global_sentence_id": 244, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems 30(2017).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.648, 0.787, 0.91, 0.792], [0.544, 0.799, 0.593, 0.805]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 35, "global_sentence_id": 245, "edu_l1_label": "EDU_O"}, {"txt": "[8] Yang You,Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.519, 0.809, 0.91, 0.817], [0.543, 0.823, 0.913, 0.83], [0.55, 0.836, 0.575, 0.842]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 36, "global_sentence_id": 246, "edu_l1_label": "EDU_O"}, {"txt": "2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.575, 0.836, 0.609, 0.843]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 36, "global_sentence_id": 247, "edu_l1_label": "EDU_O"}, {"txt": "Large batch optimization for deep learning: Training bert in 76 minutes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.609, 0.836, 0.911, 0.843], [0.543, 0.849, 0.62, 0.855]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 36, "global_sentence_id": 248, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1904.00962 (2019).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.62, 0.849, 0.825, 0.855]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 36, "global_sentence_id": 249, "edu_l1_label": "EDU_O"}], "type": "PDF"}
# NLP-Based.NET CLR Event Logs Analyzer
## Abstract
## 1 Introduction
## 2 Related Work
## 3 Algorithms
### 3.1 Event Log Encoding
### 3.2 Neural Network Configuration
## 4 Method
### 4.1 Patterns Detection
### 4.2 Anomalies Detection
## 5 Conclusion
