{"entry_id": "5a8ca540-cbe7-4fda-857b-333934801631", "infos": [{"txt": "MITIGATING UNINTENDED MEMORIZATION WITH LORA IN FEDERATED LEARNING FOR LLMS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.202, 0.12, 0.798, 0.145], [0.231, 0.145, 0.769, 0.17]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 7, "global_sentence_id": 0, "edu_l1_label": "BOT"}, {"txt": "50", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 100000, "bbox": [[0.029, 0.268, 0.055, 0.283]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 15, "global_sentence_id": 1, "edu_l1_label": "EDU_O"}, {"txt": "ThierryBossy**", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 200000, "bbox": [[0.176, 0.228, 0.293, 0.252]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 19, "global_sentence_id": 2, "edu_l1_label": "EDU_O"}, {"txt": "JulienTuánTúVignoud**", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.363, 0.23, 0.551, 0.248]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 18, "global_sentence_id": 3, "edu_l1_label": "EDU_O"}, {"txt": "julien.vignoud@epfl.ch", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 400000, "bbox": [[0.362, 0.246, 0.55, 0.261]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 8, "global_sentence_id": 4, "edu_l1_label": "EDU_O"}, {"txt": "thierry@tuneinsight.com", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.136, 0.247, 0.336, 0.262]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 16, "global_sentence_id": 5, "edu_l1_label": "EDU_O"}, {"txt": "TahseenRabbani3-JuanR.Troncosof", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 600000, "bbox": [[0.551, 0.23, 0.864, 0.248]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 17, "global_sentence_id": 6, "edu_l1_label": "EDU_O"}, {"txt": "2021", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.026, 0.277, 0.056, 0.31]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 14, "global_sentence_id": 7, "edu_l1_label": "EDU_O"}, {"txt": "20", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.029, 0.305, 0.054, 0.32]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 13, "global_sentence_id": 8, "edu_l1_label": "EDU_O"}, {"txt": "①", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.029, 0.335, 0.056, 0.352]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 11, "global_sentence_id": 9, "edu_l1_label": "EDU_O"}, {"txt": "C", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.03, 0.324, 0.054, 0.339]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 12, "global_sentence_id": 10, "edu_l1_label": "EDU_O"}, {"txt": "②", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.024, 0.344, 0.058, 0.368]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 10, "global_sentence_id": 11, "edu_l1_label": "EDU_O"}, {"txt": "Martin Jaggi‡", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.451, 0.28, 0.548, 0.29]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 9, "global_sentence_id": 12, "edu_l1_label": "EDU_O"}, {"txt": "ABSTRACT", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.453, 0.343, 0.547, 0.362]]}]}, "tags": ["title"], "label": "abstract", "web_segment_id": 5, "global_sentence_id": 13, "edu_l1_label": "BOS"}, {"txt": "Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.176, 0.367, 0.824, 0.382], [0.176, 0.381, 0.35, 0.396]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 14, "edu_l1_label": "IOS"}, {"txt": "However, data privacy issues still remain: FL-trained large language models are capable of memorizing and completing phrases and sentences contained in training data when given with their prefxes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.35, 0.381, 0.824, 0.396], [0.176, 0.394, 0.824, 0.409], [0.176, 0.408, 0.383, 0.423]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 15, "edu_l1_label": "IOS"}, {"txt": "Thus, it is possible for adversarial and honest-but-curious clients to recover training data of other participants simply through targeted prompting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.383, 0.408, 0.824, 0.423], [0.176, 0.422, 0.71, 0.437]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 16, "edu_l1_label": "IOS"}, {"txt": "In this work, we demonstrate that a popular and simple fne-tuning strategy, low-rank adaptation (LoRA), reduces memorization during FL up to a factor of 10.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.71, 0.422, 0.824, 0.437], [0.176, 0.436, 0.824, 0.451], [0.176, 0.449, 0.466, 0.465]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 17, "edu_l1_label": "IOS"}, {"txt": "We study this effect by performing a medical questionanswering fne-tuning task and injecting multiple replicas of out-of-distribution sensitive sequences drawn from an external clinical dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.466, 0.449, 0.826, 0.465], [0.176, 0.463, 0.824, 0.478], [0.176, 0.477, 0.428, 0.492]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 18, "edu_l1_label": "IOS"}, {"txt": "We observe a reduction in memorization for a wide variety of Llama 2 and 3 models, and fnd that LoRA can reduce memorization in centralized learning as well.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.428, 0.477, 0.824, 0.492], [0.176, 0.491, 0.827, 0.506]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 19, "edu_l1_label": "IOS"}, {"txt": "Furthermore, we show that LoRA can be combined with other privacy-preserving techniques such as gradient clipping and Gaussian noising, secure aggregation, and Goldfsh loss to further improve record-level privacy while maintaining performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.176, 0.505, 0.824, 0.52], [0.176, 0.518, 0.824, 0.533], [0.176, 0.532, 0.519, 0.547]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 20, "edu_l1_label": "IOS"}, {"txt": "1Introduction", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.118, 0.566, 0.253, 0.585]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 4, "global_sentence_id": 21, "edu_l1_label": "BOS"}, {"txt": "Large language models (LLMs) have been shown to achieve state-of-the-art performance over most relevant natural language processing (NLP) tasks [Zhao et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.118, 0.598, 0.882, 0.613], [0.118, 0.612, 0.457, 0.627]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 22, "edu_l1_label": "IOS"}, {"txt": "There is an emerging and signifcant interest in fne-tuning LLMs to conduct tasks over specialized domains such as medicine [Thirunavukarasu et al., 2023, Yang et al., 2022] and fnance [Wu et al., 2023a, Li et al., 2023a].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.457, 0.612, 0.882, 0.627], [0.118, 0.626, 0.882, 0.641], [0.118, 0.639, 0.397, 0.655]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 23, "edu_l1_label": "IOS"}, {"txt": "These felds handle inherently sensitive user data, necessitating additional mechanisms to prevent data exposure.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.397, 0.639, 0.882, 0.655], [0.118, 0.653, 0.367, 0.668]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 24, "edu_l1_label": "IOS"}, {"txt": "A well-studied paradigm for collaboratively training a machine learning (ML) model over a cluster of clients without sharing local data is federated learning (FL) [McMahan et al., 2016, Kairouz et al., 2021].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.367, 0.653, 0.883, 0.668], [0.118, 0.667, 0.882, 0.682], [0.118, 0.681, 0.199, 0.696]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 25, "edu_l1_label": "IOS"}, {"txt": "Although FL respects data sovereignty by allowing training samples to remain decentralized, most FL works do not address the memorization problem:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.117, 0.702, 0.882, 0.717], [0.118, 0.715, 0.346, 0.73]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 26, "edu_l1_label": "IOS"}, {"txt": " an FL-trained LLM may still memorize client training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.346, 0.715, 0.736, 0.73]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 27, "edu_l1_label": "IOS"}, {"txt": "Indeed, memorization is observable in most, if not all, LLMs [Carlini et al., 2019, 2022, 2021], with some work arguing that memorization is required to learn natural speech patterns [Dourish, 2004, Feldman, 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.736, 0.715, 0.882, 0.73], [0.118, 0.729, 0.882, 0.744], [0.118, 0.743, 0.597, 0.758]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 28, "edu_l1_label": "IOS"}, {"txt": "While there is a wealth of research focused on preventing data reconstruction [Huang et al., 2021] and improving differential privacy [El Ouadrhiri and Abdelhadi, 2022] within the FL literature, very few have explored the propensity and prevention of FL-trained LLMs to leak training data [Thakkar et al., 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.597, 0.743, 0.882, 0.758], [0.118, 0.757, 0.884, 0.772], [0.118, 0.77, 0.883, 0.786], [0.118, 0.784, 0.349, 0.799]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 29, "edu_l1_label": "IOS"}, {"txt": "In this work, we demonstrate an intuitive and effcient strategy for reducing memorization during LLM fne-tuning:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.114, 0.804, 0.888, 0.848]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 30, "edu_l1_label": "IOS"}, {"txt": " low-rank adaptation (LoRA) [Hu et al., 2021].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.114, 0.804, 0.888, 0.848]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 31, "edu_l1_label": "IOS"}, {"txt": "In fact, we observe that LoRA fne-tuning mitigates regurgitation of synthetically-injected sensitive data in both the federated and centralized settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.114, 0.804, 0.888, 0.848]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 32, "edu_l1_label": "IOS"}, {"txt": "This includes exact token matching", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.114, 0.804, 0.888, 0.848]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 33, "edu_l1_label": "IOS"}, {"txt": "∗Equal contribution†Tune Insight SA, Switzerland‡EPFL, Switzerland§Yale University, USA", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.137, 0.857, 0.253, 0.871], [0.138, 0.871, 0.318, 0.885], [0.138, 0.885, 0.256, 0.898], [0.138, 0.898, 0.271, 0.912]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 6, "global_sentence_id": 34, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.268, 0.045, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 35, "edu_l1_label": "EDU_O"}, {"txt": "[Carlini et al., 2022] and approximate reproduction [Ippolito et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.118, 0.092, 0.597, 0.107]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 36, "edu_l1_label": "IOS"}, {"txt": "As LoRA combines the benefts of reduced computational [Hu et al., 2021], memory [Dettmers et al., 2024], and communication overhead [Liu et al., 2024], its added beneft of preventing memorization makes it an ideal strategy for FL fne-tuning of LLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.597, 0.092, 0.882, 0.107], [0.118, 0.106, 0.882, 0.121], [0.118, 0.119, 0.747, 0.135]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 37, "edu_l1_label": "IOS"}, {"txt": "Our contributions are as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.118, 0.14, 0.331, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 38, "edu_l1_label": "IOS"}, {"txt": "• We discover and demonstrate that LoRA mitigates memorization in federated and centralized learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.162, 0.166, 0.848, 0.181]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 39, "edu_l1_label": "IOS"}, {"txt": "Thisincludes exact match rate (repeating training data exactly) and paraphrasing (partial overlap).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.848, 0.166, 0.882, 0.181], [0.176, 0.18, 0.791, 0.195]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 40, "edu_l1_label": "IOS"}, {"txt": "Compared to full fne-tuning, LoRA can signifcantly reduce memorization even when sensitive data is replicated and the LLM is prompted with long prefxes of a sequence.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.791, 0.18, 0.882, 0.195], [0.176, 0.194, 0.882, 0.209], [0.176, 0.208, 0.51, 0.223]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 41, "edu_l1_label": "IOS"}, {"txt": "• We comprehensively test models of varying size from the Llama-2 family, Llama-3 family, and Mistral-v0.3 on medical question-answering tasks to simulate a data-sensitive scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.162, 0.227, 0.885, 0.242], [0.176, 0.241, 0.705, 0.256]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 42, "edu_l1_label": "IOS"}, {"txt": "LoRA effectively reduces memorization while preserving high performance accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.705, 0.241, 0.882, 0.256], [0.176, 0.255, 0.564, 0.27]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 43, "edu_l1_label": "IOS"}, {"txt": "• We experimentally explore how LoRA interacts with other privacy strategies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.162, 0.274, 0.667, 0.289]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 44, "edu_l1_label": "IOS"}, {"txt": "This includes differential privacymechanisms such as gradient noising and clipping, Goldfsh loss [Hans et al., 2024], and post-training noise injection.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.667, 0.274, 0.883, 0.289], [0.176, 0.288, 0.882, 0.303], [0.176, 0.302, 0.237, 0.317]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 45, "edu_l1_label": "IOS"}, {"txt": "We fnd that LoRA works synergistically with these other approaches.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.237, 0.302, 0.698, 0.317]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 46, "edu_l1_label": "IOS"}, {"txt": "• To facilitate reproducibility and further research, we publicly release our code and instructions at https://github.com/tuneinsight/federated-llms.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.162, 0.321, 0.887, 0.337], [0.175, 0.336, 0.514, 0.35]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 47, "edu_l1_label": "IOS"}, {"txt": "2Related Work", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.118, 0.369, 0.263, 0.388]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 10, "global_sentence_id": 48, "edu_l1_label": "BOS"}, {"txt": "2.1Privacy in LLMs", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.118, 0.401, 0.272, 0.417]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 9, "global_sentence_id": 49, "edu_l1_label": "IOS"}, {"txt": "Exposure of sensitive data via generative models has been extensively considered in existing literature, though the choice of the privacy evaluation metric continues to evolve.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.118, 0.427, 0.882, 0.442], [0.118, 0.441, 0.504, 0.456]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 50, "edu_l1_label": "IOS"}, {"txt": "Differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.118, 0.461, 0.259, 0.477]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 51, "edu_l1_label": "IOS"}, {"txt": "Classical (ϵ, δ)-differential privacy (DP) frameworks formally measure the privacy-preservingcapacity of an algorithm by analyzing whether the probability of observing an output changes by ϵ when the underlyingdatabase excludes or includes a user record [Dwork et al., 2006].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.259, 0.461, 0.882, 0.477], [0.118, 0.475, 0.882, 0.49], [0.118, 0.489, 0.555, 0.504]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 52, "edu_l1_label": "IOS"}, {"txt": "The application of this framework to generative language tasks, in general, has proven complicated due to the rigid defnition of a user record [Jayaraman and Evans, 2019].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.555, 0.489, 0.882, 0.504], [0.118, 0.503, 0.884, 0.518], [0.118, 0.516, 0.161, 0.532]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 53, "edu_l1_label": "IOS"}, {"txt": "When directly applying DP to prevent sensitive data reconstruction, it has been shown that a non-negligible compromise on privacy is required to maintain performance [Lukas et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.161, 0.516, 0.882, 0.532], [0.118, 0.53, 0.635, 0.545]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 54, "edu_l1_label": "IOS"}, {"txt": "The conventional technique of adding Gaussian noise onto clipped gradients [Abadi et al., 2016] to boost privacy has also been shown to affect model outputs: the randomness of the noise alone can signifcantly alter the outputs of two equally-private models [Kulynych et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.635, 0.53, 0.882, 0.545], [0.118, 0.544, 0.885, 0.559], [0.118, 0.558, 0.884, 0.573], [0.118, 0.572, 0.161, 0.587]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 55, "edu_l1_label": "IOS"}, {"txt": "One must consider the context and length of a prompt that goads an LLM into leaking sensitive information [Nissenbaum, 2004, Dourish, 2004] – a condition absent from the DP perspective [Brown et al., 2022].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.161, 0.572, 0.882, 0.587], [0.118, 0.585, 0.787, 0.601]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 56, "edu_l1_label": "IOS"}, {"txt": "Memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.118, 0.606, 0.222, 0.622]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 57, "edu_l1_label": "IOS"}, {"txt": "The ability of language models (large or otherwise) to regurgitate pieces of their training data is well-documented.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.222, 0.606, 0.882, 0.621], [0.117, 0.62, 0.236, 0.635]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 58, "edu_l1_label": "IOS"}, {"txt": "However, the question of how best to quantify the memorization capacity of an LLM is an active area of research.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.236, 0.62, 0.882, 0.635], [0.118, 0.634, 0.229, 0.649]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 59, "edu_l1_label": "IOS"}, {"txt": "A seminal work by Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.229, 0.634, 0.457, 0.649]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 60, "edu_l1_label": "IOS"}, {"txt": "introduced “canaries\", which are synthetic, out-of-distributionpieces of text injected into training data (such as \"My SSN is XXX-XX-XXXX\") [Carlini et al., 2019].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.457, 0.634, 0.882, 0.649], [0.118, 0.647, 0.787, 0.663]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 61, "edu_l1_label": "IOS"}, {"txt": "The approach is computationally expensive, as it requires perplexity comparisons against many thousands of random sequences, and canaries should be inserted anywhere from 1 to 10,000 times to gather a full picture of exposure, thus requiring signifcant fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.787, 0.647, 0.882, 0.663], [0.118, 0.661, 0.884, 0.676], [0.118, 0.675, 0.882, 0.69], [0.118, 0.689, 0.266, 0.704]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 62, "edu_l1_label": "IOS"}, {"txt": "However, it has found use in production-level studies [Ramaswamy et al., 2020] and adjacent felds such as machine unlearning [Jagielski et al., 2022].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.266, 0.689, 0.882, 0.704], [0.118, 0.702, 0.482, 0.718]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 63, "edu_l1_label": "IOS"}, {"txt": "An alternative proposal of memorization [Carlini et al., 2022], the completion metric, adopted by our work, measures how often an LLM completes a piece of text taken from the training text when prompted on an initial portion (prefx) of it.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.482, 0.702, 0.884, 0.718], [0.118, 0.716, 0.882, 0.731], [0.118, 0.73, 0.523, 0.745]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 64, "edu_l1_label": "IOS"}, {"txt": "2.2Federated Learning", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.118, 0.761, 0.292, 0.778]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 65, "edu_l1_label": "IOS"}, {"txt": "Privacy in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.118, 0.787, 0.22, 0.803]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 66, "edu_l1_label": "IOS"}, {"txt": "Federated learning, although initially designed to protect user data [McMahan et al., 2017], did not foresee leakage in the form of regurgitation as its advent preceded the development of high-performing generative language models [Kairouz et al., 2021].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.22, 0.787, 0.882, 0.802], [0.118, 0.801, 0.882, 0.816], [0.118, 0.815, 0.384, 0.83]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 67, "edu_l1_label": "IOS"}, {"txt": "Consequently, studies on the memorization capacity of FL-trained LLMs remain limited.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.384, 0.815, 0.882, 0.83], [0.118, 0.829, 0.218, 0.844]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 68, "edu_l1_label": "IOS"}, {"txt": "An early survey demonstrated that federated averaging [Thakkar et al., 2020] ameliorates unintended memorization, though only for a tiny 1.3M parameter next-word predictor [Hard et al., 2018].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.218, 0.829, 0.882, 0.844], [0.118, 0.842, 0.735, 0.858]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 69, "edu_l1_label": "IOS"}, {"txt": "However, the authors’observations on the success of non-independent and identically distributed (non-IID) clustering for improved privacy informed our federated training strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.735, 0.842, 0.885, 0.858], [0.118, 0.856, 0.883, 0.871], [0.118, 0.87, 0.383, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 70, "edu_l1_label": "IOS"}, {"txt": "The addition of the DP Gaussian mechanism was shown to improve canarybased memorization for a production FL setting [Ramaswamy et al., 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.383, 0.87, 0.885, 0.885], [0.118, 0.884, 0.59, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 71, "edu_l1_label": "IOS"}, {"txt": "Similar to us, Liu et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.59, 0.884, 0.738, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 72, "edu_l1_label": "IOS"}, {"txt": "[2024] leverage LoRA to conduct effcient fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.738, 0.884, 0.883, 0.899], [0.118, 0.897, 0.325, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 73, "edu_l1_label": "IOS"}, {"txt": "However, this work is exclusively interested in studying performance under varying", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.325, 0.897, 0.882, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 74, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.493, 0.937, 0.506, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 75, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.268, 0.044, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 76, "edu_l1_label": "EDU_O"}, {"txt": "budgets within the (ϵ, δ)-DP framework and does not consider memorization under the canary or completion-basedframework.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.118, 0.092, 0.882, 0.107], [0.118, 0.106, 0.193, 0.121]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 77, "edu_l1_label": "IOS"}, {"txt": "Medical applications.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.118, 0.126, 0.267, 0.142]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 78, "edu_l1_label": "IOS"}, {"txt": "Our emphasis on medical datasets is relevant: LLMs have been shown to regurgitate sensitive medical data in Lehman et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.267, 0.126, 0.882, 0.141], [0.118, 0.14, 0.312, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 79, "edu_l1_label": "IOS"}, {"txt": "[2021], though their work relies on an older BERT model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.312, 0.14, 0.698, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 80, "edu_l1_label": "IOS"}, {"txt": "Mireshghallah et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.698, 0.14, 0.834, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 81, "edu_l1_label": "IOS"}, {"txt": "[2022] study the success of membership inference attacks on i2b2, though they also do not use any memorization metrics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.834, 0.14, 0.882, 0.155], [0.118, 0.154, 0.885, 0.169]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 82, "edu_l1_label": "IOS"}, {"txt": "Although federated learning has been studied and championed as an ideal paradigm for clinical settings [Xu et al., 2021, Nguyen et al., 2022, Antunes et al., 2022], there is a relative lack of literature in the context of clinical memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.117, 0.168, 0.884, 0.183], [0.118, 0.181, 0.877, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 83, "edu_l1_label": "IOS"}, {"txt": "3Preliminaries", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.118, 0.223, 0.259, 0.243]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 8, "global_sentence_id": 84, "edu_l1_label": "BOS"}, {"txt": "LoRA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.118, 0.26, 0.165, 0.276]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 85, "edu_l1_label": "IOS"}, {"txt": "To reduce computational and memory requirements when fne-tuning LLMs, Low-Rank Adaptation (LoRA) [Hu et al., 2021] was introduced to drastically reduce the number of trainable parameters while fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.165, 0.26, 0.883, 0.275], [0.118, 0.274, 0.832, 0.289]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 86, "edu_l1_label": "IOS"}, {"txt": "This is achieved by representing the weight updates ∆W as the product ∆W = BA of two low-rank matrices A and B. LoRAenables effcient adaptation of LLMs to specifc tasks while preserving the generalization capabilities of the underlying model, as gradients often exhibit a low intrinsic dimension [Li et al., 2018, Aghajanyan et al., 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.832, 0.274, 0.882, 0.289], [0.118, 0.288, 0.883, 0.303], [0.118, 0.301, 0.882, 0.317], [0.118, 0.315, 0.791, 0.33]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 87, "edu_l1_label": "IOS"}, {"txt": "Additionally, LoRA offers a notable advantage in an FL scenario by drastically reducing the amount of data exchanged between participants during each round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.791, 0.315, 0.884, 0.33], [0.118, 0.329, 0.882, 0.344], [0.118, 0.343, 0.32, 0.358]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 88, "edu_l1_label": "IOS"}, {"txt": "In our experiments, we achieved a reduction by a factor of 130.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.32, 0.343, 0.737, 0.358]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 89, "edu_l1_label": "IOS"}, {"txt": "Federated Learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.118, 0.363, 0.259, 0.379]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 90, "edu_l1_label": "IOS"}, {"txt": "Federated learning (FL) has been widely-studied for deep learning models in cross-silo settings Huang et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.259, 0.363, 0.882, 0.379], [0.118, 0.377, 0.194, 0.392]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 91, "edu_l1_label": "IOS"}, {"txt": "[2022], where a limited number of resource-rich clients, such as organizations or institutions, collaboratively train ML models without sharing their data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.194, 0.377, 0.883, 0.392], [0.118, 0.391, 0.398, 0.406]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 92, "edu_l1_label": "IOS"}, {"txt": "In conventional FL, the global objective function of N clients is defned as", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.398, 0.391, 0.882, 0.406]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 93, "edu_l1_label": "IOS"}, {"txt": "\\operatorname* { m i n } _ { W } F ( W ) = \\sum _ { k = 1 } ^ { N } p _ { k } f _ { k } ( W ) ,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.396, 0.418, 0.592, 0.462]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 11, "global_sentence_id": 94, "edu_l1_label": "IOS"}, {"txt": "(1)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.864, 0.434, 0.883, 0.449]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 10, "global_sentence_id": 95, "edu_l1_label": "IOS"}, {"txt": "Local training data Dk between clients often heterogeneous.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.118, 0.494, 0.513, 0.51]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 96, "edu_l1_label": "IOS"}, {"txt": "A common strategy for solving Equation 1 is Federated Averaging (FedAvg) [McMahan et al., 2016].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.513, 0.494, 0.882, 0.51], [0.117, 0.508, 0.418, 0.523]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 97, "edu_l1_label": "IOS"}, {"txt": "In FedAvg, clients conduct a round t of training and θt+1 (parametersafter round t) is updated as the pk-weighted average of the respective k gradients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.418, 0.508, 0.882, 0.523], [0.118, 0.522, 0.657, 0.537]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 98, "edu_l1_label": "IOS"}, {"txt": "These gradient weights pk can be set as pk = PNk|=1 |Dk| Dk| to mitigate data size bias, which we use in this work.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.657, 0.522, 0.882, 0.537], [0.118, 0.539, 0.193, 0.554], [0.2, 0.543, 0.268, 0.56], [0.222, 0.538, 0.243, 0.553], [0.268, 0.539, 0.62, 0.554]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 99, "edu_l1_label": "IOS"}, {"txt": "FL has been recently applied to LLMs Ye et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.62, 0.539, 0.882, 0.554], [0.117, 0.557, 0.171, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 100, "edu_l1_label": "IOS"}, {"txt": "[2024], Thakkar et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.171, 0.557, 0.321, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 101, "edu_l1_label": "IOS"}, {"txt": "[2020], Liu et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.321, 0.557, 0.438, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 102, "edu_l1_label": "IOS"}, {"txt": "[2024], Ramaswamy et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.438, 0.557, 0.617, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 103, "edu_l1_label": "IOS"}, {"txt": "[2020] leveraging FedAvg to aggregate locally-trained model updates.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.617, 0.557, 0.882, 0.572], [0.118, 0.571, 0.319, 0.586]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 104, "edu_l1_label": "IOS"}, {"txt": "In this work, we conduct experiments using LoRA-based fne-tuning and full model fne-tuning for local iterations in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.319, 0.571, 0.882, 0.586], [0.118, 0.585, 0.352, 0.6]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 105, "edu_l1_label": "IOS"}, {"txt": "Besides reducing communication costs, clients beneft computationally from using LoRA during local training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.352, 0.585, 0.882, 0.6], [0.118, 0.598, 0.299, 0.614]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 106, "edu_l1_label": "IOS"}, {"txt": "Memorization Defnition.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.118, 0.619, 0.297, 0.635]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 107, "edu_l1_label": "IOS"}, {"txt": "Following previous work [Ippolito et al., 2023, Huang et al., 2024, Hans et al., 2024], we adopt the \"extractable memorization\" defnition of Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.297, 0.619, 0.884, 0.634], [0.117, 0.633, 0.575, 0.648]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 108, "edu_l1_label": "IOS"}, {"txt": "[2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.575, 0.633, 0.629, 0.648]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 109, "edu_l1_label": "IOS"}, {"txt": "Consider a string representable as a concatenation [p||s] where p is a prefx of length k and s is the remainder of the string.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.629, 0.633, 0.882, 0.648], [0.118, 0.647, 0.694, 0.662]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 110, "edu_l1_label": "IOS"}, {"txt": "We defne the string s to be memorized with k tokens of context by a language model f if [p||s] is contained in the training data of f, and f produces s when prompted with p using greedy decoding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.694, 0.647, 0.882, 0.662], [0.118, 0.66, 0.882, 0.676], [0.118, 0.676, 0.431, 0.689]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 111, "edu_l1_label": "IOS"}, {"txt": "In other words, we consider a string from training data memorized if an LLM can generate it when prompted by a prefx.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.431, 0.676, 0.882, 0.689], [0.118, 0.688, 0.454, 0.703]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 112, "edu_l1_label": "IOS"}, {"txt": "4Empirical Evaluation", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.118, 0.729, 0.326, 0.749]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 9, "global_sentence_id": 113, "edu_l1_label": "BOS"}, {"txt": "In this section, we study how LoRA affects memorization of out-of-distribution sequences injected into fne-tuning training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.118, 0.767, 0.882, 0.782], [0.118, 0.78, 0.206, 0.796]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 114, "edu_l1_label": "IOS"}, {"txt": "We introduce the experimental setting in Section 4.1 and explain how we quantify memorization in Section 4.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.206, 0.78, 0.882, 0.796], [0.118, 0.794, 0.195, 0.809]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 115, "edu_l1_label": "IOS"}, {"txt": "We consider conventional centralized learning in Section 4.3, where all training samples are trained on by a single client.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.117, 0.815, 0.885, 0.83]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 116, "edu_l1_label": "IOS"}, {"txt": "We then consider an FL setting in Section 4.4, where training data is split among several clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.117, 0.829, 0.745, 0.844]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 117, "edu_l1_label": "IOS"}, {"txt": "Our FL experiments are designed to mimic a medical setting where training data contains sensitive information at an unknown rate, which is a common scenario as few if not any data anonymization tools can guarantee a complete removal of sensitive data [Langarizadeh et al., 2018].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.745, 0.829, 0.882, 0.844], [0.118, 0.842, 0.882, 0.858], [0.118, 0.856, 0.882, 0.871], [0.118, 0.87, 0.298, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 118, "edu_l1_label": "IOS"}, {"txt": "In fact, Heider et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.298, 0.87, 0.433, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 119, "edu_l1_label": "IOS"}, {"txt": "[2020] measured the accuracy of three off-the-shelf de-identifcation tools on the i2b2 medical record dataset [Stubbs and Özlem Uzuner, 2015], which our experiments also use, and foundthat no system could perform a full removal.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.433, 0.87, 0.882, 0.885], [0.118, 0.884, 0.882, 0.899], [0.118, 0.897, 0.407, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 120, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.493, 0.936, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 121, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 122, "edu_l1_label": "EDU_O"}, {"txt": "4.1Experimental setup", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.118, 0.091, 0.29, 0.108]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 15, "global_sentence_id": 123, "edu_l1_label": "IOS"}, {"txt": "All fne-tuning was performed on a single NVIDIA A100 80GB GPU within an HPC cluster.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.117, 0.117, 0.725, 0.132]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 124, "edu_l1_label": "IOS"}, {"txt": "We leveraged HuggingFace’s Transformers library [Wolf et al., 2020] to access and fne-tune pre-trained models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.725, 0.117, 0.885, 0.132], [0.118, 0.131, 0.726, 0.146]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 125, "edu_l1_label": "IOS"}, {"txt": "The experiments wereconducted in a Python 3.11.9 environment, with PyTorch 2.4.0 and CUDA 12.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.726, 0.131, 0.882, 0.146], [0.118, 0.145, 0.631, 0.16]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 126, "edu_l1_label": "IOS"}, {"txt": "Further training details are included in Appendix B.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.631, 0.145, 0.882, 0.16], [0.117, 0.158, 0.212, 0.174]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 127, "edu_l1_label": "IOS"}, {"txt": "We fne-tune models for domain adaptation to medical question-answering (QA).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.117, 0.179, 0.661, 0.194]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 128, "edu_l1_label": "IOS"}, {"txt": "Despite medical scenarios being extensively promoted by FL applications [Xu et al., 2021, Nguyen et al., 2022, Antunes et al., 2022], and the availability of resources such as de-anonymized sensitive medical datasets [Johnson et al., 2016, Stubbs and Özlem Uzuner, 2015],clinical memorization remains an area of uncertainty in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.661, 0.179, 0.882, 0.194], [0.118, 0.193, 0.883, 0.208], [0.118, 0.207, 0.884, 0.222], [0.118, 0.22, 0.505, 0.236]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 129, "edu_l1_label": "IOS"}, {"txt": "Fine-tuning Datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.118, 0.241, 0.263, 0.257]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 130, "edu_l1_label": "IOS"}, {"txt": "In order to reproduce a plausible FL environment with non-IID data, we select 3 popular medical datasets with different types of QA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.263, 0.241, 0.882, 0.256], [0.118, 0.255, 0.35, 0.27]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 131, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.156, 0.28, 0.168, 0.296]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 132, "edu_l1_label": "IOS"}, {"txt": "MedMCQA [Pal et al., 2022] is composed of multiple-choice questions, containing almost 190k entrance exam questions (AIIMS & NEET PG).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.168, 0.28, 0.882, 0.296], [0.176, 0.294, 0.43, 0.309]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 133, "edu_l1_label": "IOS"}, {"txt": "We fne-tune on the training split and leave aside validation data as a downstream evaluation benchmark.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.43, 0.294, 0.882, 0.309], [0.176, 0.308, 0.407, 0.323]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 134, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.156, 0.327, 0.168, 0.342]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 135, "edu_l1_label": "IOS"}, {"txt": "PubMedQA [Jin et al., 2019] consists of Yes/No/Maybe questions created from PubMed abstracts.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.168, 0.327, 0.804, 0.342]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 136, "edu_l1_label": "IOS"}, {"txt": "The dataset contains 1k expert-annotated (PQA-L) and 211k artifcially generated QA instances (PQA-A).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.804, 0.327, 0.882, 0.342], [0.176, 0.34, 0.778, 0.356]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 137, "edu_l1_label": "IOS"}, {"txt": "We include 500 questions from the train and validation sets of PQA-L and 50k questions of PQA-A.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.778, 0.34, 0.882, 0.356], [0.176, 0.354, 0.723, 0.369]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 138, "edu_l1_label": "IOS"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.156, 0.373, 0.168, 0.388]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 139, "edu_l1_label": "IOS"}, {"txt": "Medical Meadow fashcards [Han et al., 2023] contains 39k questions created from Anki Medical Curriculum fashcards compiled by medical students.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.168, 0.373, 0.882, 0.388], [0.176, 0.387, 0.443, 0.402]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 140, "edu_l1_label": "IOS"}, {"txt": "We include 10k instances for fne-tuning data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.443, 0.387, 0.748, 0.402]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 141, "edu_l1_label": "IOS"}, {"txt": "Medical Benchmarks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.118, 0.412, 0.273, 0.428]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 142, "edu_l1_label": "IOS"}, {"txt": "To measure the downstream performance of the fne-tuned models, we evaluate models on 4 medical benchmarks following existing methodology [Wu et al., 2023b, Singhal et al., 2023a,b, Chen et al., 2023]: MedQA, PubMedQA, MedMCQA, and MMLU-Medical.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.273, 0.412, 0.882, 0.427], [0.117, 0.426, 0.885, 0.441], [0.118, 0.44, 0.494, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 143, "edu_l1_label": "IOS"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.156, 0.465, 0.168, 0.481]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 144, "edu_l1_label": "IOS"}, {"txt": "MedQA’s 4-option questions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.168, 0.465, 0.371, 0.481]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 145, "edu_l1_label": "IOS"}, {"txt": "MedQA [Jin et al., 2020] consists of US Medical License Exam (USMLE)multiple-choice questions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.371, 0.465, 0.883, 0.481], [0.176, 0.479, 0.345, 0.494]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 146, "edu_l1_label": "IOS"}, {"txt": "The test set contains 1278 questions with both 4 and 5-option questions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.345, 0.479, 0.812, 0.494]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 147, "edu_l1_label": "IOS"}, {"txt": "Following Chen et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.812, 0.479, 0.882, 0.494], [0.176, 0.493, 0.246, 0.508]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 148, "edu_l1_label": "IOS"}, {"txt": "[2023], we report each case separately, respectively MedQA-4 and MedQA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.246, 0.493, 0.745, 0.508]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 149, "edu_l1_label": "IOS"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.156, 0.512, 0.168, 0.527]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 150, "edu_l1_label": "IOS"}, {"txt": "MedQA’s 5-option questions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.168, 0.512, 0.366, 0.527]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 16, "global_sentence_id": 151, "edu_l1_label": "IOS"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.156, 0.53, 0.168, 0.546]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 152, "edu_l1_label": "IOS"}, {"txt": "PubMedQA’s test set contains 500 expert-annotated questions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.168, 0.53, 0.589, 0.546]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 153, "edu_l1_label": "IOS"}, {"txt": "No artifcially-generated questions are usedduring evaluation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.589, 0.53, 0.882, 0.546], [0.176, 0.544, 0.294, 0.559]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 154, "edu_l1_label": "IOS"}, {"txt": "4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.156, 0.563, 0.168, 0.578]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 155, "edu_l1_label": "IOS"}, {"txt": "MedMCQA’s test set does not provide answer labels, therefore we rely on the validation set, containing 4183instances, to benchmark downstream performance following Wu et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.168, 0.563, 0.882, 0.578], [0.176, 0.577, 0.632, 0.592]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 156, "edu_l1_label": "IOS"}, {"txt": "[2023b] and Chen et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.632, 0.577, 0.789, 0.592]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 157, "edu_l1_label": "IOS"}, {"txt": "[2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.789, 0.577, 0.841, 0.592]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 158, "edu_l1_label": "IOS"}, {"txt": "5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.156, 0.595, 0.168, 0.611]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 159, "edu_l1_label": "IOS"}, {"txt": "MMLU-Medical.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.168, 0.595, 0.286, 0.611]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 160, "edu_l1_label": "IOS"}, {"txt": "MMLU [Hendrycks et al., 2021] is a collection of 4-option multiple-choice exam questions covering 57 subjects.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.286, 0.595, 0.882, 0.611], [0.176, 0.609, 0.317, 0.624]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 161, "edu_l1_label": "IOS"}, {"txt": "We follow Chen et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.317, 0.609, 0.469, 0.624]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 162, "edu_l1_label": "IOS"}, {"txt": "[2023] and select a subset of 9 subjects that are most relevant to medical and clinical knowledge: high school biology, college biology, college medicine, professional medicine, medical genetics, virology, clinical knowledge, nutrition, and anatomy, and group them into one medical-related benchmark: MMLU-Medical.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.469, 0.609, 0.882, 0.624], [0.176, 0.623, 0.882, 0.638], [0.176, 0.637, 0.882, 0.652], [0.176, 0.651, 0.477, 0.666]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 163, "edu_l1_label": "IOS"}, {"txt": "We use 3-shot in-context learning without any chain-of-thought reasoning and average the accuracy over 3 seeds.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.117, 0.676, 0.853, 0.691]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 17, "global_sentence_id": 164, "edu_l1_label": "IOS"}, {"txt": "Models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.118, 0.696, 0.172, 0.713]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 165, "edu_l1_label": "IOS"}, {"txt": "To account for the effect of model size on memorization [Carlini et al., 2023, Tirumala et al., 2022], we study pre-trained models ranging from 1B to 8B parameters: Llama 3.2 1B, Llama 3.2 3B, Llama 3 8B [Dubey et al., 2024], Llama 2 7B [Touvron et al., 2023], and Mistral 7B v0.3 [Jiang et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.172, 0.696, 0.883, 0.712], [0.118, 0.711, 0.884, 0.726], [0.118, 0.724, 0.609, 0.739]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 13, "global_sentence_id": 166, "edu_l1_label": "IOS"}, {"txt": "4.2Quantifying memorization", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5400000, "bbox": [[0.118, 0.755, 0.339, 0.771]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 14, "global_sentence_id": 167, "edu_l1_label": "IOS"}, {"txt": "How we measure memorization is largely inspired by Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.118, 0.78, 0.564, 0.796]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 168, "edu_l1_label": "IOS"}, {"txt": "[2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.564, 0.78, 0.618, 0.796]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 169, "edu_l1_label": "IOS"}, {"txt": "In short, we inject sensitive sequences, so-called “canaries\" [Carlini et al., 2019, Jagielski et al., 2023, Thakkar et al., 2020], into fne-tuning data and thenmeasure the models’ ability to regurgitate this information when prompted with the beginning of these sequences.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.618, 0.78, 0.884, 0.796], [0.118, 0.794, 0.882, 0.809], [0.118, 0.808, 0.864, 0.823]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 170, "edu_l1_label": "IOS"}, {"txt": "InAppendix C.2, we give an example of memorization scores for Llama 2 7B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.864, 0.808, 0.882, 0.823], [0.117, 0.822, 0.61, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 171, "edu_l1_label": "IOS"}, {"txt": "Canaries.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.118, 0.842, 0.184, 0.858]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 172, "edu_l1_label": "IOS"}, {"txt": "Unlike prior works that evaluate memorization of all training data [Carlini et al., 2023, Ippolito et al., 2023, Hans et al., 2024], we are interested in measuring how much sensitive information is memorized.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.184, 0.842, 0.884, 0.858], [0.118, 0.856, 0.754, 0.871]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 173, "edu_l1_label": "IOS"}, {"txt": "Similar to Lehman et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.754, 0.856, 0.882, 0.871], [0.118, 0.87, 0.149, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 174, "edu_l1_label": "IOS"}, {"txt": "[2021] and Mireshghallah et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.149, 0.87, 0.358, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 175, "edu_l1_label": "IOS"}, {"txt": "[2022], we inject medical records into our training set originating from the 2014 i2b2/UTHealth corpus dataset [Stubbs and Özlem Uzuner, 2015].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.358, 0.87, 0.882, 0.885], [0.118, 0.884, 0.54, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 176, "edu_l1_label": "IOS"}, {"txt": "The i2b2 dataset contains 1304 longitudinal medicalrecords that describe 296 patients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.54, 0.884, 0.882, 0.899], [0.118, 0.897, 0.34, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 177, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.494, 0.938, 0.507, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 178, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.27, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 179, "edu_l1_label": "EDU_O"}, {"txt": "Since data duplication has been shown to greatly infuence memorization [Carlini et al., 2023, Lee et al., 2022, Kandpal et al., 2022], we randomly select 30% of the medical records and duplicate them 10 times within our fne-tuning data in order to study data duplication in our experiments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.118, 0.092, 0.882, 0.107], [0.118, 0.106, 0.882, 0.121], [0.118, 0.119, 0.447, 0.135]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 180, "edu_l1_label": "IOS"}, {"txt": "Prompting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.118, 0.14, 0.195, 0.156]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 181, "edu_l1_label": "IOS"}, {"txt": "To measure unintended memorization after fne-tuning, we randomly select test sequences from the medical records (one sequence per record) and split each sequence into a prefx p and a suffx s. Conditioned on the prefx, the model generates text via greedy decoding and the generated suffx is compared with the ground truth.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.195, 0.14, 0.882, 0.155], [0.118, 0.154, 0.882, 0.169], [0.118, 0.168, 0.767, 0.183]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 182, "edu_l1_label": "IOS"}, {"txt": "We set the length of the generated suffx s to 50 tokens, in line with Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.767, 0.168, 0.882, 0.183], [0.118, 0.181, 0.526, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 183, "edu_l1_label": "IOS"}, {"txt": "[2023], Ippolito et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.526, 0.181, 0.669, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 184, "edu_l1_label": "IOS"}, {"txt": "[2023] and Hans et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.669, 0.181, 0.817, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 185, "edu_l1_label": "IOS"}, {"txt": "[2024].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.817, 0.181, 0.868, 0.197]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 186, "edu_l1_label": "IOS"}, {"txt": "Following Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.118, 0.202, 0.266, 0.217]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 187, "edu_l1_label": "IOS"}, {"txt": "[2023], we measure the effect of the context size by prompting the model on each test sequence several times with prompts of lengths in {10, 50, 100, 200, 500}.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.266, 0.202, 0.882, 0.217], [0.118, 0.216, 0.551, 0.231]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 188, "edu_l1_label": "IOS"}, {"txt": "The different prompts for one test sequence are constructed such that the suffx s is kept identical while varying the prompt length.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.551, 0.216, 0.882, 0.231], [0.118, 0.23, 0.671, 0.245]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 189, "edu_l1_label": "IOS"}, {"txt": "This ensures a fair comparison between prompt lengths, since different suffxes may be more or less diffcult to regurgitate.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.671, 0.23, 0.882, 0.245], [0.118, 0.243, 0.715, 0.259]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 190, "edu_l1_label": "IOS"}, {"txt": "Memorization scores.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.118, 0.264, 0.267, 0.28]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 191, "edu_l1_label": "IOS"}, {"txt": "To compare generated text with the ground truth, we rely on two metrics: (1) the exact token match rate and (2) the BLEU score to measure approximate reproduction, as prior works suggest that the exact match rate does not capture subtler forms of memorization [Ippolito et al., 2023].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.267, 0.264, 0.882, 0.28], [0.118, 0.277, 0.882, 0.293], [0.118, 0.292, 0.624, 0.307]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 192, "edu_l1_label": "IOS"}, {"txt": "In line with this work, we consider a sequence memorized if the generated suffx and the ground truth yields a BLEU score > 0.75.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.624, 0.292, 0.882, 0.307], [0.118, 0.305, 0.727, 0.321]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 193, "edu_l1_label": "IOS"}, {"txt": "For both metrics, lower is better and a score of 1 denotes the complete memorization of all test sequences.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.727, 0.305, 0.883, 0.321], [0.118, 0.319, 0.663, 0.334]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 194, "edu_l1_label": "IOS"}, {"txt": "In Appendix C.2, we provide an example for Llama 2 7B fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.663, 0.319, 0.882, 0.334], [0.118, 0.333, 0.356, 0.348]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 195, "edu_l1_label": "IOS"}, {"txt": "4.3Centralized Learning", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.118, 0.371, 0.304, 0.388]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 10, "global_sentence_id": 196, "edu_l1_label": "IOS"}, {"txt": "To the best of our knowledge, the impact of LoRA on memorization has not been previously quantifed;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.117, 0.4, 0.792, 0.415]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 197, "edu_l1_label": "IOS"}, {"txt": " therefore, we begin by studying LoRA in the context of centralized learning (CL) before considering federated learning (FL).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.792, 0.4, 0.882, 0.415], [0.118, 0.414, 0.843, 0.429]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 198, "edu_l1_label": "IOS"}, {"txt": "Training details.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.117, 0.434, 0.235, 0.45]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 199, "edu_l1_label": "IOS"}, {"txt": "In the centralized learning setting, we merge PubMedQA, MedMCQA and Medical Meadow Flashcards into one fne-tuning dataset in which we inject the i2b2 medical records to benchmark memorization after fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.235, 0.434, 0.882, 0.45], [0.117, 0.448, 0.883, 0.463], [0.118, 0.462, 0.195, 0.477]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 200, "edu_l1_label": "IOS"}, {"txt": "We use a validation split of 10% and for each model we search for the learning rate yielding the lowest validation loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.195, 0.462, 0.882, 0.477], [0.117, 0.476, 0.215, 0.491]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 201, "edu_l1_label": "IOS"}, {"txt": "More details on hyperparameters can be found in Appendix B.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.215, 0.476, 0.639, 0.491]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 202, "edu_l1_label": "IOS"}, {"txt": "Accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.117, 0.496, 0.186, 0.512]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 203, "edu_l1_label": "IOS"}, {"txt": "To study how LoRA mitigates unintended memorization, we must frst assess if it comes at a cost in model performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.186, 0.496, 0.882, 0.512], [0.118, 0.51, 0.204, 0.525]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 204, "edu_l1_label": "IOS"}, {"txt": "Figure 1 illustrates the average accuracy over fne-tuning strategies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.204, 0.51, 0.651, 0.525]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 205, "edu_l1_label": "IOS"}, {"txt": "Comparing full fne-tuning against LoRA, we fnd that LoRA comes with a relatively negligible cost in accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.651, 0.51, 0.882, 0.525], [0.118, 0.524, 0.633, 0.539]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 206, "edu_l1_label": "IOS"}, {"txt": "Every fne-tuning yields a signifcant accuracy improvement of the pre-trained model except for Llama 3.1 8B, in which performance minimally improved.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.633, 0.524, 0.882, 0.539], [0.118, 0.538, 0.882, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 207, "edu_l1_label": "IOS"}, {"txt": "We hypothesize that part or all of our fne-tuning dataset has already been trained on during Llama 3.1 8B’s pre-trainingphase.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.117, 0.559, 0.882, 0.574], [0.118, 0.572, 0.159, 0.587]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 208, "edu_l1_label": "IOS"}, {"txt": "Accordingly, we exclude Llama 3.1 8B from subsequent experiments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.159, 0.572, 0.618, 0.587]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 209, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.311, 0.602, 0.695, 0.783]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 5, "global_sentence_id": 210, "edu_l1_label": "EDU_O"}, {"txt": "Figure 1: Downstream accuracy of centralized learning averaged across the 5 benchmarks. LoRA matches full fne-tuning accuracy on every model tested. We report the out-of-the-box accuracy of the pre-trained models as a control. A breakdown per benchmark is included in Appendix C.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6800000, "bbox": [[0.115, 0.793, 0.886, 0.838]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 6, "global_sentence_id": 211, "edu_l1_label": "EDU_O"}, {"txt": "Memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.118, 0.856, 0.222, 0.872]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 212, "edu_l1_label": "IOS"}, {"txt": "Given that LoRA matches full fne-tuning performance in our experiments, we now measure the unintended memorization occurring during fne-tuning, illustrated in Figure 2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.222, 0.856, 0.882, 0.871], [0.118, 0.87, 0.614, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 213, "edu_l1_label": "IOS"}, {"txt": "To account for prompt length, we include a fgure (plots (c) and (f)) for each metric with the highest memorization score obtained across settings, which is systematically reached on duplicated documents with the longest prompt.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.614, 0.87, 0.882, 0.885], [0.118, 0.884, 0.882, 0.899], [0.118, 0.897, 0.596, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 214, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.494, 0.937, 0.506, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 215, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.268, 0.045, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 216, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.3, 0.104, 0.703, 0.324]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 6, "global_sentence_id": 217, "edu_l1_label": "EDU_O"}, {"txt": "Figure 2: LoRA vs full fne-tuning memorization scores in centralized learning. LoRA consistently yields lower memorization scores (lower is better). Unless stated otherwise, scores are averaged across prompt lengths. Values are shown when bars are too small. Right-most fgures denote the worst-case setting where memorization scores are the highest. Plots (a)-(c) show memorization using exact match rate with no duplication, 10x document duplication, and 10x document duplication with a 500 tokens prompt length, while (d)-(f) use BLEU score.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.115, 0.331, 0.884, 0.405]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 0, "global_sentence_id": 218, "edu_l1_label": "EDU_O"}, {"txt": "Analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.117, 0.434, 0.18, 0.451]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 219, "edu_l1_label": "IOS"}, {"txt": "Across all model sizes, data duplication greatly increases memorization and longer prompt lengths increase the extraction success.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.18, 0.434, 0.882, 0.45], [0.118, 0.449, 0.263, 0.464]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 220, "edu_l1_label": "IOS"}, {"txt": "Figure 2 also illustrates that larger models memorize more [Carlini et al., 2023, Tirumala et al., 2022].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.263, 0.449, 0.884, 0.464], [0.118, 0.462, 0.16, 0.478]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 221, "edu_l1_label": "IOS"}, {"txt": "Most importantly, we see that models fne-tuned in centralized learning with LoRA consistently exhibit lower memorization scores, suggesting the adequacy of using of LoRA as a memorization-mitigating technique with little to no performance cost.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.16, 0.462, 0.882, 0.477], [0.118, 0.476, 0.882, 0.491], [0.118, 0.49, 0.255, 0.505]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 222, "edu_l1_label": "IOS"}, {"txt": "Additionally, we compute the memorization scores of pre-trained models without fne-tuning, to obtain control values.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.117, 0.511, 0.885, 0.526]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 223, "edu_l1_label": "IOS"}, {"txt": "This is equivalent to computing the models’ ability to “guess\" the suffx without having seen previously the medicalrecords.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.117, 0.524, 0.882, 0.54], [0.118, 0.538, 0.169, 0.553]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 224, "edu_l1_label": "IOS"}, {"txt": "We obtained scores an order of magnitude lower than any fne-tuned model score, which additionally confrms that none of the models had already been trained on the i2b2 dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.169, 0.538, 0.882, 0.553], [0.118, 0.552, 0.563, 0.567]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 225, "edu_l1_label": "IOS"}, {"txt": "Thus, while some scores in Figure 2 may appear low at frst glance, the lowest memorization depicted in this fgure is >10 times higher than the control.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.563, 0.552, 0.883, 0.567], [0.118, 0.566, 0.791, 0.581]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 226, "edu_l1_label": "IOS"}, {"txt": "4.3.1Utility-privacy tradeoff", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7400000, "bbox": [[0.118, 0.603, 0.328, 0.619]]}]}, "tags": ["title"], "label": "title3", "web_segment_id": 7, "global_sentence_id": 227, "edu_l1_label": "IOS"}, {"txt": "To further confrm that the privacy gains observed on models trained with LoRA do not come at the cost of utility, and that the privacy loss observed with full fne-tuning is not due to overftting or preventable by early stopping, we analyzed the utility-privacy tradeoff throughout the fne-tuning process.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.117, 0.63, 0.884, 0.645], [0.118, 0.644, 0.882, 0.659], [0.118, 0.658, 0.575, 0.673]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 228, "edu_l1_label": "IOS"}, {"txt": "Figure 3 illustrates the evolution of privacy and utility for Llama 3.2 3B during both LoRA and full fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.575, 0.658, 0.882, 0.673], [0.118, 0.671, 0.533, 0.686]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 229, "edu_l1_label": "IOS"}, {"txt": "The fgure shows that LoRA fne-tuning consistently follows a more privacy-preserving trend, with lower memorization scores compared to full fne-tuning at similar utility levels.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.533, 0.671, 0.883, 0.686], [0.118, 0.685, 0.883, 0.7], [0.118, 0.699, 0.16, 0.714]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 230, "edu_l1_label": "IOS"}, {"txt": "Furthermore, after a certain number of fne-tuning steps, the model’s tendency to memorize data increaseswithout signifcant improvements in utility, due to overftting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.16, 0.699, 0.882, 0.714], [0.117, 0.713, 0.516, 0.728]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 231, "edu_l1_label": "IOS"}, {"txt": "This highlights that early stopping during LLM training not only improves effciency, but also helps privacy by reducing the risk of memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.516, 0.713, 0.882, 0.728], [0.118, 0.726, 0.697, 0.741]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 232, "edu_l1_label": "IOS"}, {"txt": "4.4Federated Learning", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.118, 0.765, 0.292, 0.782]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 233, "edu_l1_label": "IOS"}, {"txt": "Having empirically measured how LoRA reduces unintended memorization in centralized learning, we now turn to federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.118, 0.794, 0.882, 0.809], [0.118, 0.808, 0.24, 0.823]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 234, "edu_l1_label": "IOS"}, {"txt": "The federated learning framework contains multiple key differences with centralized learning that may impact memorization, such as Federated Averaging or non-IID data across participants [Thakkar et al., 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.24, 0.808, 0.882, 0.823], [0.118, 0.822, 0.863, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 235, "edu_l1_label": "IOS"}, {"txt": "Training details.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.117, 0.842, 0.232, 0.858]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 236, "edu_l1_label": "IOS"}, {"txt": "We defne a heterogeneous setting with one client per dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.232, 0.842, 0.641, 0.858]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 237, "edu_l1_label": "IOS"}, {"txt": "In other words, we fne-tune models with 3 participants, where each participant trains locally on one of the 3 datasets MedMCQA, PubMedQA, and Medical Meadow fashcards.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.641, 0.842, 0.882, 0.858], [0.117, 0.856, 0.882, 0.871], [0.118, 0.87, 0.246, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 238, "edu_l1_label": "IOS"}, {"txt": "We split and inject i2b2 medical records into each dataset proportionally to their size.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.246, 0.87, 0.801, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 239, "edu_l1_label": "IOS"}, {"txt": "Participants fne-tune over their local dataset for one epoch between each global weight update, for a total of 5 rounds.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.801, 0.87, 0.882, 0.885], [0.118, 0.884, 0.815, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 240, "edu_l1_label": "IOS"}, {"txt": "For every model, we fne-tune the learning rate on each local dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.815, 0.884, 0.883, 0.899], [0.118, 0.897, 0.501, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 241, "edu_l1_label": "IOS"}, {"txt": "More training details are included in Appendix B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.501, 0.897, 0.83, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 242, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.492, 0.938, 0.507, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 243, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 244, "edu_l1_label": "EDU_O"}, {"txt": "Llama 3.2, 3B, Base modelLlama 3.2, 3B, LoRALlama 3.2, 3B, Full FT", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8000000, "bbox": [[0.325, 0.115, 0.678, 0.128]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 9, "global_sentence_id": 245, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8100000, "bbox": [[0.321, 0.122, 0.689, 0.26]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 5, "global_sentence_id": 246, "edu_l1_label": "EDU_O"}, {"txt": "Memorization score", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8200000, "bbox": [[0.475, 0.265, 0.583, 0.279]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 8, "global_sentence_id": 247, "edu_l1_label": "EDU_O"}, {"txt": "Figure 3: Accuracy vs. privacy across fne-tuning steps. We track accuracy and memorization (BLEU score) during Llama 3.2 3B fne-tuning (10× document duplication) using full fne-tuning (Full FT) and LoRA, compared to the basemodel. Numbers above data points indicate completed fne-tuning steps.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8300000, "bbox": [[0.115, 0.296, 0.887, 0.341]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 2, "global_sentence_id": 248, "edu_l1_label": "EDU_O"}, {"txt": "To provide fair comparisons between multiple federated learning fne-tuning, Figures 4 and 6 report metrics for the last federated communication round. This ensures that each model has been fne-tuned on the medical records the same number of times. Additionally, we include the accuracy and memorization metrics for each round in Appendix C.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.112, 0.365, 0.887, 0.409]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 6, "global_sentence_id": 249, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.315, 0.42, 0.697, 0.619]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 3, "global_sentence_id": 250, "edu_l1_label": "EDU_O"}, {"txt": "Figure 4: Downstream accuracy in federated learning. LoRA yields relatively similar accuracy to full fne-tuning for several LLMs in a heterogeneous FL setting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.115, 0.627, 0.886, 0.659]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 4, "global_sentence_id": 251, "edu_l1_label": "EDU_O"}, {"txt": "Accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.117, 0.69, 0.187, 0.707]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 252, "edu_l1_label": "IOS"}, {"txt": "Figure 4 depicts downstream accuracy of federated fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.187, 0.69, 0.615, 0.706]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 253, "edu_l1_label": "IOS"}, {"txt": "All fne-tunings show relatively similar accuracy values between full fne-tuning and LoRA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.615, 0.69, 0.883, 0.706], [0.118, 0.705, 0.459, 0.72]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 254, "edu_l1_label": "IOS"}, {"txt": "This suggests that LoRA is a competitive technique in federated learning and can replace full fne-tuning at relatively little cost, in addition to lowering the hardware requirements and the communication overheads.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.459, 0.705, 0.882, 0.72], [0.118, 0.718, 0.882, 0.733], [0.118, 0.732, 0.317, 0.747]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 255, "edu_l1_label": "IOS"}, {"txt": "Memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.118, 0.752, 0.222, 0.769]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 256, "edu_l1_label": "IOS"}, {"txt": "We frst start by comparing memorization in federated learning to centralized learning in Figure 5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.222, 0.752, 0.885, 0.768]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 257, "edu_l1_label": "IOS"}, {"txt": "We observe that FL can enhance privacy by reducing memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.117, 0.767, 0.566, 0.782]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 258, "edu_l1_label": "IOS"}, {"txt": "This is consistent with previous work [Thakkar et al., 2020] suggesting that FedAvg and a non-IID data distribution contribute to reducing unintended memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.566, 0.767, 0.883, 0.782], [0.118, 0.78, 0.885, 0.796]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 259, "edu_l1_label": "IOS"}, {"txt": "However, we note that memorization increases monotonically with the number of rounds (i.e.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.118, 0.794, 0.742, 0.809]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 260, "edu_l1_label": "IOS"}, {"txt": "the number of times medical records are seen).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.742, 0.794, 0.882, 0.809], [0.118, 0.808, 0.287, 0.823]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 261, "edu_l1_label": "IOS"}, {"txt": "Therefore, a model fne-tuned via FL can reach similar or even greater memorization levels as the number of rounds increases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.287, 0.808, 0.882, 0.823], [0.118, 0.822, 0.343, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 262, "edu_l1_label": "IOS"}, {"txt": "In fact, Figure 8 shows that, after a certain number of rounds, fne-tuning Llama 2 7B exhibits more memorization across several metrics in FL than in CL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.343, 0.822, 0.882, 0.837], [0.117, 0.835, 0.584, 0.851]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 263, "edu_l1_label": "IOS"}, {"txt": "Thus, our results expand on previous work by focusing on how memorization increases throughout the rounds.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.584, 0.835, 0.883, 0.851], [0.118, 0.849, 0.534, 0.864]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 264, "edu_l1_label": "IOS"}, {"txt": "Comparisons for all models and metrics are included in Appendix C.3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.534, 0.849, 0.882, 0.864], [0.118, 0.863, 0.23, 0.878]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 265, "edu_l1_label": "IOS"}, {"txt": "Analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.117, 0.883, 0.179, 0.9]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 266, "edu_l1_label": "IOS"}, {"txt": "Despite FL showing lower memorization than CL, all federated fne-tunings exhibit signifcant memorization, thus showing the need for additional privacy-preserving techniques.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.179, 0.883, 0.884, 0.899], [0.118, 0.897, 0.564, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 267, "edu_l1_label": "IOS"}, {"txt": "Figure 6 shows how using LoRA instead of full", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.564, 0.897, 0.882, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 268, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.494, 0.937, 0.506, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 269, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.271, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 270, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.306, 0.086, 0.7, 0.255]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 8, "global_sentence_id": 271, "edu_l1_label": "EDU_O"}, {"txt": "Figure 5: Exact match rates of FL and CL. We compare memorization between CL and FL when fne-tuning Llama 3.2 3B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.114, 0.27, 0.886, 0.298]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 6, "global_sentence_id": 272, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.304, 0.322, 0.706, 0.548]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 5, "global_sentence_id": 273, "edu_l1_label": "EDU_O"}, {"txt": "Figure 6: Memorization of LoRA vs full fne-tuning in federated learning. LoRA yields signifcantly lower memorization scores in every setting for an equivalent performance. Plots (a)-(c) show memorization using exact match rate with no duplication, 10x document duplication, and 10x document duplication with a 500 tokens prompt length, while (d)-(f) use BLEU score.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.115, 0.556, 0.888, 0.614]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 0, "global_sentence_id": 274, "edu_l1_label": "EDU_O"}, {"txt": "fne-tuning impacts memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.118, 0.638, 0.349, 0.653]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 275, "edu_l1_label": "IOS"}, {"txt": "Fine-tuning federated LLMs with LoRA displays lower memorization than full fne-tuning across all metrics and models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.349, 0.638, 0.882, 0.653], [0.118, 0.651, 0.4, 0.666]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 276, "edu_l1_label": "IOS"}, {"txt": "LoRA fne-tuning can reduce memorization up to 10× for a negligibleaccuracy loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.4, 0.651, 0.882, 0.667], [0.118, 0.665, 0.207, 0.68]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 277, "edu_l1_label": "IOS"}, {"txt": "We do note that the memorization impact of LoRA differs between similarly sized models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.207, 0.665, 0.795, 0.68]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 278, "edu_l1_label": "IOS"}, {"txt": "For example, fne-tuning Llama 2 7B with LoRA shows a drastic memorization improvement over full fne-tuning, whereas Mistral v0.3 7B shows a lower impact.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.795, 0.665, 0.884, 0.68], [0.118, 0.679, 0.882, 0.694], [0.117, 0.693, 0.318, 0.708]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 279, "edu_l1_label": "IOS"}, {"txt": "We also fnd that not all trends observed in centralized learning hold in federated learning:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.117, 0.713, 0.719, 0.729]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 280, "edu_l1_label": "IOS"}, {"txt": " data duplication, longer context and considering paraphrasing all yield higher memorization scores, however Figure 6 shows that bigger models do not necessarily result in more memorization with full fne-tuning, as Llama 3.2 1B reaches higher memorization scores than Llama 3.2 3B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.719, 0.713, 0.883, 0.729], [0.118, 0.727, 0.882, 0.742], [0.118, 0.741, 0.882, 0.756], [0.118, 0.755, 0.287, 0.77]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 281, "edu_l1_label": "IOS"}, {"txt": "Yet the trend still holds when looking at LoRA fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.287, 0.755, 0.675, 0.77]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 282, "edu_l1_label": "IOS"}, {"txt": "We leave further exploration of how model size infuences memorization in federated learning for future work.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.675, 0.755, 0.882, 0.77], [0.118, 0.768, 0.63, 0.784]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 283, "edu_l1_label": "IOS"}, {"txt": "Finally, LoRA drastically reduces FL communication overhead.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9600000, "bbox": [[0.118, 0.789, 0.529, 0.804]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 284, "edu_l1_label": "IOS"}, {"txt": "For instance, each round of our setting requires a total data exchange of 74GB for a 7B model, and using LoRA reduces the load by a factor of 152, decreasing the overhead to 498MB.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9600000, "bbox": [[0.529, 0.789, 0.882, 0.804], [0.118, 0.803, 0.882, 0.818], [0.118, 0.817, 0.186, 0.832]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 285, "edu_l1_label": "IOS"}, {"txt": "4.4.1 Secure Aggregations", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9700000, "bbox": [[0.118, 0.846, 0.15, 0.862], [0.166, 0.846, 0.309, 0.862]]}]}, "tags": ["title"], "label": "title3", "web_segment_id": 7, "global_sentence_id": 286, "edu_l1_label": "IOS"}, {"txt": "FL’s privacy benefts can be compromised if participants gain access to each other’s fne-tuned local models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.118, 0.87, 0.837, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 287, "edu_l1_label": "IOS"}, {"txt": "WhileFigure 8 highlights reduced memorization after model aggregation, unsecured local models may still expose additional information regarding participants’ datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.837, 0.87, 0.882, 0.885], [0.118, 0.884, 0.882, 0.899], [0.118, 0.897, 0.4, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 288, "edu_l1_label": "IOS"}, {"txt": "In Appendix D, we show how secure aggregation addresses this vulnerabil-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.4, 0.897, 0.885, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 289, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.495, 0.938, 0.506, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 290, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.268, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 291, "edu_l1_label": "EDU_O"}, {"txt": "ity by using a third party to aggregate encrypted local contributions using Fully Homomorphic Encryption (FHE) and decrypting the aggregated model collectively through Secure Multiparty Computation (SMPC), as described in Sébertet al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.118, 0.092, 0.882, 0.107], [0.118, 0.106, 0.882, 0.121], [0.118, 0.119, 0.149, 0.135]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 292, "edu_l1_label": "IOS"}, {"txt": "[2022].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.149, 0.119, 0.201, 0.135]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 293, "edu_l1_label": "IOS"}, {"txt": "Experiments were conducted using the open-source Lattigo library [Lattigo v6, Mouchet et al., 2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.201, 0.119, 0.869, 0.135]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 294, "edu_l1_label": "IOS"}, {"txt": "4.5Combining LoRA with other methods", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10000000, "bbox": [[0.118, 0.155, 0.417, 0.172]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 9, "global_sentence_id": 295, "edu_l1_label": "IOS"}, {"txt": "Although LoRA mitigates unintended memorization on its own, we investigate whether it can be combined with other privacy-persevering techniques without compromising performance or increasing memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.117, 0.183, 0.883, 0.198], [0.118, 0.197, 0.733, 0.212]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 296, "edu_l1_label": "IOS"}, {"txt": "If users are focused on reducing extractable memorization in pre-training, then they may be interested in Goldfsh loss (LoRA is preferred for fne-tuning), but we investigate and verify its potential for fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.733, 0.197, 0.882, 0.212], [0.118, 0.211, 0.883, 0.226], [0.118, 0.225, 0.581, 0.24]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 297, "edu_l1_label": "IOS"}, {"txt": "Gradient noising and clipping can be used to satisfy (ϵ, δ)-differential-privacy guarantees, which LoRA alone has not been formally proven to provide.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.581, 0.225, 0.882, 0.24], [0.118, 0.238, 0.804, 0.254]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 298, "edu_l1_label": "IOS"}, {"txt": "Nonetheless, we emphasize that Goldfsh loss and DP noising/clipping are not effcient strategies, as both require calculation of the full gradient.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.118, 0.259, 0.882, 0.274], [0.118, 0.273, 0.32, 0.288]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 299, "edu_l1_label": "IOS"}, {"txt": "Hence, users will choose LoRA if they are concerned about backpropagation costs or communication overhead, which is a common scenario in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.32, 0.273, 0.883, 0.288], [0.118, 0.287, 0.52, 0.302]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 300, "edu_l1_label": "IOS"}, {"txt": "4.5.1Goldfsh loss", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10300000, "bbox": [[0.118, 0.321, 0.255, 0.337]]}]}, "tags": ["title"], "label": "title3", "web_segment_id": 11, "global_sentence_id": 301, "edu_l1_label": "IOS"}, {"txt": "The Goldfsh loss [Hans et al., 2024] has been introduced recently as a memorization mitigating technique for pretraining language models via a new next-token training objective.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.117, 0.347, 0.885, 0.362], [0.118, 0.361, 0.55, 0.376]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 302, "edu_l1_label": "IOS"}, {"txt": "The training procedure randomly excludes tokens from the loss computation in order to prevent verbatim reproduction of training sequences.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.55, 0.361, 0.882, 0.376], [0.118, 0.375, 0.699, 0.39]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 303, "edu_l1_label": "IOS"}, {"txt": "In Appendix E, we evaluate the memorization and accuracy of Llama 3.2 3B fne-tuned with LoRA in combination with Goldfsh loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.699, 0.375, 0.882, 0.39], [0.118, 0.388, 0.824, 0.404]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 304, "edu_l1_label": "IOS"}, {"txt": "We also compare it to the same model fully fne-tuned with Goldfsh loss only.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.824, 0.388, 0.882, 0.404], [0.118, 0.402, 0.583, 0.417]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 305, "edu_l1_label": "IOS"}, {"txt": "The combination of LoRA with Goldfsh loss synergistically achieves lower memorization beyond what either strategy achieves alone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.583, 0.402, 0.882, 0.417], [0.118, 0.416, 0.694, 0.431]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 306, "edu_l1_label": "IOS"}, {"txt": "4.5.2Differential privacy", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10500000, "bbox": [[0.118, 0.45, 0.303, 0.467]]}]}, "tags": ["title"], "label": "title3", "web_segment_id": 8, "global_sentence_id": 307, "edu_l1_label": "IOS"}, {"txt": "(ϵ, δ)-Differential privacy (DP) provides formal guarantees that an individual’s data cannot be inferred from a model’soutput, by quantifying the model’s sensitivity to changes in input data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.116, 0.479, 0.882, 0.492], [0.118, 0.49, 0.572, 0.505]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 308, "edu_l1_label": "IOS"}, {"txt": "Following Li et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.572, 0.49, 0.697, 0.505]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 309, "edu_l1_label": "IOS"}, {"txt": "[2021] and Liu et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.697, 0.49, 0.833, 0.505]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 310, "edu_l1_label": "IOS"}, {"txt": "[2024],we defne sensitivity as the maximum change in model output resulting from the inclusion or removal of a single data point in the training dataset (record-level DP).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.833, 0.49, 0.884, 0.505], [0.117, 0.504, 0.882, 0.519], [0.118, 0.518, 0.418, 0.533]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 311, "edu_l1_label": "IOS"}, {"txt": "Implementing DP requires modifcations to the fne-tuning pipeline to limit the infuence of individual data points on model parameters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.118, 0.538, 0.882, 0.554], [0.118, 0.552, 0.24, 0.567]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 312, "edu_l1_label": "IOS"}, {"txt": "Gradient clipping, which constrains the magnitude of gradient updates, is a key technique in this process.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.24, 0.552, 0.882, 0.567], [0.118, 0.566, 0.172, 0.581]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 313, "edu_l1_label": "IOS"}, {"txt": "In our experiments (see Appendix G.1), applying a gradient clipping value of 0.0001 signifcantly reduces memorization and improves accuracy compared to the default value of 1.0.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.172, 0.566, 0.882, 0.581], [0.118, 0.58, 0.611, 0.595]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 314, "edu_l1_label": "IOS"}, {"txt": "This demonstrates gradient clipping as a privacy-enhancing method in itself, even without the addition of noise.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.611, 0.58, 0.882, 0.595], [0.118, 0.594, 0.566, 0.609]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 315, "edu_l1_label": "IOS"}, {"txt": "But the use of stochastic gradient descent (SGD), required for DP-SGD, presents challenges in fne-tuning the Llama 3.2 3B model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.566, 0.594, 0.884, 0.609], [0.118, 0.607, 0.667, 0.623]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 316, "edu_l1_label": "IOS"}, {"txt": "Despite an extensive search for optimal learning rates, SGD consistently underperforms compared to Adam-derived optimizers (see Appendix G.2).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.667, 0.607, 0.883, 0.623], [0.118, 0.621, 0.873, 0.636]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 317, "edu_l1_label": "IOS"}, {"txt": "5Discussion", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10800000, "bbox": [[0.118, 0.66, 0.235, 0.679]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 10, "global_sentence_id": 318, "edu_l1_label": "BOS"}, {"txt": "Our experimental evaluation demonstrates that LoRA reduces memorization in both centralized and FL settings, which naturally raises the question:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.118, 0.695, 0.882, 0.71], [0.118, 0.709, 0.31, 0.724]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 319, "edu_l1_label": "IOS"}, {"txt": " why does this happen?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.31, 0.709, 0.47, 0.724]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 320, "edu_l1_label": "IOS"}, {"txt": "We argue that the mechanisms by which FedAvg and LoRA mitigate memorization should be considered independently.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.47, 0.709, 0.883, 0.724], [0.118, 0.723, 0.517, 0.738]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 321, "edu_l1_label": "IOS"}, {"txt": "Carlini et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.517, 0.723, 0.609, 0.738]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 322, "edu_l1_label": "IOS"}, {"txt": "[2022] empirically establish a log-linear relationship between canary duplication and memorization, thus we frame our discussion of memorization in the context of overftting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.609, 0.723, 0.883, 0.738], [0.118, 0.736, 0.882, 0.752], [0.118, 0.75, 0.205, 0.765]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 323, "edu_l1_label": "IOS"}, {"txt": "How and why in-distribution, non-duplicated sequences can still be regurgitated [Carlini et al., 2019] is a question that we leave to future work.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.205, 0.75, 0.882, 0.765], [0.118, 0.764, 0.363, 0.779]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 324, "edu_l1_label": "IOS"}, {"txt": "Federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.118, 0.784, 0.256, 0.801]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 325, "edu_l1_label": "IOS"}, {"txt": "While it is known that FedAvg can reduce memorization for simpler LSTM-based next-word predictors (NWPs) [Ramaswamy et al., 2020, Thakkar et al., 2020], we hope that our verifcation of this phenomenon for LLMs on longer canaries can encourage formal investigation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.256, 0.784, 0.882, 0.8], [0.118, 0.798, 0.882, 0.814], [0.118, 0.812, 0.556, 0.827]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 326, "edu_l1_label": "IOS"}, {"txt": "Nevertheless, we note the following: in the IID FedAvg setting with identical hyperparameter settings (same number of local updates, learning rate, and initialization) k resemble a single stochastic gradient in a centralized setting taken over a sNingdle large batch of ∼ size Nk since fk and Dkare homogeneous.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.556, 0.812, 0.882, 0.827], [0.118, 0.826, 0.883, 0.841], [0.635, 0.841, 0.642, 0.85], [0.118, 0.856, 0.881, 0.872], [0.118, 0.87, 0.235, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 327, "edu_l1_label": "IOS"}, {"txt": "Thus, Thakkar et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.235, 0.87, 0.37, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 328, "edu_l1_label": "IOS"}, {"txt": "[2020] observe more memorization in IID settings with larger batch sizes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.37, 0.87, 0.852, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 329, "edu_l1_label": "IOS"}, {"txt": "The non-IID setting is signifcantly more complex: the optimization problem and associated loss landscape of Equation 1 differs from the centralized problem.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.852, 0.87, 0.882, 0.885], [0.118, 0.884, 0.884, 0.899], [0.118, 0.897, 0.351, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 330, "edu_l1_label": "IOS"}, {"txt": "We observe in Figures 5 and 6 that non-IID FL signifcantly reduces memorization,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.351, 0.897, 0.884, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 331, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.494, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 332, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.268, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 333, "edu_l1_label": "EDU_O"}, {"txt": "which Thakkar et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.117, 0.092, 0.251, 0.107]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 334, "edu_l1_label": "IOS"}, {"txt": "[2020] also observe for their NWPs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.251, 0.092, 0.491, 0.107]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 335, "edu_l1_label": "IOS"}, {"txt": "While they do not fne-tune their learning rates to eliminate this as a confounding variable, we do5, thus suggesting that FedAvg itself is a memorization-reducing mechanism.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.491, 0.092, 0.882, 0.107], [0.118, 0.106, 0.861, 0.121]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 336, "edu_l1_label": "IOS"}, {"txt": "LoRA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.118, 0.126, 0.165, 0.142]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 337, "edu_l1_label": "IOS"}, {"txt": "It is possible that LoRA reduces benign overftting [Bartlett et al., 2020], which occurs when training data is overftted without affecting performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.165, 0.126, 0.882, 0.141], [0.118, 0.14, 0.381, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 338, "edu_l1_label": "IOS"}, {"txt": "Notably, Tang et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.381, 0.14, 0.512, 0.155]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 339, "edu_l1_label": "IOS"}, {"txt": "[2023a] prove that benign overftting can preserve out-ofdistribution generalization for overparameterized linear models if there is a strong correlation between the dominant eigenvectors/components of the source and target distributions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.512, 0.14, 0.885, 0.155], [0.118, 0.154, 0.882, 0.169], [0.118, 0.168, 0.535, 0.183]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 340, "edu_l1_label": "IOS"}, {"txt": "It is possible then that our LLMs are displaying this phenomenon: in both the centralized and FL settings, our fne-tuning datasets, while heterogeneous, contain aligned components due to their shared domain.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.535, 0.168, 0.882, 0.183], [0.118, 0.181, 0.882, 0.197], [0.118, 0.195, 0.38, 0.21]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 341, "edu_l1_label": "IOS"}, {"txt": "LoRA may reduce benign overftting by ignoring minor components, which only explain a minimal (and possibly noisy) portion of the data covariance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.38, 0.195, 0.882, 0.21], [0.118, 0.209, 0.606, 0.224]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 342, "edu_l1_label": "IOS"}, {"txt": "Specifc to FL, an alternative hypothesis is that the low-rank approximation of ∆W resembles a δ-compressionoperator [Karimireddy et al., 2019], i.e., ||LORA(∆W) −∆W||2 ≤(1 −δ)||∆W||2, and that low-δ compressors reducememorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.118, 0.23, 0.882, 0.245], [0.118, 0.243, 0.882, 0.259], [0.118, 0.257, 0.214, 0.272]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 343, "edu_l1_label": "IOS"}, {"txt": "Low-bias compressors, such as certain randomized projections [Dorfman et al., 2023, Rabbani et al., 2021, Ivkin et al., 2019] and other low-rank approximations [Makkuva et al., 2023] have been shown to preserve model performance in non-IID distributed settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.214, 0.257, 0.884, 0.272], [0.118, 0.271, 0.882, 0.286], [0.118, 0.285, 0.406, 0.3]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 344, "edu_l1_label": "IOS"}, {"txt": "While the effects of these other operators on memorization has not been extensively studied, the effcacy of gradient clipping in lowering memorization while maintaining accuracy (Table 8) lends further credence to this hypothesis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.406, 0.285, 0.882, 0.3], [0.118, 0.299, 0.882, 0.314], [0.118, 0.312, 0.409, 0.327]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 345, "edu_l1_label": "IOS"}, {"txt": "Clipping is a low-bias compressor for heavy-tailed gradients, which is observed for general SGD [Mireshghallah et al., 2022] and LLM fne-tuning [Kenton and Toutanova, 2019].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.409, 0.312, 0.882, 0.327], [0.118, 0.326, 0.829, 0.341]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 346, "edu_l1_label": "IOS"}, {"txt": "Further exploration of δ-compressors is warranted.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.829, 0.326, 0.883, 0.341], [0.118, 0.34, 0.396, 0.355]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 347, "edu_l1_label": "IOS"}, {"txt": "6Conclusion and Limitations", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11400000, "bbox": [[0.118, 0.373, 0.378, 0.393]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 348, "edu_l1_label": "BOS"}, {"txt": "In this work, we demonstrate that LoRA is capable of reducing memorization of fne-tuning training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.118, 0.405, 0.797, 0.42]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 349, "edu_l1_label": "IOS"}, {"txt": "In particular, this effect is observable in both centralized learning and federated learning (FL), and we fnd this effect is especially pronounced in the latter.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.797, 0.405, 0.884, 0.42], [0.118, 0.419, 0.883, 0.434], [0.118, 0.433, 0.282, 0.448]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 350, "edu_l1_label": "IOS"}, {"txt": "Moreover, it is possible to further reduce memorization by combining LoRA with other strategies such as Goldfsh loss or conventional privacy-preserving mechanisms such as Gaussian noising and gradient clipping.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.282, 0.433, 0.883, 0.448], [0.118, 0.446, 0.882, 0.461], [0.118, 0.46, 0.175, 0.475]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 351, "edu_l1_label": "IOS"}, {"txt": "FL was previously shown to reduce memorization for simple LSTM-based next-word predictors [Hard et al., 2018, Thakkar et al., 2020] and we demonstrate that generative LLMs inherit this beneft as well.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.175, 0.46, 0.884, 0.475], [0.118, 0.474, 0.764, 0.489]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 352, "edu_l1_label": "IOS"}, {"txt": "However, further theoretical analysis of this phenomenon, which may relate to the LoRA reductive effect, is needed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.764, 0.474, 0.883, 0.489], [0.118, 0.488, 0.76, 0.503]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 353, "edu_l1_label": "IOS"}, {"txt": "We note that LoRA is only suitable for fne-tuning, while other techniques are required for the pre-training phase.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11600000, "bbox": [[0.117, 0.508, 0.885, 0.523]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 354, "edu_l1_label": "IOS"}, {"txt": "The impact of LoRA on memorization during pre-training remains an open question for future work.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11600000, "bbox": [[0.117, 0.522, 0.791, 0.537]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 355, "edu_l1_label": "IOS"}, {"txt": "Additionally, further research is needed to determine whether LoRA mitigates data regurgitation under alternative defnitions of memorization [Schwarzschild et al., 2024].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11600000, "bbox": [[0.791, 0.522, 0.884, 0.537], [0.118, 0.536, 0.882, 0.551], [0.118, 0.55, 0.398, 0.565]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 356, "edu_l1_label": "IOS"}, {"txt": "Impact statement", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.118, 0.583, 0.264, 0.602]]}]}, "tags": ["title"], "label": "content", "web_segment_id": 11, "global_sentence_id": 357, "edu_l1_label": "EDU_O"}, {"txt": "This paper presents work whose goal is to advance the feld of Machine Learning, especially enhancing privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.117, 0.615, 0.885, 0.63]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 358, "edu_l1_label": "IOS"}, {"txt": "Among the many potential societal consequences of our work, we specifcally acknowledge that techniques mitigating unintended memorization can incidentally facilitate the concealment of unlawful use of copyrighted data by preventing its regurgitation post-training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.117, 0.629, 0.882, 0.644], [0.118, 0.642, 0.882, 0.657], [0.118, 0.656, 0.319, 0.671]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 359, "edu_l1_label": "IOS"}, {"txt": "However, we believe that the beneft of enhanced safeguards for confdential data protection combined with the current advances of other methods such as watermarking [Li et al., 2023b, Tang et al., 2023b, Cui et al., 2024] can effectively mitigate this risk and provide stronger overall data protection.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.319, 0.656, 0.882, 0.671], [0.118, 0.67, 0.884, 0.685], [0.118, 0.684, 0.777, 0.699]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 360, "edu_l1_label": "IOS"}, {"txt": "Acknowledgments", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11900000, "bbox": [[0.118, 0.717, 0.27, 0.736]]}]}, "tags": ["title"], "label": "content", "web_segment_id": 10, "global_sentence_id": 361, "edu_l1_label": "EDU_O"}, {"txt": "This research is conducted as part of an Innovation Project supported by Innosuisse.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.117, 0.749, 0.649, 0.764]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 362, "edu_l1_label": "EDU_O"}, {"txt": "The authors gratefully acknowledge fnancial support from Innosuisse under the Innovation Projects with Implementation Partner funding scheme.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.649, 0.749, 0.882, 0.764], [0.118, 0.763, 0.809, 0.778]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 363, "edu_l1_label": "EDU_O"}, {"txt": "Additional support was provided by the European Union within the framework of the Phase IV AI Project.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.809, 0.763, 0.882, 0.778], [0.118, 0.776, 0.738, 0.791]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 364, "edu_l1_label": "EDU_O"}, {"txt": "References", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.118, 0.809, 0.208, 0.829]]}]}, "tags": ["title"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 365, "edu_l1_label": "EDU_O"}, {"txt": "Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.118, 0.834, 0.882, 0.85], [0.134, 0.848, 0.301, 0.863]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 366, "edu_l1_label": "EDU_O"}, {"txt": "A survey of large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.301, 0.848, 0.537, 0.863]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 367, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2303.18223, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.537, 0.848, 0.804, 0.863]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 368, "edu_l1_label": "EDU_O"}, {"txt": "5While it is possible that performing centralized learning in a curriculum-style manner with heterogeneous learning rates over training data can reduce memorization, given the small performance gap against non-IID FL, it is highly unlikely that this alone can improve its signifcantly worse memorization scores.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.138, 0.872, 0.883, 0.887], [0.118, 0.886, 0.882, 0.9], [0.118, 0.899, 0.427, 0.912]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 369, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.492, 0.937, 0.508, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 370, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 371, "edu_l1_label": "EDU_O"}, {"txt": "Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.092, 0.882, 0.107], [0.134, 0.106, 0.227, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 372, "edu_l1_label": "EDU_O"}, {"txt": "Large language models in medicine.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.227, 0.106, 0.469, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 373, "edu_l1_label": "EDU_O"}, {"txt": "Nature medicine, 29(8):1930–1940, 2023.Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Anthony B Costa, Mona G Flores, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.469, 0.106, 0.749, 0.121], [0.118, 0.124, 0.884, 0.14], [0.134, 0.138, 0.495, 0.153]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 374, "edu_l1_label": "EDU_O"}, {"txt": "A large language model for electronic health records.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.495, 0.138, 0.848, 0.153]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 375, "edu_l1_label": "EDU_O"}, {"txt": "NPJ digital medicine, 5(1):194, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.848, 0.138, 0.882, 0.153], [0.134, 0.152, 0.348, 0.167]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 376, "edu_l1_label": "EDU_O"}, {"txt": "Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.171, 0.882, 0.186], [0.134, 0.185, 0.329, 0.2]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 377, "edu_l1_label": "EDU_O"}, {"txt": "Bloomberggpt: A large language model for fnance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.329, 0.185, 0.664, 0.2]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 378, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2303.17564, 2023a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.664, 0.185, 0.884, 0.2], [0.134, 0.198, 0.178, 0.214]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 379, "edu_l1_label": "EDU_O"}, {"txt": "Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.217, 0.471, 0.232]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 380, "edu_l1_label": "EDU_O"}, {"txt": "Large language models in fnance: A survey.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.471, 0.217, 0.763, 0.232]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 381, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the fourth ACM international conference on AI in fnance, pages 374–382, 2023a.H.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.763, 0.217, 0.882, 0.232], [0.134, 0.231, 0.665, 0.246], [0.118, 0.25, 0.133, 0.265]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 382, "edu_l1_label": "EDU_O"}, {"txt": "B. McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.133, 0.25, 0.713, 0.265]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 383, "edu_l1_label": "EDU_O"}, {"txt": "Communication-effcientlearning of deep networks from decentralized data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.713, 0.25, 0.882, 0.265], [0.134, 0.264, 0.481, 0.279]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 384, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Artifcial Intelligence and Statistics, 2016.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.481, 0.264, 0.882, 0.279], [0.134, 0.277, 0.237, 0.293]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 385, "edu_l1_label": "EDU_O"}, {"txt": "URL https://api.semanticscholar.org/CorpusID:14955348.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.237, 0.277, 0.702, 0.293]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 386, "edu_l1_label": "EDU_O"}, {"txt": "Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, KallistaBonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.296, 0.882, 0.311], [0.134, 0.31, 0.604, 0.325]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 387, "edu_l1_label": "EDU_O"}, {"txt": "Advances and open problems in federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.604, 0.31, 0.882, 0.325], [0.134, 0.324, 0.191, 0.339]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 388, "edu_l1_label": "EDU_O"}, {"txt": "Foundations and trends® in machine learning, 14(1–2):1–210, 2021.Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.191, 0.324, 0.647, 0.339], [0.118, 0.344, 0.611, 0.359]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 389, "edu_l1_label": "EDU_O"}, {"txt": "The secret sharer: Evaluating and testingunintended memorization in neural networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.611, 0.344, 0.882, 0.359], [0.134, 0.358, 0.436, 0.373]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 390, "edu_l1_label": "EDU_O"}, {"txt": "In 28th USENIX security symposium (USENIX security 19), pages 267–284, 2019.Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.436, 0.358, 0.882, 0.373], [0.134, 0.372, 0.236, 0.387], [0.118, 0.391, 0.8, 0.406]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 391, "edu_l1_label": "EDU_O"}, {"txt": "Quantifying memorization across neural language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.8, 0.391, 0.882, 0.406], [0.134, 0.404, 0.432, 0.42]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 392, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2202.07646, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.432, 0.404, 0.698, 0.42]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 393, "edu_l1_label": "EDU_O"}, {"txt": "Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.423, 0.884, 0.438], [0.133, 0.437, 0.463, 0.452]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 394, "edu_l1_label": "EDU_O"}, {"txt": "Extracting training data from large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.463, 0.437, 0.826, 0.452]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 395, "edu_l1_label": "EDU_O"}, {"txt": "In 30th USENIX Security Symposium (USENIX Security 21), pages 2633–2650, 2021.Paul Dourish.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.826, 0.437, 0.882, 0.452], [0.133, 0.451, 0.641, 0.466], [0.118, 0.47, 0.207, 0.485]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 396, "edu_l1_label": "EDU_O"}, {"txt": "What we talk about when we talk about context.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.207, 0.47, 0.527, 0.485]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 397, "edu_l1_label": "EDU_O"}, {"txt": "Personal and ubiquitous computing, 8:19–30, 2004.Vitaly Feldman.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.527, 0.47, 0.87, 0.485], [0.118, 0.489, 0.221, 0.504]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 398, "edu_l1_label": "EDU_O"}, {"txt": "Does learning require memorization?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.221, 0.489, 0.466, 0.504]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 399, "edu_l1_label": "EDU_O"}, {"txt": "a short tale about a long tail.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.466, 0.489, 0.653, 0.504]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 400, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954–959, 2020.Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.653, 0.489, 0.882, 0.504], [0.133, 0.502, 0.618, 0.517], [0.118, 0.521, 0.602, 0.536]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 401, "edu_l1_label": "EDU_O"}, {"txt": "Evaluating gradient inversion attacks and defenses in federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.602, 0.521, 0.882, 0.536], [0.134, 0.535, 0.333, 0.55]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 402, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 34:7232–7241, 2021.Ahmed El Ouadrhiri and Ahmed Abdelhadi.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.333, 0.535, 0.82, 0.55], [0.118, 0.554, 0.415, 0.569]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 403, "edu_l1_label": "EDU_O"}, {"txt": "Differential privacy for deep and federated learning: A survey.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.415, 0.554, 0.839, 0.569]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 404, "edu_l1_label": "EDU_O"}, {"txt": "IEEE access, 10:22359–22380, 2022.Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, and Françoise Beaufays.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.839, 0.554, 0.882, 0.569], [0.134, 0.568, 0.34, 0.583], [0.118, 0.586, 0.616, 0.602]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 405, "edu_l1_label": "EDU_O"}, {"txt": "Understanding unintended memorizationin federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.616, 0.586, 0.882, 0.602], [0.134, 0.6, 0.273, 0.615]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 406, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2006.07490, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.273, 0.6, 0.539, 0.615]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 407, "edu_l1_label": "EDU_O"}, {"txt": "Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.619, 0.841, 0.634]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 408, "edu_l1_label": "EDU_O"}, {"txt": "Lora: Low-rank adaptation of large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.841, 0.619, 0.885, 0.634], [0.134, 0.633, 0.448, 0.648]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 409, "edu_l1_label": "EDU_O"}, {"txt": "CoRR, abs/2106.09685, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.448, 0.633, 0.653, 0.648]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 410, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/ 2106.09685.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.653, 0.633, 0.884, 0.649], [0.134, 0.648, 0.223, 0.662]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 411, "edu_l1_label": "EDU_O"}, {"txt": "Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A.Choquette-Choo, and Nicholas Carlini.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.665, 0.885, 0.681], [0.134, 0.679, 0.385, 0.694]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 412, "edu_l1_label": "EDU_O"}, {"txt": "Preventing verbatim memorization in language models gives a false sense of privacy, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.385, 0.679, 0.882, 0.694], [0.134, 0.693, 0.225, 0.708]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 413, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2210.17546.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.225, 0.693, 0.545, 0.708]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 414, "edu_l1_label": "EDU_O"}, {"txt": "Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.712, 0.585, 0.727]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 415, "edu_l1_label": "EDU_O"}, {"txt": "Qlora: Effcient fnetuning of quantized llms.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.585, 0.712, 0.885, 0.727]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 416, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 36, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.133, 0.726, 0.543, 0.741]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 417, "edu_l1_label": "EDU_O"}, {"txt": "Xiao-Yang Liu, Rongyi Zhu, Daochen Zha, Jiechao Gao, Shan Zhong, Matt White, and Meikang Qiu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.744, 0.788, 0.76]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 418, "edu_l1_label": "EDU_O"}, {"txt": "Differentially private low-rank adaptation of large language model using federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.788, 0.744, 0.883, 0.76], [0.134, 0.758, 0.644, 0.773]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 419, "edu_l1_label": "EDU_O"}, {"txt": "ACM Transactions on Management Information Systems, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.644, 0.758, 0.882, 0.773], [0.134, 0.772, 0.312, 0.787]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 420, "edu_l1_label": "EDU_O"}, {"txt": "Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.791, 0.884, 0.806], [0.134, 0.805, 0.55, 0.82]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 421, "edu_l1_label": "EDU_O"}, {"txt": "Be like a goldfsh, don’t memorize!", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.55, 0.805, 0.804, 0.82]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 422, "edu_l1_label": "EDU_O"}, {"txt": "mitigatingmemorization in generative llms.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.804, 0.805, 0.882, 0.82], [0.134, 0.818, 0.348, 0.834]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 423, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2406.10209, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.348, 0.818, 0.615, 0.834]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 424, "edu_l1_label": "EDU_O"}, {"txt": "Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.118, 0.837, 0.568, 0.852]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 425, "edu_l1_label": "EDU_O"}, {"txt": "Calibrating noise to sensitivity in private data analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.568, 0.837, 0.882, 0.852], [0.134, 0.851, 0.192, 0.866]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 426, "edu_l1_label": "EDU_O"}, {"txt": "In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.192, 0.851, 0.885, 0.866], [0.133, 0.865, 0.246, 0.88]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 427, "edu_l1_label": "EDU_O"}, {"txt": "Proceedings 3, pages 265–284.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.246, 0.865, 0.454, 0.88]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 428, "edu_l1_label": "EDU_O"}, {"txt": "Springer, 2006.Bargav Jayaraman and David Evans.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.454, 0.865, 0.558, 0.88], [0.118, 0.884, 0.359, 0.899]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 429, "edu_l1_label": "EDU_O"}, {"txt": "Evaluating differentially private machine learning in practice.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.359, 0.884, 0.769, 0.899]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 430, "edu_l1_label": "EDU_O"}, {"txt": "In 28th USENIX Security Symposium (USENIX Security 19), pages 1895–1912, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.769, 0.884, 0.882, 0.899], [0.134, 0.897, 0.581, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 431, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.491, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 432, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 433, "edu_l1_label": "EDU_O"}, {"txt": "Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.092, 0.807, 0.107]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 434, "edu_l1_label": "EDU_O"}, {"txt": "Analyzingleakage of personally identifable information in language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.807, 0.092, 0.882, 0.107], [0.134, 0.106, 0.556, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 435, "edu_l1_label": "EDU_O"}, {"txt": "In 2023 IEEE Symposium on Security and Privacy (SP), pages 346–363.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.556, 0.106, 0.882, 0.121], [0.133, 0.119, 0.272, 0.135]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 436, "edu_l1_label": "EDU_O"}, {"txt": "IEEE, 2023.Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.272, 0.119, 0.356, 0.135], [0.118, 0.138, 0.807, 0.154]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 437, "edu_l1_label": "EDU_O"}, {"txt": "Deep learning with differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.807, 0.138, 0.885, 0.154], [0.134, 0.152, 0.315, 0.167]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 438, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016.Bogdan Kulynych, Hsiang Hsu, Carmela Troncoso, and Flavio P Calmon.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.315, 0.152, 0.882, 0.167], [0.134, 0.166, 0.336, 0.181], [0.118, 0.185, 0.621, 0.2]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 439, "edu_l1_label": "EDU_O"}, {"txt": "Arbitrary decisions are a hidden cost of differentially private training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.621, 0.185, 0.882, 0.2], [0.134, 0.199, 0.35, 0.214]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 440, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1609–1623, 2023.Helen Nissenbaum.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.35, 0.199, 0.882, 0.214], [0.133, 0.212, 0.389, 0.228], [0.118, 0.231, 0.245, 0.246]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 441, "edu_l1_label": "EDU_O"}, {"txt": "Privacy as contextual integrity.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.245, 0.231, 0.451, 0.246]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 442, "edu_l1_label": "EDU_O"}, {"txt": "Wash. L.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.451, 0.231, 0.513, 0.246]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 443, "edu_l1_label": "EDU_O"}, {"txt": "Rev., 79:119, 2004.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.513, 0.231, 0.642, 0.246]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 444, "edu_l1_label": "EDU_O"}, {"txt": "Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.25, 0.733, 0.265]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 445, "edu_l1_label": "EDU_O"}, {"txt": "What does it mean fora language model to preserve privacy?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.733, 0.25, 0.883, 0.265], [0.134, 0.264, 0.382, 0.279]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 446, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency, pages 2280–2292, 2022.Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H Brendan McMahan, and Françoise Beaufays.Training production language models without memorizing user data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.382, 0.264, 0.882, 0.279], [0.134, 0.278, 0.386, 0.293], [0.118, 0.297, 0.885, 0.312], [0.133, 0.31, 0.58, 0.326]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 447, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2009.10031, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.58, 0.31, 0.847, 0.326]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 448, "edu_l1_label": "EDU_O"}, {"txt": "Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.329, 0.884, 0.345], [0.134, 0.343, 0.522, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 449, "edu_l1_label": "EDU_O"}, {"txt": "Measuring forgetting of memorized training examples.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.522, 0.343, 0.885, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 450, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2207.00099, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.134, 0.357, 0.395, 0.372]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 451, "edu_l1_label": "EDU_O"}, {"txt": "Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.376, 0.717, 0.391]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 452, "edu_l1_label": "EDU_O"}, {"txt": "Communication-effcient learning of deep networks from decentralized data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.717, 0.376, 0.882, 0.391], [0.134, 0.39, 0.46, 0.405]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 453, "edu_l1_label": "EDU_O"}, {"txt": "In Artifcial intelligence and statistics, pages 1273–1282.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.46, 0.39, 0.833, 0.405]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 454, "edu_l1_label": "EDU_O"}, {"txt": "PMLR,2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.833, 0.39, 0.884, 0.405], [0.134, 0.403, 0.171, 0.419]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 455, "edu_l1_label": "EDU_O"}, {"txt": "Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, HubertEichner, Chloé Kiddon, and Daniel Ramage.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.422, 0.882, 0.438], [0.134, 0.436, 0.432, 0.451]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 456, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning for mobile keyboard prediction.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.432, 0.436, 0.78, 0.451]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 457, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprintarXiv:1811.03604, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.78, 0.436, 0.882, 0.451], [0.134, 0.45, 0.298, 0.465]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 458, "edu_l1_label": "EDU_O"}, {"txt": "Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C Wallace.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.469, 0.636, 0.484]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 459, "edu_l1_label": "EDU_O"}, {"txt": "Does bert pretrained on clinical notes reveal sensitive data?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.636, 0.469, 0.882, 0.484], [0.134, 0.483, 0.272, 0.498]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 460, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2104.07762, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.272, 0.483, 0.539, 0.498]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 461, "edu_l1_label": "EDU_O"}, {"txt": "Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.502, 0.796, 0.517]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 462, "edu_l1_label": "EDU_O"}, {"txt": "Quantifying privacy risks of masked language models using membership inference attacks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.796, 0.502, 0.882, 0.517], [0.134, 0.515, 0.654, 0.53]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 463, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2203.03929, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.654, 0.515, 0.884, 0.53], [0.134, 0.529, 0.171, 0.544]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 464, "edu_l1_label": "EDU_O"}, {"txt": "Jie Xu, Benjamin S Glicksberg, Chang Su, Peter Walker, Jiang Bian, and Fei Wang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.548, 0.661, 0.563]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 465, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning for healthcare informatics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.661, 0.548, 0.882, 0.563], [0.134, 0.562, 0.212, 0.577]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 466, "edu_l1_label": "EDU_O"}, {"txt": "Journal of healthcare informatics research, 5:1–19, 2021.Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia Dobre, and Won-Joo Hwang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.212, 0.562, 0.594, 0.577], [0.118, 0.581, 0.882, 0.596], [0.133, 0.595, 0.247, 0.61]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 467, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning for smart healthcare: A survey.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.247, 0.595, 0.582, 0.61]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 468, "edu_l1_label": "EDU_O"}, {"txt": "ACM Computing Surveys (Csur), 55(3):1–37,2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.582, 0.595, 0.884, 0.61], [0.134, 0.608, 0.171, 0.623]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 469, "edu_l1_label": "EDU_O"}, {"txt": "Rodolfo Stoffel Antunes, Cristiano André da Costa, Arne Küderle, Imrana Abdullahi Yari, and Björn Eskofer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.627, 0.815, 0.642]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 470, "edu_l1_label": "EDU_O"}, {"txt": "Federatedlearning for healthcare: Systematic review and architecture proposal.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.815, 0.627, 0.882, 0.642], [0.134, 0.641, 0.581, 0.656]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 471, "edu_l1_label": "EDU_O"}, {"txt": "ACM Transactions on Intelligent Systems and Technology (TIST), 13(4):1–23, 2022.Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.581, 0.641, 0.882, 0.656], [0.133, 0.655, 0.379, 0.67], [0.118, 0.674, 0.562, 0.689]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 472, "edu_l1_label": "EDU_O"}, {"txt": "Measuring the intrinsic dimension of objective landscapes, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.562, 0.674, 0.882, 0.689], [0.134, 0.687, 0.249, 0.703]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 473, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/1804.08838.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.249, 0.687, 0.569, 0.703]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 474, "edu_l1_label": "EDU_O"}, {"txt": "Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.706, 0.481, 0.722]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 475, "edu_l1_label": "EDU_O"}, {"txt": "Intrinsic dimensionality explains the effectiveness of language model fne-tuning, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.481, 0.706, 0.882, 0.722], [0.134, 0.72, 0.295, 0.735]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 476, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2012.13255.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.295, 0.72, 0.615, 0.735]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 477, "edu_l1_label": "EDU_O"}, {"txt": "Chao Huang, Jianwei Huang, and Xin Liu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.739, 0.4, 0.754]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 478, "edu_l1_label": "EDU_O"}, {"txt": "Cross-silo federated learning: Challenges and opportunities, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.4, 0.739, 0.844, 0.754]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 479, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2206.12949.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.844, 0.739, 0.883, 0.754], [0.134, 0.754, 0.411, 0.768]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 480, "edu_l1_label": "EDU_O"}, {"txt": "Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.772, 0.885, 0.787]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 481, "edu_l1_label": "EDU_O"}, {"txt": "Openfedllm: Training large language models on decentralized private data via federated learning, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.134, 0.786, 0.84, 0.801]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 482, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2402.06954.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.84, 0.786, 0.883, 0.801], [0.134, 0.8, 0.411, 0.815]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 483, "edu_l1_label": "EDU_O"}, {"txt": "Jing Huang, Diyi Yang, and Christopher Potts.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.818, 0.421, 0.833]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 484, "edu_l1_label": "EDU_O"}, {"txt": "Demystifying verbatim memorization in large language models, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.421, 0.818, 0.885, 0.833]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 485, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2407.17817.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.134, 0.832, 0.448, 0.847]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 486, "edu_l1_label": "EDU_O"}, {"txt": "Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.851, 0.8, 0.866]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 487, "edu_l1_label": "EDU_O"}, {"txt": "Quantifying memorization across neural language models, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.8, 0.851, 0.882, 0.866], [0.134, 0.865, 0.473, 0.88]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 488, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2202.07646.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.473, 0.865, 0.793, 0.88]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 489, "edu_l1_label": "EDU_O"}, {"txt": "Mostafa Langarizadeh, Azam Orooji, and Abbas Sheikhtaheri.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.118, 0.884, 0.524, 0.899]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 490, "edu_l1_label": "EDU_O"}, {"txt": "Effectiveness of anonymization methods in preserving patients’ privacy: A systematic literature review.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.524, 0.884, 0.882, 0.899], [0.134, 0.897, 0.449, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 491, "edu_l1_label": "EDU_O"}, {"txt": "Stud.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.449, 0.897, 0.488, 0.912]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 492, "edu_l1_label": "EDU_O"}, {"txt": "Health Technol.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.488, 0.897, 0.595, 0.912]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 493, "edu_l1_label": "EDU_O"}, {"txt": "Inform., 248, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.595, 0.897, 0.725, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 494, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.491, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 495, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 496, "edu_l1_label": "EDU_O"}, {"txt": "Paul M Heider, Jihad S Obeid, and Stéphane M Meystre.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.092, 0.501, 0.107]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 497, "edu_l1_label": "EDU_O"}, {"txt": "A comparative analysis of speed and accuracy for threeoff-the-shelf DE-identifcation tools.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.501, 0.092, 0.882, 0.107], [0.134, 0.106, 0.372, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 498, "edu_l1_label": "EDU_O"}, {"txt": "AMIA Summits Transl.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.372, 0.106, 0.525, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 499, "edu_l1_label": "EDU_O"}, {"txt": "Sci.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.525, 0.106, 0.553, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 500, "edu_l1_label": "EDU_O"}, {"txt": "Proc., 2020, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.553, 0.106, 0.678, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 501, "edu_l1_label": "EDU_O"}, {"txt": "Amber Stubbs and Özlem Uzuner.Annotating longitudinal clinical narratives for de-identifcation:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.127, 0.84, 0.142]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 502, "edu_l1_label": "EDU_O"}, {"txt": "The2014 i2b2/uthealth corpus.Journal of Biomedical Informatics, 58:S20–S29, 2015.ISSN 1532-0464.doi:https://doi.org/10.1016/j.jbi.2015.07.020.URL https://www.sciencedirect.com/science/article/ pii/S1532046415001823.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.84, 0.127, 0.882, 0.142], [0.134, 0.141, 0.885, 0.156], [0.134, 0.154, 0.884, 0.171], [0.134, 0.169, 0.317, 0.183]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 503, "edu_l1_label": "EDU_O"}, {"txt": "Supplement: Proceedings of the 2014 i2b2/UTHealth Shared-Tasks and Workshop on Challenges in Natural Language Processing for Clinical Data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.317, 0.169, 0.882, 0.183], [0.134, 0.182, 0.536, 0.197]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 504, "edu_l1_label": "EDU_O"}, {"txt": "Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite,Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.203, 0.882, 0.218], [0.134, 0.217, 0.884, 0.232], [0.134, 0.231, 0.885, 0.246]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 505, "edu_l1_label": "EDU_O"}, {"txt": "Huggingface’s transformers: State-of-the-art natural language processing, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.134, 0.244, 0.654, 0.26]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 506, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/1910.03771.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.654, 0.244, 0.884, 0.261], [0.133, 0.259, 0.223, 0.273]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 507, "edu_l1_label": "EDU_O"}, {"txt": "Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.279, 0.882, 0.295], [0.134, 0.293, 0.545, 0.308]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 508, "edu_l1_label": "EDU_O"}, {"txt": "Mimic-iii, a freely accessible critical care database.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.545, 0.293, 0.885, 0.308]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 509, "edu_l1_label": "EDU_O"}, {"txt": "Scientifc data, 3(1):1–9, 2016.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.134, 0.307, 0.335, 0.322]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 510, "edu_l1_label": "EDU_O"}, {"txt": "Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.328, 0.58, 0.343]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 511, "edu_l1_label": "EDU_O"}, {"txt": "Medmcqa: A large-scale multi-subject multichoice dataset for medical domain question answering.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.58, 0.328, 0.885, 0.343], [0.134, 0.342, 0.497, 0.357]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 512, "edu_l1_label": "EDU_O"}, {"txt": "In Gerardo Flores, George H Chen, Tom Pollard, Joyce C Ho, and Tristan Naumann, editors, Proceedings of the Conference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pages 248–260.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.497, 0.342, 0.882, 0.357], [0.134, 0.356, 0.882, 0.371], [0.133, 0.37, 0.603, 0.385]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 513, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 07–08 Apr 2022.URL https://proceedings.mlr.press/v174/pal22a.html.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.603, 0.37, 0.887, 0.386], [0.133, 0.384, 0.479, 0.399]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 514, "edu_l1_label": "EDU_O"}, {"txt": "Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.405, 0.627, 0.42]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 515, "edu_l1_label": "EDU_O"}, {"txt": "PubMedQA: A dataset for biomedical research question answering.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.627, 0.405, 0.882, 0.42], [0.134, 0.418, 0.329, 0.434]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 516, "edu_l1_label": "EDU_O"}, {"txt": "In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567–2577, Hong Kong, China, November2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.329, 0.418, 0.882, 0.433], [0.134, 0.432, 0.882, 0.447], [0.133, 0.446, 0.883, 0.461], [0.134, 0.46, 0.171, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 517, "edu_l1_label": "EDU_O"}, {"txt": "Association for Computational Linguistics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.171, 0.46, 0.466, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 518, "edu_l1_label": "EDU_O"}, {"txt": "doi:10.18653/v1/D19-1259.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.466, 0.46, 0.661, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 519, "edu_l1_label": "EDU_O"}, {"txt": "URL https://aclanthology.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.661, 0.46, 0.888, 0.476]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 520, "edu_l1_label": "EDU_O"}, {"txt": "org/D19-1259.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.134, 0.475, 0.241, 0.489]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 521, "edu_l1_label": "EDU_O"}, {"txt": "Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, DanielTruhn, and Keno K. Bressem.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.495, 0.882, 0.51], [0.133, 0.508, 0.335, 0.524]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 522, "edu_l1_label": "EDU_O"}, {"txt": "Medalpaca – an open-source collection of medical conversational ai models andtraining data, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.335, 0.508, 0.882, 0.524], [0.134, 0.522, 0.261, 0.537]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 523, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2304.08247.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.261, 0.522, 0.581, 0.537]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 524, "edu_l1_label": "EDU_O"}, {"txt": "Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.543, 0.683, 0.559]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 525, "edu_l1_label": "EDU_O"}, {"txt": "Pmc-llama: Towards building open-source language models for medicine, 2023b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.683, 0.543, 0.882, 0.559], [0.134, 0.557, 0.467, 0.572]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 526, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2304.14454.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.467, 0.557, 0.787, 0.572]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 527, "edu_l1_label": "EDU_O"}, {"txt": "Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfeld, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Nenad Tomasev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.578, 0.883, 0.594], [0.134, 0.592, 0.884, 0.607], [0.134, 0.606, 0.884, 0.621], [0.134, 0.62, 0.884, 0.635], [0.133, 0.634, 0.427, 0.649]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 528, "edu_l1_label": "EDU_O"}, {"txt": "Towards expert-level medical question answering with large language models, 2023a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.427, 0.634, 0.882, 0.649], [0.134, 0.647, 0.233, 0.662]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 529, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2305.09617.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.233, 0.647, 0.553, 0.662]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 530, "edu_l1_label": "EDU_O"}, {"txt": "Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfeld, Dina Demner-Fushman, Blaise Agüera y Arcas, DaleWebster, Greg S Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.669, 0.884, 0.684], [0.134, 0.682, 0.884, 0.697], [0.134, 0.696, 0.882, 0.711], [0.133, 0.71, 0.884, 0.725], [0.134, 0.724, 0.672, 0.739]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 531, "edu_l1_label": "EDU_O"}, {"txt": "Large language models encode clinical knowledge.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.672, 0.724, 0.882, 0.739], [0.134, 0.737, 0.261, 0.753]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 532, "edu_l1_label": "EDU_O"}, {"txt": "Nature, 620(7972):172–180, August 2023b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.261, 0.737, 0.553, 0.753]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 533, "edu_l1_label": "EDU_O"}, {"txt": "Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi,Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexandre Sallinen, Alireza Sakhaeirad,Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, and Antoine Bosselut.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.759, 0.884, 0.774], [0.134, 0.772, 0.884, 0.788], [0.133, 0.786, 0.882, 0.801], [0.134, 0.8, 0.325, 0.815]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 534, "edu_l1_label": "EDU_O"}, {"txt": "Meditron-70b: Scaling medical pretraining for large language models, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.325, 0.8, 0.843, 0.815]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 535, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2311.16079.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.843, 0.8, 0.883, 0.815], [0.134, 0.815, 0.411, 0.829]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 536, "edu_l1_label": "EDU_O"}, {"txt": "Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.835, 0.72, 0.85]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 537, "edu_l1_label": "EDU_O"}, {"txt": "What disease does this patient have?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.72, 0.835, 0.882, 0.85], [0.134, 0.849, 0.223, 0.864]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 538, "edu_l1_label": "EDU_O"}, {"txt": "a large-scale open domain question answering dataset from medical exams, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.223, 0.849, 0.787, 0.864]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 539, "edu_l1_label": "EDU_O"}, {"txt": "URL https: //arxiv.org/abs/2009.13081.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.787, 0.849, 0.887, 0.865], [0.133, 0.864, 0.359, 0.878]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 540, "edu_l1_label": "EDU_O"}, {"txt": "Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.118, 0.884, 0.81, 0.899]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 541, "edu_l1_label": "EDU_O"}, {"txt": "Measuring massive multitask language understanding, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.81, 0.884, 0.882, 0.899], [0.134, 0.897, 0.454, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 542, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2009.03300.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.454, 0.897, 0.774, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 543, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.49, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 544, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.268, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 545, "edu_l1_label": "EDU_O"}, {"txt": "Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.092, 0.652, 0.107]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 546, "edu_l1_label": "EDU_O"}, {"txt": "Memorization without overftting: Analyzing the training dynamics of large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.652, 0.092, 0.885, 0.107], [0.133, 0.106, 0.513, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 547, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 35: 38274–38290, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.513, 0.106, 0.885, 0.121], [0.134, 0.119, 0.268, 0.135]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 548, "edu_l1_label": "EDU_O"}, {"txt": "Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.138, 0.882, 0.153], [0.134, 0.151, 0.476, 0.167]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 549, "edu_l1_label": "EDU_O"}, {"txt": "The llama 3 herd of models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.476, 0.151, 0.663, 0.167]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 550, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2407.21783, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.663, 0.151, 0.884, 0.167], [0.134, 0.165, 0.171, 0.18]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 551, "edu_l1_label": "EDU_O"}, {"txt": "Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.183, 0.884, 0.199], [0.134, 0.197, 0.884, 0.212], [0.134, 0.211, 0.882, 0.226], [0.134, 0.225, 0.883, 0.24], [0.134, 0.238, 0.884, 0.254], [0.134, 0.252, 0.883, 0.267], [0.134, 0.266, 0.884, 0.281], [0.134, 0.28, 0.882, 0.295], [0.134, 0.294, 0.882, 0.309], [0.134, 0.307, 0.569, 0.323]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 552, "edu_l1_label": "EDU_O"}, {"txt": "Llama 2: Open foundation and fne-tuned chat models, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.569, 0.307, 0.882, 0.323], [0.134, 0.321, 0.226, 0.336]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 553, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2307.09288.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.226, 0.321, 0.546, 0.336]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 554, "edu_l1_label": "EDU_O"}, {"txt": "Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux,Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.339, 0.884, 0.355], [0.134, 0.353, 0.884, 0.368], [0.134, 0.367, 0.807, 0.382]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 555, "edu_l1_label": "EDU_O"}, {"txt": "Mistral 7b,2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.807, 0.367, 0.884, 0.382], [0.134, 0.381, 0.171, 0.396]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 556, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2310.06825.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.171, 0.381, 0.491, 0.396]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 557, "edu_l1_label": "EDU_O"}, {"txt": "Matthew Jagielski, Om Thakkar, Florian Tramèr, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace,Shuang Song, Abhradeep Thakurta, Nicolas Papernot, and Chiyuan Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.399, 0.884, 0.414], [0.134, 0.413, 0.636, 0.428]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 558, "edu_l1_label": "EDU_O"}, {"txt": "Measuring forgetting of memorized training examples, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.636, 0.413, 0.882, 0.428], [0.134, 0.426, 0.295, 0.442]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 559, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2207.00099.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.295, 0.426, 0.615, 0.442]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 560, "edu_l1_label": "EDU_O"}, {"txt": "Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.445, 0.882, 0.46], [0.134, 0.458, 0.184, 0.474]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 561, "edu_l1_label": "EDU_O"}, {"txt": "Deduplicating training data makes language models better, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.184, 0.458, 0.615, 0.474]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 562, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2107.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.615, 0.458, 0.888, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 563, "edu_l1_label": "EDU_O"}, {"txt": "06499.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.134, 0.473, 0.181, 0.487]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 564, "edu_l1_label": "EDU_O"}, {"txt": "Nikhil Kandpal, Eric Wallace, and Colin Raffel.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.49, 0.424, 0.506]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 565, "edu_l1_label": "EDU_O"}, {"txt": "Deduplicating training data mitigates privacy risks in language models, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.424, 0.49, 0.884, 0.506], [0.134, 0.504, 0.171, 0.519]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 566, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2202.06539.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.171, 0.504, 0.491, 0.519]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 567, "edu_l1_label": "EDU_O"}, {"txt": "Arnaud Grivet Sébert, Renaud Sirdey, Oana Stan, and Cédric Gouy-Pailler.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.522, 0.597, 0.538]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 568, "edu_l1_label": "EDU_O"}, {"txt": "Protecting data from all parties: Combiningfhe and dp in federated learning, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.597, 0.522, 0.882, 0.538], [0.134, 0.536, 0.386, 0.551]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 569, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2205.04330.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.386, 0.536, 0.706, 0.551]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 14, "global_sentence_id": 570, "edu_l1_label": "EDU_O"}, {"txt": "Lattigo v6.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.554, 0.19, 0.57]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 571, "edu_l1_label": "EDU_O"}, {"txt": "Lattigo open-source repository.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.19, 0.554, 0.403, 0.57]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 572, "edu_l1_label": "EDU_O"}, {"txt": "Online: https://github.com/tuneinsight/lattigo, August 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.403, 0.554, 0.885, 0.57]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 573, "edu_l1_label": "EDU_O"}, {"txt": "EPFL-LDS, Tune Insight SA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.134, 0.568, 0.327, 0.583]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 16, "global_sentence_id": 574, "edu_l1_label": "EDU_O"}, {"txt": "Christian Vincent Mouchet, Jean-Philippe Bossuat, Juan Ramón Troncoso-Pastoriza, and Jean-Pierre Hubaux.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.586, 0.812, 0.602]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 575, "edu_l1_label": "EDU_O"}, {"txt": "Lattigo: Amultiparty homomorphic encryption library in go.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.812, 0.586, 0.883, 0.602], [0.134, 0.6, 0.449, 0.615]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 576, "edu_l1_label": "EDU_O"}, {"txt": "In 8th Workshop on Encrypted Computing & Applied Homomorphic Cryptography (WAHC 2020), page 64–70, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.449, 0.6, 0.882, 0.615], [0.133, 0.614, 0.453, 0.629]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 577, "edu_l1_label": "EDU_O"}, {"txt": "ISBN 978-3-000677-98-4. doi:10.25835/0072999.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.453, 0.614, 0.793, 0.629]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 578, "edu_l1_label": "EDU_O"}, {"txt": "URL https://infoscience.epfl.ch/handle/20.500.14299/193451.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.793, 0.614, 0.887, 0.63], [0.133, 0.629, 0.547, 0.643]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 579, "edu_l1_label": "EDU_O"}, {"txt": "Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.646, 0.554, 0.661]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 580, "edu_l1_label": "EDU_O"}, {"txt": "Large language models can be strong differentially private learners.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.554, 0.646, 0.883, 0.661], [0.134, 0.66, 0.238, 0.675]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 581, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2110.05679, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.238, 0.66, 0.505, 0.675]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 582, "edu_l1_label": "EDU_O"}, {"txt": "Peter L Bartlett, Philip M Long, Gábor Lugosi, and Alexander Tsigler.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.678, 0.608, 0.693]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 583, "edu_l1_label": "EDU_O"}, {"txt": "Benign overftting in linear regression.Proceedings of the National Academy of Sciences, 117(48):30063–30070, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.608, 0.678, 0.885, 0.693], [0.133, 0.692, 0.655, 0.707]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 11, "global_sentence_id": 584, "edu_l1_label": "EDU_O"}, {"txt": "Qiaoyue Tang, Frederick Shpilevskiy, and Mathias Lécuyer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.71, 0.504, 0.725]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 585, "edu_l1_label": "EDU_O"}, {"txt": "Dp-adambc: Your dp-adam is actually dp-sgd (unless youapply bias correction), 2023a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.504, 0.71, 0.882, 0.725], [0.134, 0.724, 0.328, 0.739]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 586, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2312.14334.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.328, 0.724, 0.648, 0.739]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 10, "global_sentence_id": 587, "edu_l1_label": "EDU_O"}, {"txt": "Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian Stich, and Martin Jaggi.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.742, 0.629, 0.757]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 588, "edu_l1_label": "EDU_O"}, {"txt": "Error feedback fxes signsgd and other gradient compression schemes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.629, 0.742, 0.883, 0.757], [0.134, 0.756, 0.336, 0.771]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 589, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pages 3252–3261.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.336, 0.756, 0.793, 0.771]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 590, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.793, 0.756, 0.885, 0.771]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 13, "global_sentence_id": 591, "edu_l1_label": "EDU_O"}, {"txt": "Ron Dorfman, Shay Vargaftik, Yaniv Ben-Itzhak, and Kfr Yehuda Levy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.774, 0.574, 0.789]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 592, "edu_l1_label": "EDU_O"}, {"txt": "Docof: Downlink compression for cross-device federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.574, 0.774, 0.882, 0.789], [0.134, 0.788, 0.256, 0.803]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 593, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pages 8356–8388.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.256, 0.788, 0.715, 0.803]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 594, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.715, 0.788, 0.809, 0.803]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 12, "global_sentence_id": 595, "edu_l1_label": "EDU_O"}, {"txt": "Tahseen Rabbani, Brandon Feng, Yifan Yang, Arjun Rajkumar, Amitabh Varshney, and Furong Huang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.806, 0.807, 0.821]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 596, "edu_l1_label": "EDU_O"}, {"txt": "Comfetch: Federated learning of large networks on memory-constrained clients via sketching.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.807, 0.806, 0.885, 0.821], [0.134, 0.82, 0.662, 0.835]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 597, "edu_l1_label": "EDU_O"}, {"txt": "arXiv e-prints, pages arXiv–2109,2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.662, 0.82, 0.884, 0.835], [0.134, 0.833, 0.171, 0.849]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 9, "global_sentence_id": 598, "edu_l1_label": "EDU_O"}, {"txt": "Nikita Ivkin, Daniel Rothchild, Enayat Ullah, Ion Stoica, Raman Arora, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.852, 0.616, 0.867]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 599, "edu_l1_label": "EDU_O"}, {"txt": "Communication-effcient distributed sgd with sketching.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.616, 0.852, 0.882, 0.867], [0.133, 0.865, 0.233, 0.881]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 600, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 32, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.233, 0.865, 0.649, 0.881]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 15, "global_sentence_id": 601, "edu_l1_label": "EDU_O"}, {"txt": "Ashok Vardhan Makkuva, Marco Bondaschi, Thijs Vogels, Martin Jaggi, Hyeji Kim, and Michael C Gastpar.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.118, 0.884, 0.837, 0.899]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 602, "edu_l1_label": "EDU_O"}, {"txt": "Laser: Linear compression in wireless distributed optimization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.837, 0.884, 0.885, 0.899], [0.134, 0.897, 0.501, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 603, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2310.13033, 2023.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.501, 0.897, 0.767, 0.913]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 17, "global_sentence_id": 604, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.492, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 605, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 606, "edu_l1_label": "EDU_O"}, {"txt": "Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.092, 0.547, 0.107]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 607, "edu_l1_label": "EDU_O"}, {"txt": "Bert: Pre-training of deep bidirectional transformers for language understanding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.547, 0.092, 0.882, 0.107], [0.134, 0.106, 0.316, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 608, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of naacL-HLT, volume 1, page 2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.316, 0.106, 0.641, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 609, "edu_l1_label": "EDU_O"}, {"txt": "Minneapolis, Minnesota, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.641, 0.106, 0.848, 0.121]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 5, "global_sentence_id": 610, "edu_l1_label": "EDU_O"}, {"txt": "Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary C Lipton, and J Zico Kolter.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.124, 0.681, 0.14]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 611, "edu_l1_label": "EDU_O"}, {"txt": "Rethinking llm memorization through the lens of adversarial compression.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.681, 0.124, 0.882, 0.14], [0.134, 0.138, 0.422, 0.153]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 612, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2404.15146, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.422, 0.138, 0.689, 0.153]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 7, "global_sentence_id": 613, "edu_l1_label": "EDU_O"}, {"txt": "Yiming Li, Mingyan Zhu, Xue Yang, Yong Jiang, Tao Wei, and Shu-Tao Xia.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.157, 0.614, 0.172]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 614, "edu_l1_label": "EDU_O"}, {"txt": "Black-box dataset ownership verifcation via backdoor watermarking, 2023b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.614, 0.157, 0.882, 0.172], [0.134, 0.171, 0.365, 0.186]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 615, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2209.06015.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.365, 0.171, 0.685, 0.186]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 6, "global_sentence_id": 616, "edu_l1_label": "EDU_O"}, {"txt": "Ruixiang Tang, Qizhang Feng, Ninghao Liu, Fan Yang, and Xia Hu.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.19, 0.576, 0.205]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 617, "edu_l1_label": "EDU_O"}, {"txt": "Did you train on my dataset?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.576, 0.19, 0.778, 0.205]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 618, "edu_l1_label": "EDU_O"}, {"txt": "towards public dataset protection with clean-label backdoor watermarking, 2023b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.778, 0.19, 0.882, 0.205], [0.134, 0.203, 0.565, 0.219]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 619, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2303.11470.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.565, 0.203, 0.885, 0.219]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 4, "global_sentence_id": 620, "edu_l1_label": "EDU_O"}, {"txt": "Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, Yue Xing, and Jiliang Tang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.222, 0.751, 0.237]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 621, "edu_l1_label": "EDU_O"}, {"txt": "Diffusionshield: A watermark for copyright protection against generative diffusion models, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.751, 0.222, 0.883, 0.237], [0.133, 0.236, 0.652, 0.251]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 622, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/ 2306.04642.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.652, 0.236, 0.884, 0.252], [0.134, 0.251, 0.223, 0.265]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 623, "edu_l1_label": "EDU_O"}, {"txt": "Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.269, 0.564, 0.284]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 624, "edu_l1_label": "EDU_O"}, {"txt": "Membership Inference Attacks Against Machine Learning Models .", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.564, 0.269, 0.882, 0.284], [0.134, 0.282, 0.255, 0.297]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 625, "edu_l1_label": "EDU_O"}, {"txt": "In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18, Los Alamitos, CA, USA,May 2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.255, 0.282, 0.884, 0.297], [0.134, 0.296, 0.204, 0.311]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 626, "edu_l1_label": "EDU_O"}, {"txt": "IEEE Computer Society.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.296, 0.368, 0.311]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 627, "edu_l1_label": "EDU_O"}, {"txt": "doi:10.1109/SP.2017.41.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.368, 0.296, 0.534, 0.311]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 628, "edu_l1_label": "EDU_O"}, {"txt": "URL https://doi.ieeecomputersociety.org/ 10.1109/SP.2017.41.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.534, 0.296, 0.884, 0.312], [0.133, 0.311, 0.291, 0.325]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 629, "edu_l1_label": "EDU_O"}, {"txt": "Chang Hongyan, Shahin Shamsabadi Ali, Katevas Kleomenis, Haddadi Hamed, and Shokri Reza.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.329, 0.776, 0.344]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 630, "edu_l1_label": "EDU_O"}, {"txt": "Context-aware membership inference attacks against pre-trained large language models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.776, 0.329, 0.882, 0.344], [0.134, 0.342, 0.615, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 631, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2409.13745, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.615, 0.342, 0.885, 0.358]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 632, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/2409.13745.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.134, 0.356, 0.448, 0.371]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 633, "edu_l1_label": "EDU_O"}, {"txt": "Sherri Truex, Nathalie Baracaldo, Anjum Anwar, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.375, 0.481, 0.39]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 634, "edu_l1_label": "EDU_O"}, {"txt": "A hybrid approach to privacy-preserving federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.481, 0.375, 0.885, 0.39]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 635, "edu_l1_label": "EDU_O"}, {"txt": "Informatik Spektrum, 42:356–357, October 2019. doi:10.1007/s00287-019-01205-x.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.134, 0.389, 0.683, 0.404]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 636, "edu_l1_label": "EDU_O"}, {"txt": "URL https://doi.org/10.1007/s00287-019-01205-x.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.683, 0.389, 0.888, 0.405], [0.133, 0.404, 0.337, 0.418]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 637, "edu_l1_label": "EDU_O"}, {"txt": "Published: 30 August 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.337, 0.404, 0.524, 0.418]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 2, "global_sentence_id": 638, "edu_l1_label": "EDU_O"}, {"txt": "Ilya Loshchilov and Frank Hutter.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.118, 0.421, 0.342, 0.437]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 639, "edu_l1_label": "EDU_O"}, {"txt": "Decoupled weight decay regularization, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.342, 0.421, 0.653, 0.437]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 640, "edu_l1_label": "EDU_O"}, {"txt": "URL https://arxiv.org/abs/ 1711.05101.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.653, 0.421, 0.884, 0.438], [0.133, 0.436, 0.223, 0.45]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 8, "global_sentence_id": 641, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.49, 0.935, 0.509, 0.95]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 642, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.269, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 643, "edu_l1_label": "EDU_O"}, {"txt": "AFurther Related Work", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.118, 0.089, 0.339, 0.109]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 644, "edu_l1_label": "EDU_O"}, {"txt": "Membership inference attacks (MIA) rely on rigorous statistical principles to assess privacy risks in machine learning models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.118, 0.121, 0.882, 0.136], [0.118, 0.135, 0.168, 0.15]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 645, "edu_l1_label": "EDU_O"}, {"txt": "[Shokri et al., 2017] introduced an approach for determining whether a specifc data point was part of a model’straining dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.168, 0.135, 0.882, 0.15], [0.118, 0.149, 0.225, 0.164]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 646, "edu_l1_label": "EDU_O"}, {"txt": "These attacks exploit differences in model behavior on training versus non-training data, posing signifcant privacy concerns for sensitive information.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.225, 0.149, 0.882, 0.164], [0.118, 0.163, 0.462, 0.178]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 647, "edu_l1_label": "EDU_O"}, {"txt": "Building on this, [Hongyan et al., 2024] extended these concepts to LLMs by incorporating contextual information.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.462, 0.163, 0.882, 0.178], [0.118, 0.176, 0.453, 0.192]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 648, "edu_l1_label": "EDU_O"}, {"txt": "This study demonstrated that LLMs are particularly vulnerable to membership inference attacks, as they often retain verbatim information from their training datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.453, 0.176, 0.882, 0.192], [0.118, 0.19, 0.811, 0.205]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 649, "edu_l1_label": "EDU_O"}, {"txt": "The work highlighted the increased privacy risks associated with LLMs due to their scale and training dynamics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.811, 0.19, 0.883, 0.205], [0.118, 0.204, 0.784, 0.219]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 650, "edu_l1_label": "EDU_O"}, {"txt": "Secure Aggregations.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.118, 0.224, 0.268, 0.241]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 651, "edu_l1_label": "EDU_O"}, {"txt": "While the conventional FL ensures that raw data is not shared between participants during collective training, it does not address the risk of data leakage through model updates shared prior to aggregation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.268, 0.224, 0.882, 0.24], [0.118, 0.238, 0.855, 0.254]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 652, "edu_l1_label": "EDU_O"}, {"txt": "For example, in the honest-but-curious scenario, a server examines whether client data can be reconstructed [Huang et al., 2021].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.855, 0.238, 0.883, 0.254], [0.118, 0.252, 0.884, 0.267], [0.118, 0.266, 0.159, 0.281]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 653, "edu_l1_label": "EDU_O"}, {"txt": "This vulnerability becomes particularly critical with LLMs, given their propensity for memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.159, 0.266, 0.809, 0.281]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 654, "edu_l1_label": "EDU_O"}, {"txt": "To address the privacy risks associated with local model exchanges in FL, [Truex et al., 2019] proposes a hybrid approach that combines differential privacy with secure multiparty computation (SMC).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.809, 0.266, 0.882, 0.281], [0.118, 0.28, 0.882, 0.295], [0.118, 0.293, 0.586, 0.309]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 655, "edu_l1_label": "EDU_O"}, {"txt": "In this framework, local models are encrypted and remain hidden from other participants prior to aggregation, thereby mitigating privacy leakage risks associated with individual local models by focusing them on the aggregated model during each aggregation round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.586, 0.293, 0.882, 0.309], [0.118, 0.307, 0.882, 0.322], [0.117, 0.321, 0.807, 0.336]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 656, "edu_l1_label": "EDU_O"}, {"txt": "While this method has been explored for general machine learning applications, to the best of our knowledge, it has not yet been investigated in the context of large language models (LLMs).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.807, 0.321, 0.882, 0.336], [0.118, 0.335, 0.882, 0.35], [0.118, 0.349, 0.515, 0.364]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 657, "edu_l1_label": "EDU_O"}, {"txt": "BTraining details", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13100000, "bbox": [[0.118, 0.382, 0.282, 0.402]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 9, "global_sentence_id": 658, "edu_l1_label": "IOS"}, {"txt": "B.1Hyperparameters", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13200000, "bbox": [[0.118, 0.414, 0.281, 0.43]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 659, "edu_l1_label": "EDU_O"}, {"txt": "In centralized learning, we sweep the learning rate ∈{1e −5, 5e −5, 1e −4, 5e −4} for full fne-tuning experiments.For LoRA experiments, we search for learning rate values ∈{5e −5, 1e −4, 5e −4, 1e −3}.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13300000, "bbox": [[0.118, 0.44, 0.885, 0.455], [0.118, 0.453, 0.739, 0.468]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 660, "edu_l1_label": "EDU_O"}, {"txt": "In federated learningexperiments, we sweep the learning rate on each dataset individually for one epoch, with the same set of values as in centralized learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13300000, "bbox": [[0.739, 0.453, 0.882, 0.468], [0.118, 0.467, 0.882, 0.482], [0.118, 0.481, 0.251, 0.496]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 661, "edu_l1_label": "EDU_O"}, {"txt": "For all experiments we fne-tune models with the AdamW optimizer [Loshchilov and Hutter, 2019] with default parameters (β1 = 0.9, β2 = 0.999, ϵ = 1e−8, weight decay of 0.01).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.118, 0.502, 0.882, 0.517], [0.118, 0.515, 0.571, 0.53]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 662, "edu_l1_label": "EDU_O"}, {"txt": "We used a context length of 1024 and ensuredthat no text inputs were longer than the context length.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.571, 0.515, 0.882, 0.53], [0.118, 0.529, 0.476, 0.544]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 663, "edu_l1_label": "EDU_O"}, {"txt": "We use a linear warmup of 100 steps with a cosine annealing schedule.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.476, 0.529, 0.882, 0.544], [0.118, 0.543, 0.18, 0.558]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 664, "edu_l1_label": "EDU_O"}, {"txt": "Unless mentioned otherwise, we use a global batch size of 32 with gradient accumulation and gradient checkpointing.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.18, 0.543, 0.882, 0.558], [0.118, 0.557, 0.215, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 665, "edu_l1_label": "EDU_O"}, {"txt": "For all LoRA experiments with use a rank of 16, an alpha of 8, drop out 0.05 and use adapters for all projection layers.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.215, 0.557, 0.882, 0.572], [0.118, 0.57, 0.231, 0.586]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 666, "edu_l1_label": "EDU_O"}, {"txt": "Additionally, we study the impact of the LoRA rank on memorization in Section B.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.231, 0.57, 0.791, 0.586]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 667, "edu_l1_label": "EDU_O"}, {"txt": "B.2The LoRA rank and memorization", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13500000, "bbox": [[0.118, 0.601, 0.399, 0.618]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 668, "edu_l1_label": "EDU_O"}, {"txt": "We measure the infuence of the LoRA hyperparameters by varying the rank and measuring the resulting memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.117, 0.627, 0.885, 0.642]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 669, "edu_l1_label": "EDU_O"}, {"txt": "We study rank values r ∈{4, 16, 64, 128, 256, 1024} and set alpha to twice the rank, following common practice.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.117, 0.641, 0.856, 0.656]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 670, "edu_l1_label": "EDU_O"}, {"txt": "Wedecrease the learning rate exponentially as the rank increase.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.856, 0.641, 0.882, 0.656], [0.118, 0.655, 0.512, 0.67]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 671, "edu_l1_label": "EDU_O"}, {"txt": "Table 1:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.117, 0.681, 0.169, 0.697]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 11, "global_sentence_id": 672, "edu_l1_label": "EDU_O"}, {"txt": " Impact of the LoRA rank on memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.169, 0.681, 0.476, 0.697]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 11, "global_sentence_id": 673, "edu_l1_label": "EDU_O"}, {"txt": "We fne-tune Llama 3.2 3B with LoRA in centralized learning on increasing LoRA ranks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.476, 0.681, 0.882, 0.697], [0.118, 0.695, 0.293, 0.71]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 11, "global_sentence_id": 674, "edu_l1_label": "EDU_O"}, {"txt": "We fnd that higher ranks lead to more memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.293, 0.695, 0.646, 0.71]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 11, "global_sentence_id": 675, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13800000, "bbox": [[0.178, 0.731, 0.822, 0.86]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 676, "edu_l1_label": "EDU_O"}, {"txt": "As shown in Table 1, increasing the rank, i.e. increasing the number of weights updated during fne-tuning, results in more memorization, ranging from virtually no verbatim memorization with a rank of 4 to almost 50% of the medical records being memorized for rank 1024 when considering duplicated medical records. We note that in our case, larger", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13900000, "bbox": [[0.112, 0.87, 0.886, 0.915]]}]}, "tags": ["table_caption"], "label": "content", "web_segment_id": 10, "global_sentence_id": 677, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.492, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 678, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.27, 0.045, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 679, "edu_l1_label": "EDU_O"}, {"txt": "ranks do not necessarily imply better accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.118, 0.092, 0.42, 0.107]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 680, "edu_l1_label": "EDU_O"}, {"txt": "We hypothesize that larger ranks might make overftting more likely to occur.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.42, 0.092, 0.882, 0.107], [0.118, 0.106, 0.157, 0.121]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 681, "edu_l1_label": "EDU_O"}, {"txt": "Additionally, each rank value can beneft from more extensive hyperparameter tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.157, 0.106, 0.722, 0.121]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 682, "edu_l1_label": "EDU_O"}, {"txt": "C Auxiliary results", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14100000, "bbox": [[0.118, 0.14, 0.132, 0.16], [0.151, 0.14, 0.289, 0.16]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 683, "edu_l1_label": "IOS"}, {"txt": "C.1 Accuracy", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14200000, "bbox": [[0.118, 0.172, 0.142, 0.189], [0.158, 0.172, 0.224, 0.189]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 9, "global_sentence_id": 684, "edu_l1_label": "EDU_O"}, {"txt": "Table 2 includes a breakdown per benchmark of the downstream accuracy of LoRA and full model fne-tuning in centralized learning as well as performance of pre-trained models without fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.117, 0.198, 0.882, 0.214], [0.118, 0.212, 0.681, 0.227]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 685, "edu_l1_label": "EDU_O"}, {"txt": "Table 3 shows the accuracy of federated fne-tuning per round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.681, 0.212, 0.882, 0.227], [0.118, 0.226, 0.325, 0.241]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 686, "edu_l1_label": "EDU_O"}, {"txt": "Table 2: Downstream accuracy in central learning. Best accuracy values are marked in bold.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.187, 0.253, 0.812, 0.269]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 12, "global_sentence_id": 687, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14500000, "bbox": [[0.115, 0.299, 0.899, 0.519]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 0, "global_sentence_id": 688, "edu_l1_label": "EDU_O"}, {"txt": "Table 3: Downstream accuracy per federated round. We emphasize in bold the earliest round where models reach their best accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.112, 0.535, 0.885, 0.566]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 1, "global_sentence_id": 689, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14700000, "bbox": [[0.255, 0.584, 0.746, 0.761]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 690, "edu_l1_label": "EDU_O"}, {"txt": "C.2Memorization Score", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.118, 0.783, 0.299, 0.8]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 691, "edu_l1_label": "EDU_O"}, {"txt": "Figure 7 illustrates with Llama 2 7B multiple trends that are consistent with results previously mentioned:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.118, 0.809, 0.807, 0.825]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 11, "global_sentence_id": 692, "edu_l1_label": "EDU_O"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.156, 0.836, 0.168, 0.851]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 693, "edu_l1_label": "EDU_O"}, {"txt": "There is signifcantly, and alarmingly, more memorization when the medical records occur multiple times in the fne-tuning data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.168, 0.836, 0.882, 0.851], [0.176, 0.849, 0.308, 0.865]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 694, "edu_l1_label": "EDU_O"}, {"txt": "2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15100000, "bbox": [[0.156, 0.869, 0.168, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 695, "edu_l1_label": "EDU_O"}, {"txt": "Longer prompts show higher memorization (discoverability phenomenon).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15100000, "bbox": [[0.168, 0.869, 0.66, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 696, "edu_l1_label": "EDU_O"}, {"txt": "3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15200000, "bbox": [[0.156, 0.889, 0.168, 0.904]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 697, "edu_l1_label": "EDU_O"}, {"txt": "There is signifcantly more memorization with approximate generation (BLEU score).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15200000, "bbox": [[0.168, 0.889, 0.732, 0.904]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 698, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.491, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 699, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.271, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 700, "edu_l1_label": "EDU_O"}, {"txt": "Documents duplicated 10x", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15400000, "bbox": [[0.547, 0.095, 0.784, 0.11]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 11, "global_sentence_id": 701, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15500000, "bbox": [[0.182, 0.092, 0.815, 0.343]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 4, "global_sentence_id": 702, "edu_l1_label": "EDU_O"}, {"txt": "Figure 7: An example of memorization scores for a full fne-tuning of Llama 2 7B. We report the exact match rate and BLEU score with respect to the prompt length, with and without duplication. We also show the memorization upper bound (\"Full memorization\") reached when every test sequence has been memorized.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.115, 0.356, 0.889, 0.401]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 2, "global_sentence_id": 703, "edu_l1_label": "EDU_O"}, {"txt": "C.3Memorization scores in FL", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.118, 0.426, 0.345, 0.442]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 9, "global_sentence_id": 704, "edu_l1_label": "EDU_O"}, {"txt": "Figure 8 shows the memorization scores per round of federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.118, 0.452, 0.589, 0.467]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 705, "edu_l1_label": "EDU_O"}, {"txt": "We can see that using LoRA results in lower unintended memorization than full fne-tuning at every round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.589, 0.452, 0.883, 0.467], [0.118, 0.466, 0.52, 0.481]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 706, "edu_l1_label": "EDU_O"}, {"txt": "DSecure Aggregations", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.118, 0.5, 0.323, 0.52]]}]}, "tags": ["title"], "label": "content", "web_segment_id": 7, "global_sentence_id": 707, "edu_l1_label": "EDU_O"}, {"txt": "Secure aggregations ensure that sensitive data remains protected and prevents the aggregator from decrypting any model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.118, 0.534, 0.885, 0.549]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 708, "edu_l1_label": "EDU_O"}, {"txt": "We evaluate the runtime performance of using secure aggregation in conjunction with LoRA in an FL setting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.117, 0.547, 0.83, 0.562]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 709, "edu_l1_label": "EDU_O"}, {"txt": "Performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.118, 0.568, 0.212, 0.584]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 710, "edu_l1_label": "EDU_O"}, {"txt": "To evaluate the performance impact of secure aggregation, we use Lattigo, an open-source library that enables secure protocols based on multiparty homomorphic encryption Lattigo v6, Mouchet et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.212, 0.568, 0.882, 0.583], [0.118, 0.582, 0.749, 0.597]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 711, "edu_l1_label": "EDU_O"}, {"txt": "[2020].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.749, 0.582, 0.8, 0.597]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 712, "edu_l1_label": "EDU_O"}, {"txt": "Specifcally, it implements the CKKS scheme, which allows effcient encrypted computations on real-valued data, making it ideal for the secure aggregation of the LoRA models trained by the clients/participants.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.8, 0.582, 0.884, 0.597], [0.118, 0.596, 0.883, 0.611], [0.118, 0.609, 0.615, 0.624]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 713, "edu_l1_label": "EDU_O"}, {"txt": "In our experiments, we consider 3 clients and confgure CKKS parameters to enable 32-bit precision.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.615, 0.609, 0.882, 0.624], [0.118, 0.623, 0.504, 0.638]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 714, "edu_l1_label": "EDU_O"}, {"txt": "Since our LoRA models are trained with 16-bit precision, this ensures that secure aggregation does not introduce any accuracy loss compared to standard aggregation in plaintext.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.504, 0.623, 0.884, 0.638], [0.118, 0.637, 0.882, 0.652], [0.118, 0.651, 0.178, 0.666]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 715, "edu_l1_label": "EDU_O"}, {"txt": "Secure aggregation introduces a time overhead due to encryption, homomorphic operations, and collective decryption.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.118, 0.671, 0.885, 0.687]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 716, "edu_l1_label": "EDU_O"}, {"txt": "The duration of encrypted aggregation is infuenced by the number of weights being aggregated, specifcally the number of LoRA weights.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.117, 0.685, 0.882, 0.7], [0.118, 0.699, 0.289, 0.714]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 717, "edu_l1_label": "EDU_O"}, {"txt": "In our experiments with Llama 3.2 3B, a LoRA update contains 24,772,608 parameters, representing approximately 0.77% of the full model’s parameters.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.289, 0.699, 0.884, 0.715], [0.118, 0.712, 0.577, 0.728]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 718, "edu_l1_label": "EDU_O"}, {"txt": "In Table 4, we report the aggregation times forvectors of varying sizes, corresponding to the number of LoRA weights.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.577, 0.712, 0.883, 0.728], [0.117, 0.726, 0.592, 0.742]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 719, "edu_l1_label": "EDU_O"}, {"txt": "Aggregating three vectors of the size of our LoRA takes 11.33 seconds, which is negligible compared to the time required for local fne-tuning at each round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.592, 0.726, 0.883, 0.742], [0.118, 0.74, 0.854, 0.755]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 720, "edu_l1_label": "EDU_O"}, {"txt": "EGoldfsh loss", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.118, 0.775, 0.256, 0.794]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 721, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we evaluate how LoRA combined with Goldfsh loss impact the accuracy and the memorization of Llama 3.2 3B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.118, 0.808, 0.882, 0.823], [0.118, 0.822, 0.21, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 722, "edu_l1_label": "EDU_O"}, {"txt": "While Goldfsh loss has been designed for pre-training, we apply it to our fne-tuning and report values for various dropping frequencies k. We use a hashing context width h = 13 following the authors’ methodology [Hanset al., 2024].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.21, 0.822, 0.882, 0.837], [0.118, 0.835, 0.882, 0.851], [0.118, 0.849, 0.199, 0.864]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 723, "edu_l1_label": "EDU_O"}, {"txt": "Table 5 shows how combining Goldfsh loss with LoRA mitigates memorization compared to a full fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16500000, "bbox": [[0.117, 0.87, 0.885, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 724, "edu_l1_label": "EDU_O"}, {"txt": "By contrasting memorization scores with control values, we can also note that the Goldfsh loss is an effective memorization-mitigation technique.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16500000, "bbox": [[0.118, 0.884, 0.882, 0.899], [0.118, 0.897, 0.352, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 725, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.492, 0.937, 0.507, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 726, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.268, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 727, "edu_l1_label": "EDU_O"}, {"txt": "Mistral-v0.3-7B", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.443, 0.416, 0.539, 0.425]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 5, "global_sentence_id": 728, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16800000, "bbox": [[0.103, 0.101, 0.891, 0.538]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 0, "global_sentence_id": 729, "edu_l1_label": "EDU_O"}, {"txt": "Figure 8: Memorization scores for central learning and federated learning with respect to rounds. In all settings, LoRA results in better privacy than a full fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.113, 0.547, 0.885, 0.578]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 3, "global_sentence_id": 730, "edu_l1_label": "EDU_O"}, {"txt": "Table 4: Execution Time of the Secure Aggregation Protocol. The protocol aggregates three equal-sized encrypted vectors for varying sizes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.115, 0.608, 0.885, 0.638]]}]}, "tags": ["figure_caption"], "label": "table_title", "web_segment_id": 2, "global_sentence_id": 731, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17100000, "bbox": [[0.397, 0.658, 0.606, 0.785]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 4, "global_sentence_id": 732, "edu_l1_label": "EDU_O"}, {"txt": "To assess the impact of LoRA in combination with Goldfsh loss, we evaluated the memorization and accuracy of fne-tuning the same model using full fne-tuning. Table 6 presents the memorization scores and accuracy of the model fne-tuned with Goldfsh loss alone, without LoRA. Our results indicate that while Goldfsh loss reduces memorization, it does not achieve the same level of reduction as the combination with LoRA, especially when duplication occurs in the fne-tuning data. In summary, combining LoRA with Goldfsh loss allows a privacy-utility tradeoff that cannot be achieved using Goldfsh loss alone.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.115, 0.827, 0.886, 0.913]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 1, "global_sentence_id": 733, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17300000, "bbox": [[0.491, 0.938, 0.507, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 734, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17300000, "bbox": [[0.27, 0.045, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 735, "edu_l1_label": "EDU_O"}, {"txt": "Table 5: Impact of Goldfsh loss on BLEU Scores and accuracy in LoRA Fine-Tuning. Llama 3.2 3B is fne-tuned with different dropping frequencies (k). Best accuracy is marked in bold.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17400000, "bbox": [[0.115, 0.088, 0.884, 0.118]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 11, "global_sentence_id": 736, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17500000, "bbox": [[0.316, 0.137, 0.686, 0.258]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 737, "edu_l1_label": "EDU_O"}, {"txt": "Table 6: Impact of Goldfsh loss on BLEU Scores and accuracy. The BLEU scores and the accuracy of Llama 3.2 3B is reported for full fne-tuning across different dropping frequencies (k). Best accuracy is marked in bold.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.115, 0.27, 0.886, 0.301]]}]}, "tags": ["figure_caption"], "label": "table_title", "web_segment_id": 10, "global_sentence_id": 738, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17700000, "bbox": [[0.317, 0.32, 0.686, 0.441]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 739, "edu_l1_label": "EDU_O"}, {"txt": "FNEFTune", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17800000, "bbox": [[0.118, 0.465, 0.23, 0.485]]}]}, "tags": ["title"], "label": "content", "web_segment_id": 5, "global_sentence_id": 740, "edu_l1_label": "EDU_O"}, {"txt": "NEFTune is a regularization technique consisting in adding random noise to the embedding vectors to improve instruction fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.118, 0.501, 0.882, 0.517], [0.118, 0.515, 0.264, 0.53]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 741, "edu_l1_label": "EDU_O"}, {"txt": "While not introduced as a privacy-preserving technique per se, we hypothesize that a fne-tuning regularization such as NEFTune may also reduce unintended memorization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.264, 0.515, 0.882, 0.53], [0.118, 0.529, 0.612, 0.544]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 742, "edu_l1_label": "EDU_O"}, {"txt": "We display results after applying NEFTune with noise value α ∈{5, 10, 15, 30, 45}.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.117, 0.55, 0.657, 0.565]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 743, "edu_l1_label": "EDU_O"}, {"txt": "We fnd that adding noise does notimprove accuracy when applied to our domain adaptation fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.657, 0.55, 0.882, 0.565], [0.118, 0.563, 0.58, 0.579]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 744, "edu_l1_label": "EDU_O"}, {"txt": "Secondly, increasing the noise does not yield better privacy, at least not until we set alpha to 45, which is greater than alpha values reported by the original work (5, 10, and 15).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.58, 0.563, 0.882, 0.579], [0.118, 0.577, 0.884, 0.592], [0.116, 0.591, 0.194, 0.606]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 745, "edu_l1_label": "EDU_O"}, {"txt": "Table 7: NEFTune impact on the BLEU score and accuracy when combined with LoRA. We analyze LoRA fne-tuning with Llama 3.2 3B and different noise scaling factors α.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.115, 0.622, 0.885, 0.654]]}]}, "tags": ["figure_caption"], "label": "table_title", "web_segment_id": 9, "global_sentence_id": 746, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18200000, "bbox": [[0.357, 0.673, 0.645, 0.783]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 4, "global_sentence_id": 747, "edu_l1_label": "EDU_O"}, {"txt": "GDifferential Privacy", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18300000, "bbox": [[0.118, 0.819, 0.317, 0.839]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 748, "edu_l1_label": "IOS"}, {"txt": "G.1Gradient clipping", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18400000, "bbox": [[0.118, 0.855, 0.282, 0.871]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 749, "edu_l1_label": "EDU_O"}, {"txt": "Table 8 illustrates the effect of different gradient clipping values on the BLEU score and accuracy achieved during the fne-tuning of LLama 3.2 3B.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18500000, "bbox": [[0.117, 0.884, 0.882, 0.899], [0.118, 0.897, 0.309, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 750, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18600000, "bbox": [[0.491, 0.938, 0.508, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 751, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18600000, "bbox": [[0.268, 0.045, 0.733, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 752, "edu_l1_label": "EDU_O"}, {"txt": "Table 8: Gradient clipping impact on the BLEU score and accuracy. The BLEU score and the accuracy of Llama 3.2 3B is reported for LoRA fne-tuning. Best accuracy is marked in bold.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18700000, "bbox": [[0.117, 0.088, 0.885, 0.118]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 9, "global_sentence_id": 753, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18800000, "bbox": [[0.325, 0.135, 0.678, 0.332]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 8, "global_sentence_id": 754, "edu_l1_label": "EDU_O"}, {"txt": "G.2Optimizer effect on loss", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18900000, "bbox": [[0.118, 0.353, 0.323, 0.37]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 755, "edu_l1_label": "EDU_O"}, {"txt": "Figure 9 illustrates the loss reduction difference between Stochastic Gradient Descent (SGD) and Paged AdamW optimizers during the fne-tuning of Llama 3.2 3B. The SGD optimizer failed to achieve the same level of loss reduction as Paged AdamW.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19000000, "bbox": [[0.111, 0.379, 0.884, 0.422]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 6, "global_sentence_id": 756, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19100000, "bbox": [[0.304, 0.451, 0.696, 0.596]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 757, "edu_l1_label": "EDU_O"}, {"txt": "Figure 9: Loss reduction comparison between optimizers. The plot compares loss reduction during the fne-tuning of Llama 3.2 3B using different optimizers: SGD (blue) and Paged AdamW (orange).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19200000, "bbox": [[0.111, 0.608, 0.885, 0.637]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 4, "global_sentence_id": 758, "edu_l1_label": "EDU_O"}, {"txt": "HPost-fne-tuning Gaussian noise injection", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19300000, "bbox": [[0.118, 0.672, 0.492, 0.691]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 5, "global_sentence_id": 759, "edu_l1_label": "IOS"}, {"txt": "This section provides details and results of the injection of noise into the weights of a model after fne-tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.117, 0.705, 0.885, 0.72]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 760, "edu_l1_label": "EDU_O"}, {"txt": "Specifcally, the noise is sampled from a Gaussian distribution N(µ, σ2), where the mean µ is set to 0, and σ2 is thevariance that determines the noise’s magnitude.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.118, 0.718, 0.882, 0.733], [0.117, 0.732, 0.43, 0.747]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 761, "edu_l1_label": "EDU_O"}, {"txt": "Unlike the DP Gaussian mechanism, this approach does not provideformal privacy guarantees.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.43, 0.732, 0.882, 0.747], [0.118, 0.746, 0.287, 0.761]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 762, "edu_l1_label": "EDU_O"}, {"txt": "However, it offers a practical and computationally light method to mitigate the memorization of sensitive information, as it does not require additional fne-tuning and can be directly applied to previously fne-tuned LLMs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.287, 0.746, 0.882, 0.761], [0.118, 0.76, 0.882, 0.775], [0.118, 0.773, 0.163, 0.789]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 763, "edu_l1_label": "EDU_O"}, {"txt": "Additionally, measuring the performance of this method can illustrate how other noise mechanisms similar to those used in DP might affect accuracy and privacy metrics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.163, 0.773, 0.882, 0.789], [0.118, 0.787, 0.507, 0.802]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 764, "edu_l1_label": "EDU_O"}, {"txt": "In Table 9, we evaluate its effect under various noise magnitudes, along with the corresponding impact on model accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19500000, "bbox": [[0.118, 0.808, 0.882, 0.823], [0.118, 0.822, 0.178, 0.837]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 765, "edu_l1_label": "EDU_O"}, {"txt": "We applied Gaussian noise to the LoRA weights of a fne-tuned Llama 3.2 3B model, as evaluated in earlier sections.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19500000, "bbox": [[0.178, 0.822, 0.883, 0.837], [0.118, 0.835, 0.174, 0.851]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 766, "edu_l1_label": "EDU_O"}, {"txt": "We then compared the model’s BLEU score and accuracy across different noise magnitudes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19500000, "bbox": [[0.174, 0.835, 0.781, 0.851]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 767, "edu_l1_label": "EDU_O"}, {"txt": "We observe that the accuracy remains unaffected up to a certain noise level (σ = 0.01) and even shows slightimprovement.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19600000, "bbox": [[0.117, 0.856, 0.882, 0.871], [0.118, 0.87, 0.209, 0.885]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 768, "edu_l1_label": "EDU_O"}, {"txt": "However, beyond this threshold, accuracy decreases and reduction in memorization similarly follows, appearing to correlate with this decrease.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19600000, "bbox": [[0.209, 0.87, 0.884, 0.885], [0.118, 0.884, 0.381, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 769, "edu_l1_label": "EDU_O"}, {"txt": "These observations suggest that this mechanism effectively reduces excessive memorization in models that have overftted onto their training data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19600000, "bbox": [[0.381, 0.884, 0.882, 0.899], [0.118, 0.897, 0.565, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 770, "edu_l1_label": "EDU_O"}, {"txt": "Therefore, this approach offers an alternative to", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19600000, "bbox": [[0.565, 0.897, 0.882, 0.913]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 771, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19700000, "bbox": [[0.491, 0.938, 0.507, 0.947]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 772, "edu_l1_label": "EDU_O"}, {"txt": "header", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19700000, "bbox": [[0.268, 0.044, 0.734, 0.058]]}]}, "tags": ["header"], "label": "O", "web_segment_id": -1, "global_sentence_id": 773, "edu_l1_label": "EDU_O"}, {"txt": "Table 9:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19800000, "bbox": [[0.117, 0.088, 0.171, 0.103]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 7, "global_sentence_id": 774, "edu_l1_label": "EDU_O"}, {"txt": " Impact of noise addition on BLEU score and accuracy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19800000, "bbox": [[0.171, 0.088, 0.568, 0.104]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 7, "global_sentence_id": 775, "edu_l1_label": "EDU_O"}, {"txt": "Llama 3.2 3B is fne-tuned with LoRA across various noise magnitudes (σ)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19800000, "bbox": [[0.568, 0.088, 0.882, 0.103], [0.117, 0.102, 0.307, 0.117]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 7, "global_sentence_id": 776, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19900000, "bbox": [[0.299, 0.137, 0.701, 0.238]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 777, "edu_l1_label": "EDU_O"}, {"txt": "early stopping for controlling memorization which can be applied post fne-tuning. Figure 10 compares the privacy and utility of Llama 3.2 3B subject to post-fne-tuning gaussian noise injection with the evolution of the model fne-tuned with LoRA accross iterations. The noisy model, represented by red dots, has been fne-tuned for 2100 iterations before injecting the gaussian noise. Gaussian noise injection of standard deviations of σ = 0.2 and σ = 0.3 have been reportedin the plot.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20000000, "bbox": [[0.115, 0.258, 0.885, 0.331]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 5, "global_sentence_id": 778, "edu_l1_label": "EDU_O"}, {"txt": "H.1Privacy-Utility tradeoff with Gaussian noise injection", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.118, 0.344, 0.529, 0.36]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 3, "global_sentence_id": 779, "edu_l1_label": "EDU_O"}, {"txt": "Figure 10 presents a dot plot comparing the privacy-utility tradeoffs of Llama 3.2 3B when fne-tuned with LoRA versus when Gaussian noise is injected after fne-tuning with LoRA. The results indicate that Gaussian noise injection does not enhance the privacy-utility tradeoff compared to fne-tuning with LoRA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20200000, "bbox": [[0.115, 0.37, 0.885, 0.413]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 6, "global_sentence_id": 780, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20300000, "bbox": [[0.164, 0.436, 0.837, 0.715]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 781, "edu_l1_label": "EDU_O"}, {"txt": "Memorization score", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20400000, "bbox": [[0.474, 0.725, 0.62, 0.741]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 4, "global_sentence_id": 782, "edu_l1_label": "EDU_O"}, {"txt": "Figure 10: Privacy-Utility tradeoff with post-fne-tuning gaussian noise injection. Accuracy and memorization (BLEU score with 10x document duplication) tradeoff of Llama 3.2 3B subject to post-fne-tuning gaussian noise injection with standard deviation. Values above the dots correspond to the number of iterations for LoRA fne-tuning evolution, and the standard deviation of injected noise for noisy models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20500000, "bbox": [[0.114, 0.767, 0.889, 0.824]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 0, "global_sentence_id": 783, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20600000, "bbox": [[0.491, 0.937, 0.508, 0.948]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 784, "edu_l1_label": "EDU_O"}], "type": "PDF"}
# MITIGATING UNINTENDED MEMORIZATION WITH LORA IN FEDERATED LEARNING FOR LLMS
## ABSTRACT
## 1Introduction
## 2Related Work
### 2.1Privacy in LLMs
#### Differential privacy.
#### Memorization.
### 2.2Federated Learning
#### Privacy in FL.
#### Medical applications.
## 3Preliminaries
### LoRA.
### Federated Learning.
### Memorization Defnition.
## 4Empirical Evaluation
### 4.1Experimental setup
#### Fine-tuning Datasets.
##### 1.MedMCQA [Pal et al., 2022] is composed of multiple-choice questions, containing almost 190k entrance exam questions (AIIMS & NEET PG).
##### 2.PubMedQA [Jin et al., 2019] consists of Yes/No/Maybe questions created from PubMed abstracts.
##### 3.Medical Meadow fashcards [Han et al., 2023] contains 39k questions created from Anki Medical Curriculum fashcards compiled by medical students.
#### Medical Benchmarks.
#### Models.
### 4.2Quantifying memorization
#### Canaries.
#### Prompting.
#### Memorization scores.
### 4.3Centralized Learning
#### 4.3.1Utility-privacy tradeoff
### 4.4Federated Learning
#### 4.4.1 Secure Aggregations
### 4.5Combining LoRA with other methods
#### 4.5.1Goldfsh loss
#### 4.5.2Differential privacy
## 5Discussion
### Federated learning.
### LoRA.
## 6Conclusion and Limitations
## Impact statement
