{"entry_id": "9ce8f526-f717-4a74-aa1a-f3e0175a1a25", "infos": [{"txt": "PRISM:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.177, 0.098, 0.276, 0.124]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 3, "global_sentence_id": 0, "edu_l1_label": "BOT"}, {"txt": "PRIVACY-PRESERVING IMPROVED STOCHASTICMASKINGFORFEDERATED GENERATIVE MODELS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 0, "bbox": [[0.276, 0.098, 0.633, 0.124], [0.669, 0.098, 0.823, 0.124], [0.177, 0.123, 0.823, 0.149], [0.177, 0.148, 0.496, 0.175]]}]}, "tags": ["title"], "label": "article_title", "web_segment_id": 3, "global_sentence_id": 1, "edu_l1_label": "BOT"}, {"txt": "Kyeongkook Seo1, Dong-Jun Han2 ∗, Jaejun Yoo1∗", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 100000, "bbox": [[0.19, 0.196, 0.539, 0.212]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 7, "global_sentence_id": 2, "edu_l1_label": "EDU_O"}, {"txt": "1Ulsan National Institute of Science and Technology (UNIST)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 200000, "bbox": [[0.186, 0.211, 0.593, 0.226]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 8, "global_sentence_id": 3, "edu_l1_label": "EDU_O"}, {"txt": "4", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.029, 0.264, 0.053, 0.278]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 17, "global_sentence_id": 4, "edu_l1_label": "EDU_O"}, {"txt": "2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.031, 0.278, 0.052, 0.29]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 16, "global_sentence_id": 5, "edu_l1_label": "EDU_O"}, {"txt": "O", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.032, 0.29, 0.051, 0.302]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 15, "global_sentence_id": 6, "edu_l1_label": "EDU_O"}, {"txt": "2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.031, 0.302, 0.051, 0.313]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 14, "global_sentence_id": 7, "edu_l1_label": "EDU_O"}, {"txt": "ar", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.034, 0.317, 0.055, 0.343]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 13, "global_sentence_id": 8, "edu_l1_label": "EDU_O"}, {"txt": "心", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.033, 0.342, 0.045, 0.348]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 12, "global_sentence_id": 9, "edu_l1_label": "EDU_O"}, {"txt": "A", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.028, 0.347, 0.053, 0.365]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 11, "global_sentence_id": 10, "edu_l1_label": "EDU_O"}, {"txt": "22", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.031, 0.368, 0.053, 0.384]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 10, "global_sentence_id": 11, "edu_l1_label": "EDU_O"}, {"txt": "2", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 300000, "bbox": [[0.031, 0.382, 0.053, 0.393]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 9, "global_sentence_id": 12, "edu_l1_label": "EDU_O"}, {"txt": "{kyeongkookseo, jaejun.yoo}@unist.ac.kr, djh@yonsei.ac.kr", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 400000, "bbox": [[0.186, 0.241, 0.74, 0.253]]}]}, "tags": ["text"], "label": "author", "web_segment_id": 6, "global_sentence_id": 13, "edu_l1_label": "EDU_O"}, {"txt": "ABSTRACT", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.455, 0.288, 0.545, 0.306]]}]}, "tags": ["title"], "label": "abstract", "web_segment_id": 5, "global_sentence_id": 14, "edu_l1_label": "BOS"}, {"txt": "Despite recent advancements in federated learning (FL), the integration of generative models into FL has been limited due to challenges such as high communication costs and unstable training in heterogeneous data environments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.235, 0.322, 0.765, 0.337], [0.235, 0.335, 0.765, 0.351], [0.235, 0.349, 0.692, 0.364]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 15, "edu_l1_label": "IOS"}, {"txt": "To address these issues, we propose PRISM, a FL framework tailored for generative models that ensures (i) stable performance in heterogeneous data distributions and (ii) resource effciency in terms of communication cost and fnal model size.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.692, 0.349, 0.765, 0.364], [0.235, 0.363, 0.765, 0.378], [0.235, 0.377, 0.765, 0.392], [0.235, 0.391, 0.706, 0.406]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 16, "edu_l1_label": "IOS"}, {"txt": "The key of our method is to search for an optimal stochastic binary mask for a random network rather than updating the model weights, identifying a sparse subnetwork with high generative performance; i.e., a “strong lottery ticket”.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.706, 0.391, 0.765, 0.406], [0.235, 0.405, 0.765, 0.42], [0.235, 0.418, 0.765, 0.434], [0.235, 0.432, 0.666, 0.447]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 17, "edu_l1_label": "IOS"}, {"txt": "By communi-cating binary masks in a stochastic manner, PRISM minimizes communication overhead.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.666, 0.432, 0.765, 0.447], [0.235, 0.446, 0.765, 0.461], [0.235, 0.46, 0.298, 0.475]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 18, "edu_l1_label": "IOS"}, {"txt": "Combined with the utilization of maximum mean discrepancy (MMD) loss and a mask-aware dynamic moving average aggregation method (MADA) on the server side, PRISM facilitates stable and strong generative capabilities by mitigating local divergence in FL scenarios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.298, 0.46, 0.765, 0.475], [0.235, 0.474, 0.765, 0.489], [0.235, 0.488, 0.765, 0.503], [0.235, 0.501, 0.528, 0.517]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 19, "edu_l1_label": "IOS"}, {"txt": "Moreover, thanks to its sparsifying characteristic, PRISM yields an lightweight model without extra pruning or quantization, making it ideal for environments such as edge devices.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.528, 0.501, 0.765, 0.517], [0.235, 0.515, 0.765, 0.53], [0.235, 0.529, 0.656, 0.544]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 20, "edu_l1_label": "IOS"}, {"txt": "Experiments on MNIST, FMNIST, CelebA, and CIFAR10 demonstrate that PRISM outperforms existing methods, while maintaining privacy with minimal communication costs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.656, 0.529, 0.765, 0.544], [0.235, 0.543, 0.765, 0.558], [0.235, 0.557, 0.765, 0.572]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 21, "edu_l1_label": "IOS"}, {"txt": "PRISM is the frst to successfully generate images under challenging non-IID and privacy-preserving FL environments on complex datasets, where previous methods have struggled.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.235, 0.571, 0.765, 0.586], [0.235, 0.584, 0.765, 0.6], [0.235, 0.598, 0.361, 0.613]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 22, "edu_l1_label": "IOS"}, {"txt": "Our code is available at PRISM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 500000, "bbox": [[0.361, 0.598, 0.575, 0.613]]}]}, "tags": ["text"], "label": "abstract", "web_segment_id": 0, "global_sentence_id": 23, "edu_l1_label": "IOS"}, {"txt": "1INTRODUCTION", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 600000, "bbox": [[0.177, 0.639, 0.337, 0.657]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 4, "global_sentence_id": 24, "edu_l1_label": "BOS"}, {"txt": "Recent generative models have demonstrated remarkable advancements in image quality and have been widely extended to various domains, including image-to-image translation (Choi et al., 2020;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.173, 0.672, 0.829, 0.811]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 25, "edu_l1_label": "IOS"}, {"txt": " Saharia et al., 2022)], layout generation (Seol et al., 2024)], text-to-image generation (Rombach et al., 2022; Ramesh et al., 2022)], and video generation (Skorokhodov et al., 2022; Kim et al., 2024b)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.173, 0.672, 0.829, 0.811]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 26, "edu_l1_label": "IOS"}, {"txt": "Achieving high-quality generation with current generative models demands increasingly large datasets, leading to concerns that publicly available data will soon exhausted (Villalobos et al., 2024)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.173, 0.672, 0.829, 0.811]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 27, "edu_l1_label": "IOS"}, {"txt": "Leveraging the vast amount of data stored on edge devices becomes a potential solution, but this poses signifcant challenges: not only does the private nature of the data make centralized training impractical, but edge computing itself faces hurdles, including limited resources and prohibitive communication costs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 700000, "bbox": [[0.173, 0.672, 0.829, 0.811]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 28, "edu_l1_label": "IOS"}, {"txt": "Federated learning (FL) (McMahan et al., 2017)] is a promising paradigm tailored to this setup, enabling clients to collaboratively train a global model without sharing their local datasets with a third party.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.176, 0.818, 0.824, 0.833], [0.176, 0.832, 0.824, 0.847], [0.176, 0.845, 0.247, 0.86]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 29, "edu_l1_label": "IOS"}, {"txt": "However, high communication costs, performance degradation due to data heterogeneity, and the need to preserve privacy remain signifcant challenges in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.247, 0.845, 0.824, 0.86], [0.176, 0.859, 0.632, 0.874]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 30, "edu_l1_label": "IOS"}, {"txt": "These challenges are further intensifed in the context of generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.632, 0.859, 0.824, 0.874], [0.176, 0.873, 0.487, 0.888]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 31, "edu_l1_label": "IOS"}, {"txt": "Unlike classifcation tasks, generative tasks lack a well-defned objective function and focus on learning the data sample distribution, making the inte-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 800000, "bbox": [[0.487, 0.873, 0.824, 0.888], [0.176, 0.887, 0.824, 0.902]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 32, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.194, 0.912, 0.355, 0.925]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 33, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.495, 0.949, 0.506, 0.961]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 34, "edu_l1_label": "EDU_O"}, {"txt": "gration of FL and generative models even more diffcult.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.176, 0.104, 0.55, 0.119]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 35, "edu_l1_label": "IOS"}, {"txt": "A few recent works have made efforts to train generative models over distributed clients (Hardy et al., 2019; Rasouli et al., 2020; Li et al., 2022; Zhang et al., 2021; Amalan et al., 2022)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.55, 0.104, 0.824, 0.119], [0.176, 0.118, 0.824, 0.133], [0.176, 0.132, 0.489, 0.147]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 36, "edu_l1_label": "IOS"}, {"txt": "These methods are generally built upon generative adversarial networks (GANs) (Goodfellow et al., 2020)], which have shown impressive results in the feld of image generation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.489, 0.132, 0.824, 0.147], [0.176, 0.146, 0.824, 0.161], [0.176, 0.16, 0.368, 0.175]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 37, "edu_l1_label": "IOS"}, {"txt": "DP-FedAvgGAN (Augenstein et al., 2019)], GS-WGAN (Chen et al., 2020)], and Private-FLGAN (Xin et al., 2020)] apply differential privacy (DP) (Dwork et al., 2006; Mironov, 2017)] to mitigate the potential privacy risk in FL setups.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.368, 0.16, 0.824, 0.175], [0.176, 0.174, 0.824, 0.189], [0.176, 0.187, 0.625, 0.202]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 38, "edu_l1_label": "IOS"}, {"txt": "However, existing works still face several challenges: 1) Previous approaches (Farnia & Ozdaglar, 2020a;b; Wang et al., 2022)] underperform, especially in non-IID (independent, identically distributed) data distribution scenarios with strong data heterogeneity across FL clients due to the notorious instability of GANs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.625, 0.187, 0.824, 0.202], [0.176, 0.201, 0.824, 0.216], [0.176, 0.215, 0.824, 0.23], [0.176, 0.229, 0.804, 0.244]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 39, "edu_l1_label": "IOS"}, {"txt": "2) Performance evaluations are limited to relatively simple datasets such as MNIST, Fashion MNIST, and EMNIST.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.804, 0.229, 0.824, 0.244], [0.176, 0.243, 0.824, 0.258], [0.176, 0.257, 0.269, 0.272]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 40, "edu_l1_label": "IOS"}, {"txt": "3) They suffer from signifcant communication overhead during model exchanges between the server and clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 900000, "bbox": [[0.269, 0.257, 0.824, 0.272], [0.176, 0.27, 0.376, 0.286]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 41, "edu_l1_label": "IOS"}, {"txt": "To overcome these challenges, we propose PRivacy-preserving Improved Stochastic-Masking for generative models (PRISM), a new strategy for training generative models in FL settings with the following key features:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.176, 0.292, 0.824, 0.307], [0.176, 0.306, 0.824, 0.321], [0.176, 0.319, 0.331, 0.335]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 42, "edu_l1_label": "IOS"}, {"txt": " First, at the heart of PRISM is the strong lottery ticket (SLT) hypothesis (Frankle & Carbin, 2018)], suggesting the existence of a highly effective subnetwork within a randomly initialized network.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.331, 0.319, 0.824, 0.335], [0.176, 0.333, 0.824, 0.348], [0.176, 0.347, 0.37, 0.362]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 43, "edu_l1_label": "IOS"}, {"txt": "PRISM shifts the focus towards identifying an optimal global binary mask, rather than updating the weights directly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.37, 0.347, 0.824, 0.362], [0.176, 0.361, 0.494, 0.376]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 44, "edu_l1_label": "IOS"}, {"txt": "This approach enables each client to transfer the binary mask to the server instead of the full model, signifcantly reducing the overload in each communication round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.494, 0.361, 0.824, 0.376], [0.176, 0.375, 0.824, 0.39], [0.176, 0.389, 0.296, 0.404]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 45, "edu_l1_label": "IOS"}, {"txt": "Moreover, when training is fnished, PRISM produces a lightweight fnal model, as each weight is already quantized, thanks to our initialization strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.296, 0.389, 0.824, 0.404], [0.176, 0.402, 0.638, 0.418]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 46, "edu_l1_label": "IOS"}, {"txt": "This feature provides significant advantages for resource-constrained edge devices.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.638, 0.402, 0.824, 0.418], [0.176, 0.416, 0.536, 0.431]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 47, "edu_l1_label": "IOS"}, {"txt": "Second, PRISM incorporates the maximum mean discrepancy (MMD) loss (Gretton et al., 2006; 2012)] during client-side updates, ensuring stable training for generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.536, 0.416, 0.824, 0.431], [0.176, 0.43, 0.824, 0.445], [0.176, 0.444, 0.424, 0.459]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 48, "edu_l1_label": "IOS"}, {"txt": "Third, a mask-aware dynamic moving average aggregation (MADA) is introduced to alleviate local model divergence.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.424, 0.444, 0.824, 0.459], [0.176, 0.458, 0.576, 0.473]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 49, "edu_l1_label": "IOS"}, {"txt": "This allows PRISM to maintain the previous global mask information and alleviate client drift under non-IID and DP-guaranteeing scenarios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.576, 0.458, 0.824, 0.473], [0.176, 0.472, 0.824, 0.487], [0.176, 0.485, 0.22, 0.501]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 50, "edu_l1_label": "IOS"}, {"txt": "By automatically adjusting the moving average parameter based on mask correlations, this approach requires neither a regularization term nor hyperparameter tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.22, 0.485, 0.824, 0.501], [0.176, 0.499, 0.67, 0.514]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 51, "edu_l1_label": "IOS"}, {"txt": "Finally, PRISM offers a hybrid strategy that can fexibly trade-off between image quality and communication cost.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.67, 0.499, 0.824, 0.514], [0.176, 0.513, 0.78, 0.528]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 52, "edu_l1_label": "IOS"}, {"txt": "Taken together, these features enable PRISM to consistently deliver robust performance in challenging non-IID and DP-guaranteeing FL settings, while maintaining minimal communication overhead.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1000000, "bbox": [[0.78, 0.513, 0.824, 0.528], [0.176, 0.527, 0.824, 0.542], [0.176, 0.541, 0.805, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 53, "edu_l1_label": "IOS"}, {"txt": "Our experimental results reveal that PRISM sets a new standard in generative model performance, signifcantly outperforming GAN-based methods in both IID and non-IID scenarios.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.176, 0.562, 0.824, 0.577], [0.176, 0.576, 0.746, 0.591]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 54, "edu_l1_label": "IOS"}, {"txt": "It acheives state-of-the-art image generation on complex datasets such as CelebA and CIFAR10, whereas previous methods were limited to simpler datasets like MNIST and FMNIST.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.746, 0.576, 0.824, 0.591], [0.176, 0.59, 0.824, 0.605], [0.176, 0.604, 0.658, 0.619]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 55, "edu_l1_label": "IOS"}, {"txt": "This highlights PRISM’spotential for scalable and resource-effcient generative model learning in distributed environments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.658, 0.604, 0.824, 0.619], [0.176, 0.618, 0.824, 0.633]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 56, "edu_l1_label": "IOS"}, {"txt": "Overall, our main contributions can be summarized as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1100000, "bbox": [[0.176, 0.631, 0.586, 0.647]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 57, "edu_l1_label": "IOS"}, {"txt": "• We propose PRISM, an effective FL framework that achieves state-of-the-art performanceon various benchmark datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.221, 0.664, 0.824, 0.679], [0.235, 0.678, 0.443, 0.693]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 58, "edu_l1_label": "IOS"}, {"txt": "It is the frst method to successfully generate images on complex datasets such as CelebA in FL scenarios that involves data heterogeneity and privacy preservation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1200000, "bbox": [[0.443, 0.678, 0.824, 0.693], [0.235, 0.692, 0.824, 0.707], [0.235, 0.705, 0.371, 0.721]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 59, "edu_l1_label": "IOS"}, {"txt": "• PRISM offers an effcient solution for federated generative models with minimal com-munication overhead by incorporating SLT with a stochastic binary mask.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.221, 0.736, 0.824, 0.752], [0.235, 0.75, 0.725, 0.765]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 60, "edu_l1_label": "IOS"}, {"txt": "Even more, in conjunction with the weight initialization strategy, the fnal model acquired from PRISM becomes signifcantly lightweight, reducing to less than half the size of the initial model.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1300000, "bbox": [[0.725, 0.75, 0.824, 0.765], [0.235, 0.764, 0.824, 0.779], [0.235, 0.778, 0.812, 0.793]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 61, "edu_l1_label": "IOS"}, {"txt": "• We further enhance the stability of federated learning for generative models by introducingMMD loss and a mask-aware dynamic moving average aggregation method (MADA).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1400000, "bbox": [[0.221, 0.809, 0.824, 0.824], [0.235, 0.823, 0.796, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 62, "edu_l1_label": "IOS"}, {"txt": "To the best of our knowledge, this is the frst work to address the challenges in communication effciency, privacy, stability, and generation performance altogether for federated generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.176, 0.855, 0.824, 0.871], [0.176, 0.869, 0.824, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 63, "edu_l1_label": "IOS"}, {"txt": "We provide new directions to this area based on several unique characteristics, including SLT with stochastic binary mask, MMD loss, mask-aware dynamic moving average aggregation strategy, and hybrid score/mask communications.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1500000, "bbox": [[0.176, 0.883, 0.824, 0.898], [0.176, 0.897, 0.824, 0.912], [0.176, 0.911, 0.412, 0.926]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 64, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1600000, "bbox": [[0.494, 0.95, 0.507, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 65, "edu_l1_label": "EDU_O"}, {"txt": "2RELATED WORK", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1700000, "bbox": [[0.177, 0.102, 0.345, 0.119]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 5, "global_sentence_id": 66, "edu_l1_label": "BOS"}, {"txt": "Federated learning for classifcation models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.176, 0.131, 0.486, 0.148]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 67, "edu_l1_label": "IOS"}, {"txt": "FL has achieved a signifcant success in training a global model in a distributed setup, eliminating the necessity of sharing individual client’s localdatasets with either the server or other clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.486, 0.131, 0.824, 0.147], [0.176, 0.146, 0.824, 0.161], [0.176, 0.16, 0.481, 0.175]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 68, "edu_l1_label": "IOS"}, {"txt": "Research has been conducted on various aspects of FL, such as data heterogeneity (Zhao et al., 2018; Li et al., 2021b)], communication effciency (Isik et al., 2022; Li et al., 2021a; Mitchell et al., 2022; Basat et al., 2022)], privacy (Wei et al., 2020)], with most focusing on image classifcation tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.481, 0.16, 0.824, 0.175], [0.176, 0.173, 0.824, 0.189], [0.176, 0.187, 0.824, 0.202], [0.176, 0.201, 0.492, 0.216]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 69, "edu_l1_label": "IOS"}, {"txt": "Related to our approach, FedPM (Isik et al., 2022)] and FedMask (Li et al., 2021a)] adopted binary mask communication to reduce the communication costs in FL in classifcation tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.492, 0.201, 0.824, 0.216], [0.176, 0.215, 0.824, 0.23], [0.176, 0.229, 0.406, 0.244]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 70, "edu_l1_label": "IOS"}, {"txt": "FedMask (Li et al., 2021a)] introduces binary mask communication, focusing on communication effciency and personalization in decentralized environments, while FedPM (Isik et al., 2022)] utilizes stochastic masks to minimize uplink overhead and proposes a bayesian aggregation method to robustly manage scenarios with partial client participation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.406, 0.229, 0.824, 0.244], [0.176, 0.243, 0.824, 0.258], [0.176, 0.256, 0.824, 0.272], [0.176, 0.27, 0.78, 0.285]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 71, "edu_l1_label": "IOS"}, {"txt": "While the concept of incorporating SLT into FL paradigm has been studied in FedMask (Li et al., 2021a)] and FedPM (Isik et al., 2022)] for classifcation tasks, PRISM stands as an independent strategy tailored for the training of generative models across distributed clients: PRISM incorporates MMD loss for more robust performance compared to GAN-based approaches and introduces MADA to maintain a stable image generation performance in heterogeneous and DP-guaranteeing FL settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.78, 0.27, 0.824, 0.285], [0.176, 0.284, 0.824, 0.299], [0.176, 0.298, 0.824, 0.313], [0.176, 0.312, 0.824, 0.327], [0.176, 0.326, 0.824, 0.341], [0.176, 0.339, 0.824, 0.355]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 72, "edu_l1_label": "IOS"}, {"txt": "potential benefts in various federated settings (See Appendix P).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.176, 0.384, 0.603, 0.399]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 73, "edu_l1_label": "IOS"}, {"txt": "Several recent works have aimed to incorporate generative models into distributed settings (Hardy et al., 2019; Amalan et al., 2022; Li et al., 2022; Zhang et al., 2021; Rasouli et al., 2020; Augenstein et al., 2019; Chen et al., 2020; Xin et al., 2020)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.603, 0.384, 0.824, 0.399], [0.176, 0.398, 0.824, 0.413], [0.176, 0.412, 0.824, 0.427], [0.176, 0.426, 0.263, 0.441]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 74, "edu_l1_label": "IOS"}, {"txt": "MD-GAN (Hardy et al., 2019)] was the frst attempt to apply generative models in the FL framework using GANs (Goodfellow et al., 2020)], extensively studied in image generation tasks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.263, 0.426, 0.824, 0.441], [0.176, 0.439, 0.824, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 75, "edu_l1_label": "IOS"}, {"txt": "In MD-GAN, each client holds a discriminator, and the server aggregates the discriminator feedback from each client to train the global generator.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.176, 0.453, 0.824, 0.468], [0.176, 0.467, 0.474, 0.482]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 76, "edu_l1_label": "IOS"}, {"txt": "To prevent overftting of local discriminators, clients exchange discriminators, incurring additional communication costs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.474, 0.467, 0.824, 0.482], [0.176, 0.481, 0.621, 0.496]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 77, "edu_l1_label": "IOS"}, {"txt": "Multi-FLGAN (Amalan et al., 2022)] proposed all vs. all game approach by employing multiple generators and multiple discriminators and then selecting the most powerful network to enhance model performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.621, 0.481, 0.824, 0.496], [0.176, 0.495, 0.824, 0.51], [0.176, 0.509, 0.73, 0.524]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 78, "edu_l1_label": "IOS"}, {"txt": "IFL-GAN (Li et al., 2022)] improves both performance and stability by weighting each client’s feedback based onthe MMD between the images generated by the global model and the local generator.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.73, 0.509, 0.824, 0.524], [0.176, 0.522, 0.824, 0.538], [0.176, 0.536, 0.727, 0.551]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 79, "edu_l1_label": "IOS"}, {"txt": "This approach maintains a balance between the generator and the discriminator, leading to Nash Equilibrium.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.727, 0.536, 0.824, 0.551], [0.176, 0.55, 0.783, 0.565]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 80, "edu_l1_label": "IOS"}, {"txt": "Other works such as (Zhang et al., 2021; Rasouli et al., 2020)] have also explored the utilization of GANs in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.783, 0.55, 0.824, 0.565], [0.176, 0.564, 0.824, 0.579], [0.176, 0.578, 0.216, 0.593]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 81, "edu_l1_label": "IOS"}, {"txt": "However, these works do not consider the challenge of privacy preservation in the context of FL and also suffer from resource issues during training and inference.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1800000, "bbox": [[0.216, 0.578, 0.824, 0.593], [0.176, 0.592, 0.63, 0.607]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 82, "edu_l1_label": "IOS"}, {"txt": "Federated learning for generative models with privacy consideration.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.176, 0.613, 0.663, 0.629]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 83, "edu_l1_label": "IOS"}, {"txt": "Only a few prior works have focused on the privacy preservation in federated generative models (Augenstein et al., 2019; Chen et al., 2020)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.663, 0.613, 0.824, 0.628], [0.176, 0.627, 0.824, 0.642], [0.176, 0.641, 0.307, 0.656]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 84, "edu_l1_label": "IOS"}, {"txt": "DP-FedAvgGAN (Augenstein et al., 2019)] introduces to combine federated generative models and differential privacy (DP) (Dwork et al., 2006; Mironov, 2017)] to ensure privacy preservation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.307, 0.641, 0.824, 0.656], [0.176, 0.655, 0.824, 0.67], [0.176, 0.668, 0.294, 0.684]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 85, "edu_l1_label": "IOS"}, {"txt": "GS-WGAN (Chen et al., 2020)] adopts Wasserstein GAN (Gulrajani et al., 2017)] to bypass the cumbersome searching for an appropriate DP-value, leveraging the Lipschitz property.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.294, 0.668, 0.824, 0.684], [0.176, 0.682, 0.824, 0.697]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 86, "edu_l1_label": "IOS"}, {"txt": "While these approaches have successfully integrated FL and generative models, they inherit drawbacks such as the notorious instability of GANs (Farnia & Ozdaglar, 2020a;b; Wang et al., 2022)] and signifcant performance drop under data heterogeneity.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.176, 0.696, 0.824, 0.711], [0.176, 0.71, 0.824, 0.725], [0.176, 0.724, 0.559, 0.739]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 87, "edu_l1_label": "IOS"}, {"txt": "Moreover, all existing approaches suffer from huge communication and storage costs during and after training, respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 1900000, "bbox": [[0.559, 0.724, 0.824, 0.739], [0.176, 0.738, 0.716, 0.753]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 88, "edu_l1_label": "IOS"}, {"txt": "3BACKGROUND", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2000000, "bbox": [[0.177, 0.77, 0.327, 0.788]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 4, "global_sentence_id": 89, "edu_l1_label": "BOS"}, {"txt": "3.1STRONG LOTTERY TICKETS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2100000, "bbox": [[0.177, 0.8, 0.404, 0.815]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 90, "edu_l1_label": "IOS"}, {"txt": "Strong Lottery Ticket (SLT) hypothesis (Frankle & Carbin, 2018;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.176, 0.827, 0.601, 0.842]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 91, "edu_l1_label": "IOS"}, {"txt": " Malach et al., 2020; Orseau et al., 2020)] suggest the existence of a sparse subnetwork within an initially random network that achieves a superior performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.601, 0.827, 0.824, 0.842], [0.176, 0.841, 0.824, 0.856], [0.176, 0.855, 0.334, 0.87]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 92, "edu_l1_label": "IOS"}, {"txt": "Edge-Popup (EP) algorithm (Ramanujan et al., 2020)] is one of the most popular methods to discover SLT within the dense network, which introduces a scoring mechanism to select potentially important weights among the widespread initialized weight values.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.334, 0.855, 0.824, 0.87], [0.176, 0.868, 0.824, 0.884], [0.176, 0.882, 0.737, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 93, "edu_l1_label": "IOS"}, {"txt": "More specifically, given a randomly initialized dense network Winit, a learnable score s is trained while keeping the weight values frozen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.737, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.338, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 94, "edu_l1_label": "IOS"}, {"txt": "These scores are designed to encapsulate the importance of each weight for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2200000, "bbox": [[0.338, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 95, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.494, 0.95, 0.506, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 96, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2300000, "bbox": [[0.171, 0.072, 0.832, 0.274]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 97, "edu_l1_label": "EDU_O"}, {"txt": "Figure 1: Overview of PRISM. PRISM fnds the supermask for generative models in a FL scenario. At every round t, each client k updates a local score skt via MMD loss (Step 1) and generates the privacy-preserving binary mask M tk (Step 2-1), which is sent to the server. The server aggregates the masks to obtain the global probability θt+1 (Step 3), which is converted to a score st+1 and broad-casted to the clients for the next round (Step 4). The global probability θt+1 is gradually updatedbased on mask correlation λ between M tg and M tg−1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2400000, "bbox": [[0.173, 0.282, 0.829, 0.371]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 0, "global_sentence_id": 98, "edu_l1_label": "EDU_O"}, {"txt": "the objective function.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.176, 0.393, 0.322, 0.408]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 99, "edu_l1_label": "IOS"}, {"txt": "As the scores get iteratively updated, the EP algorithm progressively shrinks the model by applying binary masks to weights with higher scores, indicating their potentials to be included in the winning lottery ticket.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.322, 0.393, 0.824, 0.408], [0.176, 0.407, 0.824, 0.422], [0.176, 0.421, 0.429, 0.436]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 100, "edu_l1_label": "IOS"}, {"txt": "The obtained SLT can be expressed as W = Winit ⊙M,where M is the obtained binary mask and ⊙denotes element-wise multiplication.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.429, 0.421, 0.824, 0.436], [0.176, 0.435, 0.727, 0.45]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 101, "edu_l1_label": "IOS"}, {"txt": "SLT has beenprimarily explored within the context of classifcation tasks, while Yeo et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.727, 0.435, 0.824, 0.45], [0.176, 0.448, 0.669, 0.463]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 102, "edu_l1_label": "IOS"}, {"txt": "(Yeo et al., 2023)] have recently shown that SLT can also be found in generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2500000, "bbox": [[0.669, 0.448, 0.824, 0.464], [0.176, 0.462, 0.597, 0.477]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 103, "edu_l1_label": "IOS"}, {"txt": "3.2DIFFERENTIAL PRIVACY", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2600000, "bbox": [[0.177, 0.5, 0.383, 0.515]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 104, "edu_l1_label": "IOS"}, {"txt": "Sharing each client’s model or gradient can potentially lead to a privacy risk.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.176, 0.529, 0.68, 0.544]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 105, "edu_l1_label": "IOS"}, {"txt": "(ϵ, δ)-differential pri-vacy (DP) (Dwork et al., 2006)], (α, ϵ)-Re´nyi-Differential privacy (RDP) (Mironov, 2017)] are com-monly employed when tackling the privacy concerns in FL.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2700000, "bbox": [[0.68, 0.529, 0.824, 0.544], [0.176, 0.543, 0.824, 0.558], [0.176, 0.556, 0.564, 0.572]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 106, "edu_l1_label": "IOS"}, {"txt": "Defnition 1 ((ϵ, δ)-Differential Privacy (Dwork et al., 2006)]) A randomized mechanism M :", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.176, 0.585, 0.824, 0.601]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 9, "global_sentence_id": 107, "edu_l1_label": "IOS"}, {"txt": "X →R is (ϵ, δ)-differential privacy, if for any two adjacent datasets D, D′ and for any measurablesets S: Pr[M(D) ∈S] ≤eϵPr[M(D′) ∈S] + δ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2800000, "bbox": [[0.176, 0.601, 0.824, 0.615], [0.176, 0.613, 0.511, 0.628]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 9, "global_sentence_id": 108, "edu_l1_label": "IOS"}, {"txt": "The above defnition is designed to limit the impact of individual data points by introducing randomness into M. The Gaussian mechanism (Mironov et al., 2019)] offers differential privacy guarantees ϵ222and ∆22 is L2 sensitivity.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 2900000, "bbox": [[0.176, 0.643, 0.824, 0.658], [0.176, 0.657, 0.824, 0.672], [0.598, 0.684, 0.609, 0.69], [0.639, 0.672, 0.81, 0.69]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 109, "edu_l1_label": "IOS"}, {"txt": "4METHOD", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3000000, "bbox": [[0.177, 0.712, 0.282, 0.729]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 8, "global_sentence_id": 110, "edu_l1_label": "BOS"}, {"txt": "We consider a FL setup with K clients, where each client k has its own local dataset Dk.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.176, 0.744, 0.767, 0.76]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 111, "edu_l1_label": "IOS"}, {"txt": "Starting from a randomly initialized model Winit, the clients aim to collaboratKively okbtain a global generative", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3100000, "bbox": [[0.767, 0.744, 0.824, 0.76], [0.176, 0.758, 0.824, 0.773]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 112, "edu_l1_label": "IOS"}, {"txt": "Overview of our approach.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.176, 0.8, 0.366, 0.816]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 113, "edu_l1_label": "IOS"}, {"txt": "Figure 1 shows the overview of our PRISM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.366, 0.8, 0.667, 0.815]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 114, "edu_l1_label": "IOS"}, {"txt": "PRISM fnds a subnetwork with strong generative performance from the randomly initialized generative model Winit.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.667, 0.8, 0.824, 0.815], [0.176, 0.814, 0.824, 0.829]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 115, "edu_l1_label": "IOS"}, {"txt": "Rather than updating Winit, its focus is to fnd an optimal binary mask M ∗that has either 1 or 0in its element and construct the fnal global model as W ∗= Winit ⊙M ∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.176, 0.828, 0.824, 0.843], [0.176, 0.842, 0.679, 0.857]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 116, "edu_l1_label": "IOS"}, {"txt": "At a high-level, eachclient k generates a binary mask M tk based on its local dataset at every communication round t, which is aggregated at the server.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.679, 0.842, 0.824, 0.857], [0.176, 0.855, 0.824, 0.871], [0.176, 0.869, 0.397, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 117, "edu_l1_label": "IOS"}, {"txt": "After repeating the process for multiple rounds t = 1, 2, .", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.397, 0.869, 0.783, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 118, "edu_l1_label": "IOS"}, {"txt": ".", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.783, 0.869, 0.79, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 119, "edu_l1_label": "IOS"}, {"txt": ".", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.79, 0.869, 0.798, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 120, "edu_l1_label": "IOS"}, {"txt": ", T, PRISM produces the fnal supermask M ∗= MT .", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.798, 0.869, 0.824, 0.884], [0.176, 0.883, 0.508, 0.898]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 121, "edu_l1_label": "IOS"}, {"txt": "Our approach, which aims to fnd the SLT in afederated generative model setting, is fundamentally different from existing FedGAN methods.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.508, 0.883, 0.824, 0.898], [0.176, 0.897, 0.805, 0.912]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 122, "edu_l1_label": "IOS"}, {"txt": "In the following, we describe the detailed training procedure of PRISM along with its advantages.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3200000, "bbox": [[0.805, 0.897, 0.824, 0.912], [0.176, 0.911, 0.796, 0.926]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 123, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3300000, "bbox": [[0.495, 0.949, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 124, "edu_l1_label": "EDU_O"}, {"txt": "4.1PRISM :", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.177, 0.104, 0.277, 0.119]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 125, "edu_l1_label": "IOS"}, {"txt": " PRIVACY-PRESERVING IMPROVED STOCHASTIC MASKING", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3400000, "bbox": [[0.277, 0.104, 0.682, 0.119]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 126, "edu_l1_label": "IOS"}, {"txt": "Local score updates with MMD loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.176, 0.127, 0.438, 0.144]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 127, "edu_l1_label": "IOS"}, {"txt": "Before training starts, the server randomly initializes the model Winit and broadcasts it to all clients, which remains fxed throughout the training process.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.438, 0.127, 0.824, 0.143], [0.176, 0.141, 0.803, 0.157]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 128, "edu_l1_label": "IOS"}, {"txt": "At the start of each round t, all clients download the score vector st from the server, representing the importance of each parameter in Winit.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.803, 0.141, 0.824, 0.157], [0.176, 0.155, 0.824, 0.17], [0.176, 0.169, 0.433, 0.184]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 129, "edu_l1_label": "IOS"}, {"txt": "Intuitively, if the score value of a specifc parameter is high, the corresponding weight is more likely to be included in the fnal SLT.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.433, 0.169, 0.824, 0.184], [0.176, 0.183, 0.648, 0.198]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 130, "edu_l1_label": "IOS"}, {"txt": "PRISM allows each client k to update the score vector st based on its local dataset to obtain skt , which is used to generate the local mask.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.648, 0.183, 0.824, 0.198], [0.176, 0.199, 0.824, 0.212], [0.176, 0.211, 0.251, 0.226]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 131, "edu_l1_label": "IOS"}, {"txt": "In this local score update procedure, we leverage maximum mean discrepancy (MMD) loss (Gretton et al., 2006; 2012)], providing stable convergence for training generative models (Li et al., 2017a;b; Bin´kowski et al., 2018; Santos et al., 2019; Ramanujan et al., 2020; Yeo et al., 2023)].The MMD loss measures the distance between two distributions by comparing their respective mean embeddings in a reproducing kernel hilbert space (RKHS) (Gretton et al., 2006; 2012)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.251, 0.211, 0.824, 0.226], [0.176, 0.224, 0.824, 0.24], [0.176, 0.238, 0.824, 0.253], [0.176, 0.252, 0.824, 0.267], [0.176, 0.266, 0.746, 0.281]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 132, "edu_l1_label": "IOS"}, {"txt": "As in (Santos et al., 2019; Ramanujan et al., 2020)], we take VGGNet pretrained on ImageNet as a powerful image set D fkake = {yik }Mi=1 produced by its own gener aDtor, th e{ loic }ail= o1b jective function at each client k is written as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3500000, "bbox": [[0.746, 0.266, 0.824, 0.281], [0.176, 0.28, 0.824, 0.295], [0.176, 0.309, 0.258, 0.333], [0.258, 0.317, 0.824, 0.324], [0.176, 0.324, 0.365, 0.339]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 133, "edu_l1_label": "IOS"}, {"txt": "IIP", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3600000, "bbox": [[0.757, 0.352, 0.773, 0.358]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 10, "global_sentence_id": 134, "edu_l1_label": "IOS"}, {"txt": "where ψ(·) is a  function that maps each sample to  the  VGG embedding space.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.173, 0.384, 0.829, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 135, "edu_l1_label": "IOS"}, {"txt": "Each clie nt aims tomatch the mean and covariance between real and fake samples after mapping them to the VGG embedding space using kernel ψ(·).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.173, 0.384, 0.829, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 136, "edu_l1_label": "IOS"}, {"txt": "Based on Eq.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.173, 0.384, 0.829, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 137, "edu_l1_label": "IOS"}, {"txt": "1, each client locally updates the scores to minimizethe MMD loss according to skt = st −η∇LkMMD.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.173, 0.384, 0.829, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 138, "edu_l1_label": "IOS"}, {"txt": "Here, we note that the VGG network is utilizedonly for computing the MMD loss and is discarded when training is fnished.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3700000, "bbox": [[0.173, 0.384, 0.829, 0.455]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 139, "edu_l1_label": "IOS"}, {"txt": "Binary mask generation and aggregation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.176, 0.465, 0.474, 0.481]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 140, "edu_l1_label": "IOS"}, {"txt": "After the local score update process, each client k maps the score skt to a probability value θtk ∈[0, 1] as θtk = Sigmoid(skt ), where Sigmoid(·) isthe sigmoid function.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.474, 0.465, 0.823, 0.48], [0.176, 0.479, 0.824, 0.494], [0.176, 0.493, 0.318, 0.508]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 141, "edu_l1_label": "IOS"}, {"txt": "The obtained θtk is then used as the parameter of the Bernoulli distributionto generate the stochastic binary mask M tk , according to M tk ∼Bern(θtk ).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.318, 0.493, 0.824, 0.508], [0.176, 0.506, 0.676, 0.522]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 142, "edu_l1_label": "IOS"}, {"txt": "Each client k uploadsonly this binary mask M tk to the server, signifcantly reducing the communication overhead.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.676, 0.506, 0.824, 0.522], [0.176, 0.52, 0.801, 0.535]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 143, "edu_l1_label": "IOS"}, {"txt": "At the server sidKe, the received masks are aggregated to estimate the global Bernoulli parameter as θt+1 =K1of the overall client’s weights.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.801, 0.52, 0.824, 0.535], [0.176, 0.534, 0.824, 0.549], [0.176, 0.553, 0.241, 0.559], [0.176, 0.564, 0.372, 0.579]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 144, "edu_l1_label": "IOS"}, {"txt": "θt+1 is then converted to the score through the inverse of the sigmoidfunction according to st+1 = Sigmoid−1(θt+1), which is broadcasted to the clients at the beginningof the next round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.372, 0.564, 0.824, 0.579], [0.176, 0.578, 0.824, 0.593], [0.176, 0.592, 0.292, 0.607]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 145, "edu_l1_label": "IOS"}, {"txt": "We provide the training details in Appendix C.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3800000, "bbox": [[0.292, 0.592, 0.6, 0.607]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 146, "edu_l1_label": "IOS"}, {"txt": "Model initialization for storage effciency.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.176, 0.618, 0.467, 0.634]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 147, "edu_l1_label": "IOS"}, {"txt": "When training is fnished after T rounds of FL, θT isobtained at the server.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.467, 0.618, 0.824, 0.633], [0.176, 0.632, 0.32, 0.647]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 148, "edu_l1_label": "IOS"}, {"txt": "The supermask is then generated following M ∗∼Bern(θT ), which is usedto obtain the fnal global model as W ∗= Winit⊙M ∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.32, 0.632, 0.824, 0.647], [0.176, 0.646, 0.528, 0.661]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 149, "edu_l1_label": "IOS"}, {"txt": "This fnal model W ∗can be stored effcientlyeven in resource-constrained edge devices, thanks to the model initialization strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.528, 0.646, 0.824, 0.661], [0.176, 0.66, 0.733, 0.675]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 150, "edu_l1_label": "IOS"}, {"txt": "When initializing Winit in PRISM, we employ the standard deviation of Kaiming Normal distribution (He et al., 2015)], where the weight value in layer l is sampled from the set σl ∼{−2/nl−1,2/nl−1}.Hence, by storing the scaling factorp2/nnl−1, each parameter in the initial mpodel Winipt is alreadyquantized to a 1-bit value (See Appendix E).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.733, 0.66, 0.824, 0.675], [0.176, 0.674, 0.824, 0.689], [0.176, 0.689, 0.824, 0.704], [0.176, 0.706, 0.824, 0.721], [0.176, 0.719, 0.47, 0.735]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 151, "edu_l1_label": "IOS"}, {"txt": "This makes the fnal model exceptionally lightweight without extra pruning of quantization, which will be also showed via comparison in Section 5.5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 3900000, "bbox": [[0.47, 0.719, 0.824, 0.735], [0.176, 0.733, 0.802, 0.748]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 152, "edu_l1_label": "IOS"}, {"txt": "4.2PRIVACY", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4000000, "bbox": [[0.177, 0.764, 0.277, 0.778]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 153, "edu_l1_label": "IOS"}, {"txt": "To consider the situation of potential privacy treats, we incorporate (ϵ, δ)-differential privacy(DP) (Dwork et al., 2006)] into our framework.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.176, 0.787, 0.824, 0.802], [0.176, 0.801, 0.5, 0.816]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 154, "edu_l1_label": "IOS"}, {"txt": "We provide a more detailed description related to the privacy preservation of PRISM and a specifc scenario where PRISM can achieve additional privacy benefts in Appendix A and Appendix B, respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4100000, "bbox": [[0.5, 0.801, 0.824, 0.816], [0.176, 0.815, 0.824, 0.83], [0.176, 0.829, 0.58, 0.844]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 155, "edu_l1_label": "IOS"}, {"txt": "4.3MASK-AWARE DYNAMIC MOVING AVERAGE AGGREGATION", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4200000, "bbox": [[0.177, 0.859, 0.626, 0.873]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 156, "edu_l1_label": "IOS"}, {"txt": "Data heterogeneity and privacy preservation pose signifcant challenges in accurately estimating the correct update direction.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.176, 0.882, 0.824, 0.897], [0.176, 0.896, 0.336, 0.911]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 157, "edu_l1_label": "IOS"}, {"txt": "Both the previous and current global models contain valuable information about whether the local models are diverging Praneeth Karimireddy et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.336, 0.896, 0.824, 0.911], [0.176, 0.91, 0.661, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 158, "edu_l1_label": "IOS"}, {"txt": "(2019); Li et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.661, 0.91, 0.771, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 159, "edu_l1_label": "IOS"}, {"txt": "(2020);", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4300000, "bbox": [[0.771, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 160, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.494, 0.949, 0.507, 0.961]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 161, "edu_l1_label": "EDU_O"}, {"txt": "Mendieta et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.176, 0.104, 0.275, 0.119]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 162, "edu_l1_label": "IOS"}, {"txt": "(2022).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.275, 0.104, 0.327, 0.119]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 163, "edu_l1_label": "IOS"}, {"txt": "To address this, we propose a mask-aware dynamic moving average aggregation (MADA) that leverages information from previous aggregation rounds in a mask-aware manner.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.327, 0.104, 0.824, 0.119], [0.176, 0.118, 0.824, 0.133], [0.176, 0.132, 0.2, 0.147]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 164, "edu_l1_label": "IOS"}, {"txt": "To the best of our knowledge, MADA is proposed for the frst time, introducing the mask-aware similarity to determine the moving average ratio on the server side.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.2, 0.132, 0.824, 0.147], [0.176, 0.146, 0.611, 0.161]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 165, "edu_l1_label": "IOS"}, {"txt": "Specifcally, when a client mask M tk deviates signifcantly from the global model, the newgly updated global mask M tg will also differ dominant clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.611, 0.146, 0.824, 0.161], [0.176, 0.162, 0.824, 0.175], [0.176, 0.187, 0.288, 0.202]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 166, "edu_l1_label": "IOS"}, {"txt": "To mitigate this bias, the server calcula−tes the mask correlation λ, which measuresserver then interpolates between the two global masks using λ to adjust how− much of the currentmask should be incorporated.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.288, 0.187, 0.824, 0.202], [0.176, 0.215, 0.824, 0.23], [0.176, 0.229, 0.371, 0.244]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 167, "edu_l1_label": "IOS"}, {"txt": "While various metrics can be used to compute the distance between masks, we employ the Hamming distance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.371, 0.229, 0.824, 0.244], [0.176, 0.243, 0.457, 0.258]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 168, "edu_l1_label": "IOS"}, {"txt": "We observe that using alternative distance metrics like cosine similarity yields similar performance (See Appendix L).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.457, 0.243, 0.824, 0.258], [0.176, 0.257, 0.587, 0.272]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 169, "edu_l1_label": "IOS"}, {"txt": "The server-side aggregation process is defned as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4400000, "bbox": [[0.587, 0.257, 0.824, 0.272], [0.176, 0.27, 0.314, 0.286]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 170, "edu_l1_label": "IOS"}, {"txt": "\\theta _ { t + 1 } = ( 1 - \\lambda ) \\theta _ { t - 1 } + \\lambda \\frac { 1 } { K } \\sum _ { k = 1 } ^ { K } M _ { t } ^ { k } , \\quad \\lambda : = d i s t ( M _ { t - 1 } ^ { g } , M _ { t } ^ { g } ) ,", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.285, 0.288, 0.711, 0.33]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 14, "global_sentence_id": 171, "edu_l1_label": "IOS"}, {"txt": "(2)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4500000, "bbox": [[0.805, 0.304, 0.824, 0.319]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 13, "global_sentence_id": 172, "edu_l1_label": "IOS"}, {"txt": "This prevents excessive deviation by interpolating the aggregated mask with the previous Bernoulli parameter.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.176, 0.339, 0.824, 0.355], [0.176, 0.353, 0.245, 0.368]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 173, "edu_l1_label": "IOS"}, {"txt": "As the global rounds progress, λ gradually decreases, promoting stable convergence.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4600000, "bbox": [[0.245, 0.353, 0.8, 0.368]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 8, "global_sentence_id": 174, "edu_l1_label": "IOS"}, {"txt": "4.4TRADING-OFF BETWEEN PERFORMANCE AND COMMUNICATION COST", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4700000, "bbox": [[0.177, 0.382, 0.696, 0.396]]}]}, "tags": ["text"], "label": "title2", "web_segment_id": 12, "global_sentence_id": 175, "edu_l1_label": "IOS"}, {"txt": "While PRISM optimizes for minimal communication overhead and delivers satisfactory performance, it may not always meet the demand for higher-quality image generation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.176, 0.404, 0.824, 0.419], [0.176, 0.418, 0.714, 0.433]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 176, "edu_l1_label": "IOS"}, {"txt": "To address this, we introduces a fexible solution, PRISM∗(See Appendix F).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4800000, "bbox": [[0.714, 0.418, 0.824, 0.433], [0.176, 0.432, 0.576, 0.447]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 177, "edu_l1_label": "IOS"}, {"txt": "5EXPERIMENTS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 4900000, "bbox": [[0.177, 0.459, 0.327, 0.477]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 9, "global_sentence_id": 178, "edu_l1_label": "BOS"}, {"txt": "In this section, we validate the effectiveness of PRISM on MNIST, FMNIST, CelebA, and CIFAR10 datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.176, 0.485, 0.824, 0.5], [0.176, 0.499, 0.232, 0.514]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 179, "edu_l1_label": "IOS"}, {"txt": "The training set of each dataset is distributed across 10 clients following either IID or non-IID data distributions1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.232, 0.499, 0.824, 0.514], [0.176, 0.512, 0.355, 0.528]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 180, "edu_l1_label": "IOS"}, {"txt": "We provide the detailed partitioning strategies for IID and non-IID simulation in Appendix D. In addition, for a fair comparison, we set (9.8, 10−5)-DP for all methods.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5000000, "bbox": [[0.355, 0.512, 0.824, 0.528], [0.176, 0.526, 0.807, 0.541]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 181, "edu_l1_label": "IOS"}, {"txt": "Baselines.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.176, 0.551, 0.245, 0.568]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 182, "edu_l1_label": "IOS"}, {"txt": "We compare our method with several previous approaches for federated generative models under both privacy-preserving (with DP) and privacy-free (without DP) scenarios: In the privacypreserving scenario, we consider DP-FedAvgGAN (Augenstein et al., 2019)] and GS-WGAN (Chen et al., 2020)] while in the privacy-free case, we adopt MD-GAN (Hardy et al., 2019)] and MultiFLGAN (Amalan et al., 2022)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5100000, "bbox": [[0.245, 0.551, 0.824, 0.567], [0.176, 0.566, 0.824, 0.581], [0.176, 0.579, 0.824, 0.595], [0.176, 0.593, 0.824, 0.608], [0.176, 0.607, 0.383, 0.622]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 183, "edu_l1_label": "IOS"}, {"txt": "In the case of Multi-FLGAN, the number of sync servers increases the communication cost quadratically, so we consider a 2×2 multi generator and discriminator setup.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5200000, "bbox": [[0.176, 0.628, 0.824, 0.644], [0.176, 0.642, 0.626, 0.657]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 184, "edu_l1_label": "IOS"}, {"txt": "Performance metrics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.176, 0.667, 0.325, 0.684]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 185, "edu_l1_label": "IOS"}, {"txt": "We evaluate the generative performance of each scheme using the commonly adopted metrics, including Fre´chet Inception Distance (FID) (Heusel et al., 2017)], Precision & Re-call (Kynka¨a¨nniemi et al., 2019)], Density & Coverage (Naeem et al., 2020)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.325, 0.667, 0.824, 0.683], [0.176, 0.681, 0.824, 0.697], [0.176, 0.695, 0.693, 0.71]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 186, "edu_l1_label": "IOS"}, {"txt": "We further demon-strate the effciency of PRISM by comparing the uplink cost (MB) (See Appendix G) at each FL round, the storage (MB) for the fnal models, and FLOPS (See Appendix J).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5300000, "bbox": [[0.693, 0.695, 0.824, 0.71], [0.176, 0.709, 0.824, 0.724], [0.176, 0.723, 0.671, 0.738]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 187, "edu_l1_label": "IOS"}, {"txt": "5.1IID CASE", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5400000, "bbox": [[0.177, 0.751, 0.281, 0.766]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 10, "global_sentence_id": 188, "edu_l1_label": "IOS"}, {"txt": "In this subsection, we examine an IID scenario where the training set of each dataset is uniformly distributed among clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.176, 0.774, 0.824, 0.789], [0.176, 0.788, 0.343, 0.803]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 189, "edu_l1_label": "IOS"}, {"txt": "We compare various evaluation metrics under (ϵ, δ)-DP guaranteed setting(Table 1).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.343, 0.788, 0.824, 0.803], [0.176, 0.802, 0.239, 0.817]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 190, "edu_l1_label": "IOS"}, {"txt": "Notably, PRISM outperforms current GAN-based models by a large margin.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5500000, "bbox": [[0.239, 0.802, 0.74, 0.817]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 191, "edu_l1_label": "IOS"}, {"txt": "Figure 2 reveals that existing privacy-preserving methods often produce distorted images, particularly evident in CelebA, while our method tends to generate high-quality results.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.176, 0.823, 0.824, 0.838], [0.176, 0.837, 0.704, 0.852]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 192, "edu_l1_label": "IOS"}, {"txt": "The above results support that PRISM can effectively fnd a SLT, achieving performance gains despite charging 48% less communication costs per round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.704, 0.837, 0.824, 0.852], [0.176, 0.851, 0.824, 0.866], [0.176, 0.865, 0.411, 0.88]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 193, "edu_l1_label": "IOS"}, {"txt": "It is worth noting that PRISM can further reduce the cost by applying extra techniques such as universal coding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.411, 0.865, 0.824, 0.88], [0.176, 0.878, 0.493, 0.894]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 194, "edu_l1_label": "IOS"}, {"txt": "Additionally, when comparing PRISM to PRISM†(MADA removed), PRISM demonstrates a signifcant improvement in overall performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5600000, "bbox": [[0.493, 0.878, 0.823, 0.894], [0.176, 0.892, 0.773, 0.907]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 195, "edu_l1_label": "IOS"}, {"txt": "1See Appendix H for additional results where a large number of clients and partial of clients participate.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.197, 0.91, 0.806, 0.925]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 11, "global_sentence_id": 196, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5700000, "bbox": [[0.495, 0.95, 0.506, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 197, "edu_l1_label": "EDU_O"}, {"txt": "Table 1:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.176, 0.077, 0.23, 0.092]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 198, "edu_l1_label": "EDU_O"}, {"txt": " Quantitative comparison in IID scenario with a privacy budget (ϵ, δ) = (9.8, 10−5).We compare FID, P&R, D&C, communication cost, and storage.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.23, 0.077, 0.824, 0.093], [0.176, 0.091, 0.594, 0.106]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 199, "edu_l1_label": "EDU_O"}, {"txt": "Communication cost is the number of bytes exchanged between clients and server.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.594, 0.091, 0.824, 0.106], [0.176, 0.105, 0.481, 0.12]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 200, "edu_l1_label": "EDU_O"}, {"txt": "† indicates that MADA is removed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5800000, "bbox": [[0.481, 0.105, 0.716, 0.12]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 201, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.202, 0.131, 0.795, 0.335]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 202, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 5900000, "bbox": [[0.167, 0.344, 0.82, 0.725]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 3, "global_sentence_id": 203, "edu_l1_label": "EDU_O"}, {"txt": "Figure 2: Qualitative results in IID scenario with a privacy budget (ϵ, δ) = (9.8, 10−5). Wecompare generated images from the models in Table 1 on MNIST, FMNIST, and CelebA. † indicatesthat MADA is removed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6000000, "bbox": [[0.173, 0.732, 0.827, 0.775]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 2, "global_sentence_id": 204, "edu_l1_label": "EDU_O"}, {"txt": "5.2NON-IID CASE", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6100000, "bbox": [[0.177, 0.782, 0.32, 0.797]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 4, "global_sentence_id": 205, "edu_l1_label": "IOS"}, {"txt": "We investigate a more practical yet challenging non-IID scenario, where clients exhibit diverse local data distributions, posing a signifcant challenge to train generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.176, 0.804, 0.824, 0.82], [0.176, 0.818, 0.665, 0.833]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 206, "edu_l1_label": "IOS"}, {"txt": "To establish such an environment, we split the entire trainset based on shards-partitioning strategy2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.665, 0.818, 0.824, 0.833], [0.176, 0.832, 0.666, 0.847]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 207, "edu_l1_label": "IOS"}, {"txt": "Table 2 presents a quantitative comparison of baselines and our methods when DP is guaranteed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.666, 0.832, 0.824, 0.847], [0.176, 0.846, 0.66, 0.861]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 208, "edu_l1_label": "IOS"}, {"txt": "We observe that PRISM achieves SOTA performance under the non-IID scenario, even with complex CelebA datasets3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.66, 0.846, 0.824, 0.861], [0.176, 0.86, 0.792, 0.875]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 209, "edu_l1_label": "IOS"}, {"txt": "Fig-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6200000, "bbox": [[0.792, 0.86, 0.824, 0.875]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 210, "edu_l1_label": "IOS"}, {"txt": "2We also provide the results of Dirichlet-partitioned Non-IID in Appendix D. 3Since existing works on the feld of federated generative models struggles with MNIST-level generation, we refer CelebA and CIFAR10 as complex dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.197, 0.883, 0.649, 0.898], [0.197, 0.897, 0.824, 0.912], [0.176, 0.911, 0.476, 0.925]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 5, "global_sentence_id": 211, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6300000, "bbox": [[0.495, 0.95, 0.506, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 212, "edu_l1_label": "EDU_O"}, {"txt": "Table 2:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.176, 0.077, 0.23, 0.092]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 213, "edu_l1_label": "EDU_O"}, {"txt": " Quantitative comparison in non-IID scenario with (ϵ, δ) = (9.8, 10−5).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.23, 0.077, 0.735, 0.093]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 214, "edu_l1_label": "EDU_O"}, {"txt": "We compareFID, P&R, D&C, communication cost, and storage.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.735, 0.077, 0.824, 0.092], [0.176, 0.091, 0.521, 0.106]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 215, "edu_l1_label": "EDU_O"}, {"txt": "Communication cost refers to the number of bytes exchanged between clients and server.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.521, 0.091, 0.824, 0.106], [0.176, 0.105, 0.463, 0.12]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 216, "edu_l1_label": "EDU_O"}, {"txt": "† indicates that MADA is removed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6400000, "bbox": [[0.463, 0.105, 0.698, 0.12]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 217, "edu_l1_label": "EDU_O"}, {"txt": "MINTST", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.188, 0.38, 0.198, 0.421]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 6, "global_sentence_id": 218, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6500000, "bbox": [[0.198, 0.135, 0.806, 0.711]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 219, "edu_l1_label": "EDU_O"}, {"txt": "Figure 3: Qualitative results in Non-IID scenario with a privacy budget (ϵ, δ) = (9.8, 10−5). Wecompare generated images from the models in Table 2 on MNIST, FMNIST, and CelebA. † indicatesthat MADA is removed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6600000, "bbox": [[0.175, 0.718, 0.827, 0.761]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 0, "global_sentence_id": 220, "edu_l1_label": "EDU_O"}, {"txt": "ure 3 illustrates that despite the heterogeneity of the data, our methods successfully generate high quality images, while traditional methods exhibit subpar quality. Additionally, Figure 4 shows the overview of FID scores, fnal model parameters, and communication costs for each method.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6700000, "bbox": [[0.174, 0.762, 0.827, 0.808]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 4, "global_sentence_id": 221, "edu_l1_label": "IOS"}, {"txt": "5.3 PERFORMANCE WITHOUT DIFFERENTIAL PRIVACY", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6800000, "bbox": [[0.177, 0.819, 0.199, 0.835], [0.216, 0.819, 0.562, 0.834]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 3, "global_sentence_id": 222, "edu_l1_label": "IOS"}, {"txt": "We further explore the performance of PRISM without applying differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.176, 0.841, 0.738, 0.856]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 223, "edu_l1_label": "IOS"}, {"txt": "Quantitative and qualitative comparisons with MD-GAN (Hardy et al., 2019)] and Multi-FLGAN (Amalan et al., 2022)], the current state-of-the-art under this condition, are shown in Table 3 and Figure 5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.738, 0.841, 0.824, 0.856], [0.176, 0.855, 0.824, 0.87], [0.176, 0.868, 0.77, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 224, "edu_l1_label": "IOS"}, {"txt": "PRISM not only matches but also occasionally surpasses the performance of MD-GAN and Multi-FLGAN, all while signifcantly reducing communication overhead.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.77, 0.868, 0.824, 0.884], [0.176, 0.882, 0.824, 0.897], [0.176, 0.896, 0.554, 0.911]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 225, "edu_l1_label": "IOS"}, {"txt": "Again, the fact that PRISM outperforms PRISM† clearly demonstrates the effectiveness of MADA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.554, 0.896, 0.824, 0.911], [0.176, 0.91, 0.553, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 226, "edu_l1_label": "IOS"}, {"txt": "Here, Multi-FLGAN takes 10 times more", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 6900000, "bbox": [[0.553, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 227, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.495, 0.95, 0.506, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 228, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7000000, "bbox": [[0.172, 0.073, 0.496, 0.219]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 4, "global_sentence_id": 229, "edu_l1_label": "EDU_O"}, {"txt": "(a) IID case", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7100000, "bbox": [[0.328, 0.223, 0.398, 0.237]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 8, "global_sentence_id": 230, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.509, 0.075, 0.824, 0.217]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 3, "global_sentence_id": 231, "edu_l1_label": "EDU_O"}, {"txt": "(b) Non-IID case", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7200000, "bbox": [[0.627, 0.224, 0.729, 0.237]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 7, "global_sentence_id": 232, "edu_l1_label": "EDU_O"}, {"txt": "Figure 4: The performance of baselines and our PRISM with privacy budget (ϵ, δ) (9.8, 10−5). X-axis represents the number of parameters of fnal generator, while Y-axis representsFID. The diameter of each circle denotes the required communication cost at every round. The ideal case is the bottom-left corner.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7300000, "bbox": [[0.174, 0.24, 0.827, 0.294]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 1, "global_sentence_id": 233, "edu_l1_label": "EDU_O"}, {"txt": "Table 3: Quantitative comparison in non-IID scenario without DP. We compare FID, P&R, D&C. Communication cost refers to the number of bytes exchanged between clients and server. † indicatesthat MADA is removed. We set α = 80 for PRISM∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7400000, "bbox": [[0.173, 0.304, 0.82, 0.346]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 234, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7500000, "bbox": [[0.176, 0.358, 0.826, 0.599]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 235, "edu_l1_label": "EDU_O"}, {"txt": "communication cost than PRISM due to multi GAN strategy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.176, 0.619, 0.588, 0.634]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 236, "edu_l1_label": "IOS"}, {"txt": "However, in FL setups, users may want to trade-off between the communication cost and generative performance.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.588, 0.619, 0.824, 0.634], [0.176, 0.633, 0.689, 0.648]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 237, "edu_l1_label": "IOS"}, {"txt": "As shown in Table 3 and Figure 5, PRISM∗outperforms across all benchmarks, including CIFAR10, while maintaininga communication cost comparable to MD-GAN and far lower than Multi-FLGAN.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.689, 0.633, 0.824, 0.648], [0.176, 0.646, 0.824, 0.662], [0.176, 0.66, 0.723, 0.675]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 238, "edu_l1_label": "IOS"}, {"txt": "Further details on PRISM∗can be found in Appendix F.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7600000, "bbox": [[0.723, 0.66, 0.824, 0.675], [0.176, 0.674, 0.444, 0.689]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 9, "global_sentence_id": 239, "edu_l1_label": "IOS"}, {"txt": "5.4  EFFECT OF DYNAMIC MOVING AVERAGE AGGREGATION", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7700000, "bbox": [[0.177, 0.705, 0.199, 0.72], [0.216, 0.705, 0.598, 0.72]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 240, "edu_l1_label": "IOS"}, {"txt": "In this section, we empirically demonstrate the effectiveness of dynamic moving average aggregation using the MNIST dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.176, 0.729, 0.824, 0.744], [0.176, 0.743, 0.37, 0.758]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 241, "edu_l1_label": "IOS"}, {"txt": "Figure 6 visualizes local model updates over communication rounds t. At each global round t, clients receive the aggregated global mask M tg −1 and continue local trainingfor several epochs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.37, 0.743, 0.824, 0.758], [0.176, 0.757, 0.632, 0.764], [0.629, 0.765, 0.824, 0.772], [0.176, 0.773, 0.3, 0.788]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 242, "edu_l1_label": "IOS"}, {"txt": "Afterward, the trained local mask M tk is ugploaded for the next communication ming distance between the received mask and the trained lotcal mtask to track the discrepancy of local model updates.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.3, 0.773, 0.824, 0.788], [0.176, 0.801, 0.824, 0.816], [0.176, 0.814, 0.313, 0.83]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 243, "edu_l1_label": "IOS"}, {"txt": "For simplicity, we visualize the results for the frst client, but similar trends were observed across other clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.313, 0.814, 0.824, 0.83], [0.176, 0.828, 0.403, 0.843]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 244, "edu_l1_label": "IOS"}, {"txt": "In Figure 6, PRISM exhibits impressive FID scores and reduced local updates than PRISM†, clearly demonstrating that MADA not only restricts the local modeldivergence but also achieves signifcant performance gain across various challenging FL settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.403, 0.828, 0.824, 0.843], [0.176, 0.842, 0.824, 0.857], [0.176, 0.856, 0.824, 0.871]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 245, "edu_l1_label": "IOS"}, {"txt": "This is accomplished by automatically obtaining λ based on the current mask, without requiringadditional regularization terms or hyperparameter tuning4.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7800000, "bbox": [[0.176, 0.87, 0.824, 0.885], [0.176, 0.884, 0.556, 0.899]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 246, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.194, 0.91, 0.776, 0.925]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 247, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.494, 0.95, 0.506, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 248, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 7900000, "bbox": [[0.173, 0.072, 0.824, 0.45]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 249, "edu_l1_label": "EDU_O"}, {"txt": "Figure 5: Qualitative results in non-IID scenario without considering privacy budget. Generated images from the models in Table 3 on MNIST, FMNIST, CelebA, and CIFAR10. Here, we set α = 80 for PRISM∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8000000, "bbox": [[0.174, 0.455, 0.828, 0.497]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 3, "global_sentence_id": 250, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8100000, "bbox": [[0.169, 0.501, 0.832, 0.633]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 4, "global_sentence_id": 251, "edu_l1_label": "EDU_O"}, {"txt": "Figure 6: Local divergence ∆t and FID values in the non-IID and DP-considering scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8200000, "bbox": [[0.201, 0.639, 0.799, 0.655]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 6, "global_sentence_id": 252, "edu_l1_label": "EDU_O"}, {"txt": "5.5RESOURCE EFFICIENCY AT INFERENCE TIME", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8300000, "bbox": [[0.173, 0.67, 0.525, 0.684]]}]}, "tags": ["figure_caption"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 253, "edu_l1_label": "IOS"}, {"txt": "The fnal model sizes of the baselines and our method are reported in Table 1, Table 2, and Table 3.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.176, 0.693, 0.824, 0.708]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 254, "edu_l1_label": "IOS"}, {"txt": "Note that in addition to our method, applying various lossless compression techniques (e.g., arithmetic coding (Rissanen & Langdon, 1979)]) can further reduce the required resources of PRISM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8400000, "bbox": [[0.176, 0.707, 0.824, 0.722], [0.176, 0.721, 0.824, 0.736]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 255, "edu_l1_label": "IOS"}, {"txt": "6 CONCLUSION", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8500000, "bbox": [[0.177, 0.762, 0.187, 0.78], [0.207, 0.762, 0.319, 0.779]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 5, "global_sentence_id": 256, "edu_l1_label": "BOS"}, {"txt": "While image generation has emerged as promising area in deep learning, the fusion of FL with generative models remains relatively unexplored.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.176, 0.785, 0.824, 0.801], [0.176, 0.799, 0.502, 0.814]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 257, "edu_l1_label": "IOS"}, {"txt": "In this paper, we proposed PRISM, an effcient and stable federated generative framework that capitalizes on stochastic binary masks and MMD loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.502, 0.799, 0.824, 0.814], [0.176, 0.813, 0.824, 0.828], [0.176, 0.827, 0.206, 0.842]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 258, "edu_l1_label": "IOS"}, {"txt": "To further enhance stability under non-IID and privacy-preserving scenario, we introduced a mask-aware dynamic moving average aggregation strategy (MADA) that mitigates client drift.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.206, 0.827, 0.824, 0.842], [0.176, 0.841, 0.824, 0.856]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 259, "edu_l1_label": "IOS"}, {"txt": "Additionally, PRISM offers a hybrid mask/score aggregation method, allowing for a fexible and controllable trade-off between performance and effciency.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.176, 0.855, 0.824, 0.87], [0.176, 0.868, 0.568, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 260, "edu_l1_label": "IOS"}, {"txt": "Our extensive experiments, including scenarios involving differential privacy and non-IID setups, demonstrated that PRISM is robust in unstable environments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.568, 0.868, 0.824, 0.884], [0.176, 0.882, 0.824, 0.897], [0.176, 0.896, 0.327, 0.911]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 261, "edu_l1_label": "IOS"}, {"txt": "To the best of our knowledge, PRISM is the frst framework to consistently generate high-quality images with signifcantly reduced communication overhead in FL settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8600000, "bbox": [[0.327, 0.896, 0.824, 0.911], [0.176, 0.91, 0.805, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 262, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8700000, "bbox": [[0.493, 0.95, 0.508, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 263, "edu_l1_label": "EDU_O"}, {"txt": "ACKNOWLEDGMENTS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8800000, "bbox": [[0.177, 0.102, 0.357, 0.119]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 2, "global_sentence_id": 264, "edu_l1_label": "IOS"}, {"txt": "This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2022R1C1C1008496) and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT):", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.173, 0.135, 0.827, 0.248]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 265, "edu_l1_label": "EDU_O"}, {"txt": " (No.2022-0-00959, No.RS-2022-II220959 (Part 2) Few-Shot Learning of Causal Inference in Vision and Language for Decision Making), (No.RS-2022-II220264, Comprehensive Video Understanding and Generation with Knowledge-based Deep Logic Neural Network), (No.RS-2020-II201336, Artifcial Intelligence Graduate School Program (UNIST)), (No.RS-2021-II212068, Artifcial Intelligence Innovation Hub).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 8900000, "bbox": [[0.173, 0.135, 0.827, 0.248]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 266, "edu_l1_label": "EDU_O"}, {"txt": "REFERENCES", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.177, 0.266, 0.286, 0.283]]}]}, "tags": ["title"], "label": "reference", "web_segment_id": 3, "global_sentence_id": 267, "edu_l1_label": "EDU_O"}, {"txt": "Akash Amalan, Rui Wang, Yanqi Qiao, Emmanouil Panaousis, and Kaitai Liang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.29, 0.726, 0.305]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 268, "edu_l1_label": "EDU_O"}, {"txt": "Multi-fgans: Multi-distributed adversarial networks for non-iid distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.726, 0.29, 0.824, 0.305], [0.193, 0.304, 0.598, 0.319]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 269, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2206.12178, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.598, 0.304, 0.824, 0.319], [0.193, 0.318, 0.229, 0.333]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 270, "edu_l1_label": "EDU_O"}, {"txt": "Sean Augenstein, H Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.342, 0.824, 0.357], [0.193, 0.356, 0.437, 0.371]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 271, "edu_l1_label": "EDU_O"}, {"txt": "Generative models for effective ml on private, decentralized datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.437, 0.356, 0.824, 0.371], [0.193, 0.37, 0.248, 0.385]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 272, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1911.06679, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.248, 0.37, 0.515, 0.385]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 273, "edu_l1_label": "EDU_O"}, {"txt": "Ran Ben Basat, Shay Vargaftik, Amit Portnoy, Gil Einziger, Yaniv Ben-Itzhak, and Michael Mitzenmacher.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.394, 0.824, 0.409], [0.193, 0.408, 0.29, 0.423]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 274, "edu_l1_label": "EDU_O"}, {"txt": "Quick-f: Quick unbiased compression for federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.29, 0.408, 0.716, 0.423]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 275, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2205.13341, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.716, 0.408, 0.824, 0.423], [0.193, 0.422, 0.356, 0.437]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 276, "edu_l1_label": "EDU_O"}, {"txt": "Mikołaj Bin´kowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.446, 0.687, 0.461]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 277, "edu_l1_label": "EDU_O"}, {"txt": "Demystifying mmdgans.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.687, 0.446, 0.824, 0.461], [0.193, 0.46, 0.227, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 278, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1801.01401, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.227, 0.46, 0.493, 0.475]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 279, "edu_l1_label": "EDU_O"}, {"txt": "Dingfan Chen, Tribhuvanesh Orekondy, and Mario Fritz.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.484, 0.551, 0.499]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 280, "edu_l1_label": "EDU_O"}, {"txt": "Gs-wgan: A gradient-sanitized approach for learning differentially private generators.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.551, 0.484, 0.824, 0.499], [0.193, 0.498, 0.479, 0.513]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 281, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 33:12673–12684, 2020.Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.479, 0.498, 0.824, 0.513], [0.193, 0.512, 0.348, 0.527], [0.176, 0.536, 0.565, 0.551]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 282, "edu_l1_label": "EDU_O"}, {"txt": "Stargan v2: Diverse image synthesis for multiple domains.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.565, 0.536, 0.824, 0.551], [0.193, 0.55, 0.311, 0.565]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 283, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.311, 0.55, 0.824, 0.565], [0.193, 0.564, 0.296, 0.579]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 284, "edu_l1_label": "EDU_O"}, {"txt": "8188–8197, 2020.Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.296, 0.564, 0.418, 0.579], [0.176, 0.588, 0.582, 0.603]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 285, "edu_l1_label": "EDU_O"}, {"txt": "Differentially private diffusion models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.582, 0.588, 0.824, 0.603], [0.193, 0.602, 0.215, 0.617]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 286, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2210.09929, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.215, 0.602, 0.482, 0.617]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 287, "edu_l1_label": "EDU_O"}, {"txt": "Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.626, 0.616, 0.641]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 288, "edu_l1_label": "EDU_O"}, {"txt": "Calibrating noise to sensitivity in private data analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.616, 0.626, 0.824, 0.641], [0.193, 0.64, 0.347, 0.655]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 289, "edu_l1_label": "EDU_O"}, {"txt": "In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.347, 0.64, 0.824, 0.655], [0.193, 0.654, 0.516, 0.669]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 290, "edu_l1_label": "EDU_O"}, {"txt": "Proceedings 3, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.516, 0.654, 0.641, 0.669]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 291, "edu_l1_label": "EDU_O"}, {"txt": "265–284.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.641, 0.654, 0.707, 0.669]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 292, "edu_l1_label": "EDU_O"}, {"txt": "Springer, 2006.Farzan Farnia and Asuman Ozdaglar.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.707, 0.654, 0.811, 0.669], [0.176, 0.678, 0.421, 0.693]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 293, "edu_l1_label": "EDU_O"}, {"txt": "Do gans always have nash equilibria?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.421, 0.678, 0.675, 0.693]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 294, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.675, 0.678, 0.824, 0.693], [0.193, 0.692, 0.413, 0.707]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 295, "edu_l1_label": "EDU_O"}, {"txt": "3029–3039.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.413, 0.692, 0.495, 0.707]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 296, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2020a.Farzan Farnia and Asuman Ozdaglar.Gans may have no nash equilibria.arXiv preprint arXiv:2002.09124, 2020b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.495, 0.692, 0.595, 0.707], [0.176, 0.716, 0.824, 0.731], [0.193, 0.73, 0.365, 0.745]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 297, "edu_l1_label": "EDU_O"}, {"txt": "Jonathan Frankle and Michael Carbin.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.754, 0.423, 0.769]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 298, "edu_l1_label": "EDU_O"}, {"txt": "The lottery ticket hypothesis: Finding sparse, trainable neural networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.423, 0.754, 0.824, 0.769], [0.193, 0.768, 0.256, 0.783]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 299, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1803.03635, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.256, 0.768, 0.523, 0.783]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 300, "edu_l1_label": "EDU_O"}, {"txt": "Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.792, 0.824, 0.807], [0.193, 0.806, 0.44, 0.821]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 301, "edu_l1_label": "EDU_O"}, {"txt": "Generative adversarial networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.44, 0.806, 0.663, 0.821]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 302, "edu_l1_label": "EDU_O"}, {"txt": "Communications of the ACM, 63(11):139–144, 2020.Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Scho¨lkopf, and Alex Smola.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.663, 0.806, 0.824, 0.821], [0.193, 0.82, 0.384, 0.835], [0.176, 0.844, 0.759, 0.859]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 303, "edu_l1_label": "EDU_O"}, {"txt": "A kernelmethod for the two-sample-problem.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.759, 0.844, 0.824, 0.859], [0.193, 0.858, 0.438, 0.873]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 304, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 19, 2006.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.438, 0.858, 0.824, 0.873], [0.193, 0.872, 0.229, 0.887]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 305, "edu_l1_label": "EDU_O"}, {"txt": "Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Scho¨lkopf, and Alexander Smola.A kernel two-sample test.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.176, 0.896, 0.824, 0.911], [0.193, 0.91, 0.36, 0.925]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 306, "edu_l1_label": "EDU_O"}, {"txt": "The Journal of Machine Learning Research, 13(1):723–773, 2012.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9000000, "bbox": [[0.36, 0.91, 0.799, 0.925]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 307, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.493, 0.95, 0.507, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 308, "edu_l1_label": "EDU_O"}, {"txt": "Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.104, 0.793, 0.119]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 309, "edu_l1_label": "EDU_O"}, {"txt": "Improved training of wasserstein gans.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.793, 0.104, 0.824, 0.119], [0.193, 0.118, 0.435, 0.133]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 310, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 30, 2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.435, 0.118, 0.824, 0.133], [0.193, 0.132, 0.229, 0.147]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 311, "edu_l1_label": "EDU_O"}, {"txt": "Corentin Hardy, Erwan Le Merrer, and Bruno Sericola.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.156, 0.548, 0.172]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 312, "edu_l1_label": "EDU_O"}, {"txt": "Md-gan: Multi-discriminator generative adversarial networks for distributed datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.548, 0.156, 0.824, 0.172], [0.193, 0.17, 0.483, 0.185]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 313, "edu_l1_label": "EDU_O"}, {"txt": "In 2019 IEEE international parallel and distributed processing symposium (IPDPS), pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.483, 0.17, 0.824, 0.185], [0.193, 0.184, 0.428, 0.199]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 314, "edu_l1_label": "EDU_O"}, {"txt": "866–877.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.428, 0.184, 0.493, 0.199]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 315, "edu_l1_label": "EDU_O"}, {"txt": "IEEE, 2019.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.493, 0.184, 0.577, 0.199], [0.176, 0.208, 0.561, 0.224]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 316, "edu_l1_label": "EDU_O"}, {"txt": "Delving deep into rectifers: Surpassing human-level performance on imagenet classifcation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.561, 0.208, 0.824, 0.224], [0.193, 0.222, 0.541, 0.237]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 317, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE international conference on computer vision, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.541, 0.222, 0.824, 0.237], [0.193, 0.236, 0.421, 0.251]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 318, "edu_l1_label": "EDU_O"}, {"txt": "1026–1034, 2015.Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.421, 0.236, 0.543, 0.251], [0.176, 0.261, 0.824, 0.276]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 319, "edu_l1_label": "EDU_O"}, {"txt": "Gans trained by a two time-scale update rule converge to a local nash equilibrium.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.193, 0.274, 0.75, 0.29]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 320, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 30, 2017.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.75, 0.274, 0.824, 0.289], [0.193, 0.288, 0.53, 0.303]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 321, "edu_l1_label": "EDU_O"}, {"txt": "Jonathan Ho, Ajay Jain, and Pieter Abbeel.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.313, 0.459, 0.328]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 322, "edu_l1_label": "EDU_O"}, {"txt": "Denoising diffusion probabilistic models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.459, 0.313, 0.737, 0.328]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 323, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 33:6840–6851, 2020.Rui Hu, Yuanxiong Guo, and Yanmin Gong.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.737, 0.313, 0.824, 0.328], [0.193, 0.326, 0.591, 0.342], [0.176, 0.351, 0.475, 0.366]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 324, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning with sparsifed model perturbation: Improving accuracy under client-level differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.475, 0.351, 0.824, 0.366], [0.193, 0.365, 0.615, 0.38]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 325, "edu_l1_label": "EDU_O"}, {"txt": "IEEE Transactions on Mobile Computing, 23(8):8242–8255, 2023.Jacob Imola and Kamalika Chaudhuri.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.615, 0.365, 0.824, 0.38], [0.193, 0.379, 0.431, 0.394], [0.176, 0.403, 0.427, 0.418]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 326, "edu_l1_label": "EDU_O"}, {"txt": "Privacy amplifcation via bernoulli sampling.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.427, 0.403, 0.725, 0.418]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 327, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2105.10594, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.725, 0.403, 0.824, 0.418], [0.193, 0.417, 0.356, 0.432]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 328, "edu_l1_label": "EDU_O"}, {"txt": "Berivan Isik, Francesco Pase, Deniz Gunduz, Tsachy Weissman, and Michele Zorzi.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.441, 0.721, 0.456]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 329, "edu_l1_label": "EDU_O"}, {"txt": "Sparse random networks for communication-effcient federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.721, 0.441, 0.824, 0.456], [0.193, 0.455, 0.562, 0.47]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 330, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2209.15328, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.562, 0.455, 0.824, 0.47]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 331, "edu_l1_label": "EDU_O"}, {"txt": "Zepeng Jiang, Weiwei Ni, and Yifan Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.479, 0.459, 0.495]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 332, "edu_l1_label": "EDU_O"}, {"txt": "Pate-triplegan: Privacy-preserving image synthesis with gaussian differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.459, 0.479, 0.824, 0.495], [0.193, 0.493, 0.38, 0.508]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 333, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2404.12730, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.38, 0.493, 0.646, 0.508]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 334, "edu_l1_label": "EDU_O"}, {"txt": "Do-Yeon Kim, Dong-Jun Han, Jun Seo, and Jaekyun Moon.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.518, 0.568, 0.533]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 335, "edu_l1_label": "EDU_O"}, {"txt": "Achieving lossless gradient sparsifcation via mapping to alternative space in federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.568, 0.518, 0.824, 0.533], [0.193, 0.532, 0.57, 0.547]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 336, "edu_l1_label": "EDU_O"}, {"txt": "In Forty-frst International Conference on Machine Learning, 2024a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.57, 0.532, 0.824, 0.547], [0.193, 0.545, 0.386, 0.561]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 337, "edu_l1_label": "EDU_O"}, {"txt": "Kihong Kim, Haneol Lee, Jihye Park, Seyeon Kim, Kwanghee Lee, Seungryong Kim, and Jaejun Yoo.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.57, 0.824, 0.585], [0.193, 0.584, 0.223, 0.599]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 338, "edu_l1_label": "EDU_O"}, {"txt": "Hybrid video diffusion models with 2d triplane and 3d wavelet representation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.223, 0.584, 0.727, 0.599]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 339, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2402.13729, 2024b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.727, 0.584, 0.824, 0.599], [0.193, 0.597, 0.365, 0.613]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 340, "edu_l1_label": "EDU_O"}, {"txt": "Diederik P Kingma and Jimmy Ba.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.622, 0.411, 0.637]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 341, "edu_l1_label": "EDU_O"}, {"txt": "Adam: A method for stochastic optimization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.411, 0.622, 0.72, 0.637]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 342, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1412.6980, 2014.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.72, 0.622, 0.824, 0.637], [0.193, 0.636, 0.348, 0.651]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 343, "edu_l1_label": "EDU_O"}, {"txt": "Tuomas Kynka¨a¨nniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.66, 0.749, 0.675]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 344, "edu_l1_label": "EDU_O"}, {"txt": "Improvedprecision and recall metric for assessing generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.749, 0.66, 0.824, 0.675], [0.193, 0.674, 0.595, 0.689]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 345, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 32, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.595, 0.674, 0.824, 0.689], [0.193, 0.688, 0.389, 0.703]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 346, "edu_l1_label": "EDU_O"}, {"txt": "Ang Li, Jingwei Sun, Xiao Zeng, Mi Zhang, Hai Li, and Yiran Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.712, 0.645, 0.727]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 347, "edu_l1_label": "EDU_O"}, {"txt": "Fedmask: Joint computation and communication-effcient personalized federated learning via heterogeneous masking.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.645, 0.712, 0.824, 0.727], [0.193, 0.726, 0.804, 0.741]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 348, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.804, 0.726, 0.824, 0.741], [0.193, 0.74, 0.774, 0.755]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 349, "edu_l1_label": "EDU_O"}, {"txt": "42–55,2021a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.774, 0.74, 0.824, 0.755], [0.193, 0.754, 0.237, 0.769]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 350, "edu_l1_label": "EDU_O"}, {"txt": "Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnaba´s Po´czos.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.778, 0.741, 0.793]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 351, "edu_l1_label": "EDU_O"}, {"txt": "Mmd gan:Towards deeper understanding of moment matching network.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.741, 0.778, 0.824, 0.793], [0.193, 0.792, 0.602, 0.807]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 352, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 30, 2017a.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.602, 0.792, 0.824, 0.807], [0.193, 0.806, 0.393, 0.821]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 353, "edu_l1_label": "EDU_O"}, {"txt": "Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnaba´s Po´czos.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.83, 0.741, 0.845]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 354, "edu_l1_label": "EDU_O"}, {"txt": "Mmd gan:Towards deeper understanding of moment matching network.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.741, 0.83, 0.824, 0.845], [0.193, 0.844, 0.602, 0.859]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 355, "edu_l1_label": "EDU_O"}, {"txt": "Advances in neural information processing systems, 30, 2017b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.602, 0.844, 0.824, 0.859], [0.193, 0.858, 0.394, 0.873]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 356, "edu_l1_label": "EDU_O"}, {"txt": "Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.176, 0.882, 0.824, 0.897]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 357, "edu_l1_label": "EDU_O"}, {"txt": "Federated optimization in heterogeneous networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.193, 0.896, 0.531, 0.911]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 358, "edu_l1_label": "EDU_O"}, {"txt": "Proceedings of Machine learning and systems, 2:429–450, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9100000, "bbox": [[0.531, 0.896, 0.824, 0.911], [0.193, 0.91, 0.345, 0.925]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 359, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.493, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 360, "edu_l1_label": "EDU_O"}, {"txt": "Wei Li, Jinlin Chen, Zhenyu Wang, Zhidong Shen, Chao Ma, and Xiaohui Cui.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.104, 0.7, 0.119]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 361, "edu_l1_label": "EDU_O"}, {"txt": "If-gan: Improved federated learning generative adversarial network with maximum mean discrepancy model aggregation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.7, 0.104, 0.824, 0.119], [0.193, 0.118, 0.824, 0.133], [0.193, 0.132, 0.237, 0.147]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 362, "edu_l1_label": "EDU_O"}, {"txt": "IEEE Transactions on Neural Networks and Learning Systems, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.237, 0.132, 0.694, 0.147]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 363, "edu_l1_label": "EDU_O"}, {"txt": "Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.156, 0.645, 0.171]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 364, "edu_l1_label": "EDU_O"}, {"txt": "Fedbn: Federated learning on non-iid features via local batch normalization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.645, 0.156, 0.824, 0.171], [0.193, 0.17, 0.513, 0.185]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 365, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2102.07623, 2021b.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.513, 0.17, 0.788, 0.185]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 366, "edu_l1_label": "EDU_O"}, {"txt": "Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, and Ohad Shamir.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.193, 0.648, 0.209]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 367, "edu_l1_label": "EDU_O"}, {"txt": "Proving the lottery ticket hypothesis: Pruning is all you need.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.648, 0.193, 0.824, 0.209], [0.193, 0.207, 0.422, 0.222]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 368, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.422, 0.207, 0.779, 0.222]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 369, "edu_l1_label": "EDU_O"}, {"txt": "6682–6691.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.779, 0.207, 0.824, 0.222], [0.193, 0.221, 0.229, 0.236]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 370, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.229, 0.221, 0.323, 0.236]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 371, "edu_l1_label": "EDU_O"}, {"txt": "Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.245, 0.824, 0.26]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 372, "edu_l1_label": "EDU_O"}, {"txt": "Communication-effcient learning of deep networks from decentralized data.In Artifcial intelligence and statistics, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.193, 0.259, 0.824, 0.274], [0.193, 0.273, 0.374, 0.288]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 373, "edu_l1_label": "EDU_O"}, {"txt": "1273–1282.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.374, 0.273, 0.455, 0.288]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 374, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2017.Matias Mendieta, Taojiannan Yang, Pu Wang, Minwoo Lee, Zhengming Ding, and Chen Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.455, 0.273, 0.548, 0.288], [0.176, 0.296, 0.824, 0.311]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 375, "edu_l1_label": "EDU_O"}, {"txt": "Local learning matters: Rethinking data heterogeneity in federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.193, 0.31, 0.696, 0.325]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 376, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.696, 0.31, 0.824, 0.325], [0.193, 0.324, 0.691, 0.339]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 377, "edu_l1_label": "EDU_O"}, {"txt": "8397–8406, 2022.Ilya Mironov.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.691, 0.324, 0.813, 0.339], [0.176, 0.348, 0.266, 0.363]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 378, "edu_l1_label": "EDU_O"}, {"txt": "Re´nyi differential privacy.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.266, 0.348, 0.441, 0.363]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 379, "edu_l1_label": "EDU_O"}, {"txt": "In 2017 IEEE 30th computer security foundations sympo-sium (CSF), pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.441, 0.348, 0.824, 0.363], [0.193, 0.362, 0.296, 0.377]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 380, "edu_l1_label": "EDU_O"}, {"txt": "263–275.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.296, 0.362, 0.361, 0.377]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 381, "edu_l1_label": "EDU_O"}, {"txt": "IEEE, 2017.Ilya Mironov, Kunal Talwar, and Li Zhang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.361, 0.362, 0.445, 0.377], [0.176, 0.385, 0.465, 0.401]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 382, "edu_l1_label": "EDU_O"}, {"txt": "R\\’enyi differential privacy of the sampled gaussianmechanism.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.465, 0.385, 0.824, 0.401], [0.193, 0.399, 0.271, 0.414]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 383, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1908.10530, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.271, 0.399, 0.538, 0.414]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 384, "edu_l1_label": "EDU_O"}, {"txt": "Nicole Mitchell, Johannes Balle´, Zachary Charles, and Jakub Konecˇny`.Optimizing thecommunication-accuracy trade-off in federated learning with rate-distortion theory.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.423, 0.824, 0.438], [0.193, 0.437, 0.728, 0.452]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 385, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2201.02664, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.728, 0.437, 0.824, 0.452], [0.193, 0.451, 0.356, 0.466]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 386, "edu_l1_label": "EDU_O"}, {"txt": "Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.474, 0.763, 0.49]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 387, "edu_l1_label": "EDU_O"}, {"txt": "Reliable fdelity and diversity metrics for generative models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.763, 0.474, 0.824, 0.49], [0.193, 0.488, 0.54, 0.503]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 388, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.54, 0.488, 0.824, 0.503], [0.193, 0.502, 0.281, 0.517]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 389, "edu_l1_label": "EDU_O"}, {"txt": "7176–7185.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.281, 0.502, 0.362, 0.517]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 390, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2020.Laurent Orseau, Marcus Hutter, and Omar Rivasplata.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.362, 0.502, 0.455, 0.517], [0.176, 0.526, 0.536, 0.541]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 391, "edu_l1_label": "EDU_O"}, {"txt": "Logarithmic pruning is all you need.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.536, 0.526, 0.79, 0.541]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 392, "edu_l1_label": "EDU_O"}, {"txt": "Advances in Neural Information Processing Systems, 33:2925–2934, 2020.Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.79, 0.526, 0.824, 0.541], [0.193, 0.54, 0.663, 0.555], [0.176, 0.564, 0.824, 0.579], [0.193, 0.577, 0.358, 0.593]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 393, "edu_l1_label": "EDU_O"}, {"txt": "Scaffold: Stochastic controlled averaging for federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.358, 0.577, 0.781, 0.593]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 394, "edu_l1_label": "EDU_O"}, {"txt": "arXiv e-prints, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.781, 0.577, 0.824, 0.592], [0.193, 0.591, 0.272, 0.606]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 395, "edu_l1_label": "EDU_O"}, {"txt": "arXiv–1910, 2019.Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.272, 0.591, 0.398, 0.606], [0.176, 0.615, 0.824, 0.63], [0.193, 0.629, 0.222, 0.644]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 396, "edu_l1_label": "EDU_O"}, {"txt": "What’s hidden in a randomly weighted neural network?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.222, 0.629, 0.598, 0.644]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 397, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVFconference on computer vision and pattern recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.598, 0.629, 0.824, 0.644], [0.193, 0.643, 0.579, 0.658]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 398, "edu_l1_label": "EDU_O"}, {"txt": "11893–11902, 2020.Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.579, 0.643, 0.717, 0.658], [0.176, 0.666, 0.697, 0.682]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 399, "edu_l1_label": "EDU_O"}, {"txt": "Hierarchical textconditional image generation with clip latents.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.697, 0.666, 0.824, 0.682], [0.193, 0.68, 0.495, 0.695]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 400, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2204.06125, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.495, 0.68, 0.761, 0.695]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 401, "edu_l1_label": "EDU_O"}, {"txt": "Mohammad Rasouli, Tao Sun, and Ram Rajagopal.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.704, 0.516, 0.719]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 402, "edu_l1_label": "EDU_O"}, {"txt": "Fedgan: Federated generative adversarial networks for distributed data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.516, 0.704, 0.824, 0.719], [0.193, 0.718, 0.364, 0.733]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 403, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2006.07228, 2020.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.364, 0.718, 0.631, 0.733]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 404, "edu_l1_label": "EDU_O"}, {"txt": "Jorma Rissanen and Glen G Langdon.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.176, 0.742, 0.42, 0.757]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 405, "edu_l1_label": "EDU_O"}, {"txt": "Arithmetic coding.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.42, 0.742, 0.546, 0.757]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 406, "edu_l1_label": "EDU_O"}, {"txt": "IBM Journal of research and development, 23(2):149–162, 1979.Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjo¨rn Ommer.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.546, 0.742, 0.824, 0.757], [0.193, 0.756, 0.334, 0.771], [0.176, 0.779, 0.774, 0.794]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 407, "edu_l1_label": "EDU_O"}, {"txt": "High-resolution image synthesis with latent diffusion models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.774, 0.779, 0.824, 0.794], [0.193, 0.793, 0.555, 0.808]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 408, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.555, 0.793, 0.824, 0.808], [0.193, 0.807, 0.538, 0.822]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 409, "edu_l1_label": "EDU_O"}, {"txt": "10684–10695, 2022.Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.538, 0.807, 0.676, 0.822], [0.176, 0.831, 0.824, 0.846], [0.193, 0.845, 0.402, 0.86]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 410, "edu_l1_label": "EDU_O"}, {"txt": "Palette: Image-to-image diffusion models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.402, 0.845, 0.683, 0.86]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 411, "edu_l1_label": "EDU_O"}, {"txt": "In ACM SIGGRAPH 2022 Conference Proceedings, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.683, 0.845, 0.824, 0.86], [0.193, 0.858, 0.417, 0.874]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 412, "edu_l1_label": "EDU_O"}, {"txt": "1–10, 2022.Cicero Nogueira dos Santos, Youssef Mroueh, Inkit Padhi, and Pierre Dognin.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.417, 0.858, 0.498, 0.874], [0.176, 0.882, 0.699, 0.897]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 413, "edu_l1_label": "EDU_O"}, {"txt": "Learning implicit generative models by matching perceptual features.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.699, 0.882, 0.824, 0.897], [0.193, 0.896, 0.535, 0.911]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 414, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.535, 0.896, 0.824, 0.911], [0.193, 0.91, 0.472, 0.925]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 415, "edu_l1_label": "EDU_O"}, {"txt": "4461–4470, 2019.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9200000, "bbox": [[0.472, 0.91, 0.594, 0.925]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 0, "global_sentence_id": 416, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.493, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 417, "edu_l1_label": "EDU_O"}, {"txt": "Jaejung Seol, Seojun Kim, and Jaejun Yoo.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.104, 0.457, 0.119]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 418, "edu_l1_label": "EDU_O"}, {"txt": "Posterllama: Bridging design ability of langauge model to contents-aware layout generation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.457, 0.104, 0.824, 0.119], [0.193, 0.118, 0.429, 0.133]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 419, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2404.00995, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.429, 0.118, 0.696, 0.133]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 420, "edu_l1_label": "EDU_O"}, {"txt": "Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.141, 0.599, 0.156]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 421, "edu_l1_label": "EDU_O"}, {"txt": "Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.599, 0.141, 0.824, 0.156], [0.193, 0.155, 0.602, 0.17]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 422, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.602, 0.155, 0.824, 0.17], [0.193, 0.168, 0.592, 0.184]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 423, "edu_l1_label": "EDU_O"}, {"txt": "3626–3636, 2022.Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.592, 0.168, 0.714, 0.184], [0.176, 0.191, 0.824, 0.206]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 424, "edu_l1_label": "EDU_O"}, {"txt": "Will we run out of data?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.193, 0.205, 0.357, 0.22]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 425, "edu_l1_label": "EDU_O"}, {"txt": "limits of llm scaling based on human-generated data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.357, 0.205, 0.715, 0.22]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 426, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint, 2024.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.715, 0.205, 0.824, 0.22], [0.193, 0.219, 0.229, 0.234]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 427, "edu_l1_label": "EDU_O"}, {"txt": "Jianyuan Wang, Ceyuan Yang, Yinghao Xu, Yujun Shen, Hongdong Li, and Bolei Zhou.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.241, 0.749, 0.256]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 428, "edu_l1_label": "EDU_O"}, {"txt": "Improving gan equilibrium by raising spatial awareness.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.749, 0.241, 0.824, 0.256], [0.193, 0.255, 0.493, 0.27]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 429, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.493, 0.255, 0.824, 0.27], [0.193, 0.269, 0.493, 0.284]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 430, "edu_l1_label": "EDU_O"}, {"txt": "11285–11293, 2022.Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.493, 0.269, 0.632, 0.284], [0.176, 0.291, 0.615, 0.306]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 431, "edu_l1_label": "EDU_O"}, {"txt": "Subsampled re´nyi differentialprivacy and analytical moments accountant.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.615, 0.291, 0.824, 0.306], [0.193, 0.305, 0.481, 0.32]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 432, "edu_l1_label": "EDU_O"}, {"txt": "In The 22nd International Conference on Artifcial Intelligence and Statistics, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.481, 0.305, 0.824, 0.32], [0.193, 0.319, 0.389, 0.334]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 433, "edu_l1_label": "EDU_O"}, {"txt": "1226–1235.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.389, 0.319, 0.471, 0.334]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 434, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2019.Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.471, 0.319, 0.564, 0.334], [0.176, 0.341, 0.824, 0.357], [0.193, 0.355, 0.328, 0.37]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 435, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning with differential privacy: Algorithms and performance analysis.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.328, 0.355, 0.824, 0.37], [0.193, 0.369, 0.249, 0.384]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 436, "edu_l1_label": "EDU_O"}, {"txt": "IEEE Transactions on Information Forensics and Security, 15:3454–3469, 2020.Bangzhou Xin, Wei Yang, Yangyang Geng, Sheng Chen, Shaowei Wang, and Liusheng Huang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.249, 0.369, 0.779, 0.384], [0.176, 0.392, 0.794, 0.407]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 437, "edu_l1_label": "EDU_O"}, {"txt": "Private f-gan: Differential privacy synthetic data generation based on federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.794, 0.392, 0.824, 0.407], [0.193, 0.406, 0.749, 0.421]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 438, "edu_l1_label": "EDU_O"}, {"txt": "In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.749, 0.406, 0.824, 0.421], [0.193, 0.419, 0.824, 0.435], [0.193, 0.433, 0.213, 0.448]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 439, "edu_l1_label": "EDU_O"}, {"txt": "2927–2931.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.213, 0.433, 0.294, 0.448]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 440, "edu_l1_label": "EDU_O"}, {"txt": "IEEE, 2020.Sangyeop Yeo, Yoojin Jang, Jy-yong Sohn, Dongyoon Han, and Jaejun Yoo.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.294, 0.433, 0.379, 0.448], [0.176, 0.456, 0.686, 0.471]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 441, "edu_l1_label": "EDU_O"}, {"txt": "Can we fnd strong lottery tickets in generative models?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.686, 0.456, 0.824, 0.471], [0.193, 0.47, 0.429, 0.485]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 442, "edu_l1_label": "EDU_O"}, {"txt": "In Proceedings of the AAAI Conference on Artifcial Intelligence, volume 37, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.429, 0.47, 0.824, 0.485], [0.193, 0.483, 0.336, 0.499]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 443, "edu_l1_label": "EDU_O"}, {"txt": "3267–3275, 2023.Liping Yi, Wang Gang, and Liu Xiaoguang.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.336, 0.483, 0.458, 0.499], [0.176, 0.506, 0.465, 0.521]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 444, "edu_l1_label": "EDU_O"}, {"txt": "Qsf: A two-level uplink communication optimization framework for federated learning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.465, 0.506, 0.824, 0.521], [0.193, 0.52, 0.412, 0.535]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 445, "edu_l1_label": "EDU_O"}, {"txt": "In International Conference on Machine Learning, pp.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.412, 0.52, 0.771, 0.535]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 446, "edu_l1_label": "EDU_O"}, {"txt": "25501–25513.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.771, 0.52, 0.824, 0.535], [0.193, 0.534, 0.238, 0.549]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 447, "edu_l1_label": "EDU_O"}, {"txt": "PMLR, 2022.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.238, 0.534, 0.331, 0.549]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 448, "edu_l1_label": "EDU_O"}, {"txt": "Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, et al.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.556, 0.824, 0.571], [0.193, 0.57, 0.674, 0.585]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 449, "edu_l1_label": "EDU_O"}, {"txt": "Opacus: User-friendly differential privacy library in pytorch.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.674, 0.57, 0.824, 0.585], [0.193, 0.584, 0.438, 0.599]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 450, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2109.12298, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.438, 0.584, 0.705, 0.599]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 451, "edu_l1_label": "EDU_O"}, {"txt": "Yikai Zhang, Hui Qu, Qi Chang, Huidong Liu, Dimitris Metaxas, and Chao Chen.Training federated gans with theoretical guarantees: A universal aggregation approach.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.606, 0.824, 0.621], [0.193, 0.62, 0.716, 0.635]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 452, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:2102.04655, 2021.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.716, 0.62, 0.824, 0.635], [0.193, 0.634, 0.356, 0.649]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 453, "edu_l1_label": "EDU_O"}, {"txt": "Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.656, 0.749, 0.672]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 454, "edu_l1_label": "EDU_O"}, {"txt": "Federated learning with non-iid data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.749, 0.656, 0.824, 0.672], [0.193, 0.67, 0.365, 0.685]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 455, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1806.00582, 2018.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.365, 0.67, 0.632, 0.685]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 456, "edu_l1_label": "EDU_O"}, {"txt": "Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.176, 0.693, 0.577, 0.708]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 457, "edu_l1_label": "EDU_O"}, {"txt": "Trained ternary quantization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.577, 0.693, 0.778, 0.708]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 458, "edu_l1_label": "EDU_O"}, {"txt": "arXiv preprint arXiv:1612.01064, 2016.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9300000, "bbox": [[0.778, 0.693, 0.824, 0.708], [0.193, 0.707, 0.413, 0.722]]}]}, "tags": ["text"], "label": "reference", "web_segment_id": 1, "global_sentence_id": 459, "edu_l1_label": "EDU_O"}, {"txt": "AAPPENDIX", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.177, 0.745, 0.298, 0.762]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 3, "global_sentence_id": 460, "edu_l1_label": "IOS"}, {"txt": "APRIVACY", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9400000, "bbox": [[0.177, 0.778, 0.285, 0.795]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 2, "global_sentence_id": 461, "edu_l1_label": "EDU_O"}, {"txt": "Diffenrential privacy (DP) and Re´nyi Differential Privacy (RDP) (Dwork et al., 2006;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.176, 0.809, 0.756, 0.825]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 462, "edu_l1_label": "EDU_O"}, {"txt": " Mironov,2017)] are the most popular defnitions to analysis the privacy in FL environments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.756, 0.809, 0.824, 0.825], [0.176, 0.823, 0.744, 0.838]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 463, "edu_l1_label": "EDU_O"}, {"txt": "These help mitigate privacy concerns by limiting the contribution of individual data points.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.744, 0.823, 0.824, 0.838], [0.176, 0.837, 0.687, 0.852]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 464, "edu_l1_label": "EDU_O"}, {"txt": "(ϵ, δ)-DP and (α, ϵ)-RDP basically calculates the distance of outcome for the algorithm of adjacent datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9500000, "bbox": [[0.687, 0.837, 0.824, 0.852], [0.176, 0.851, 0.747, 0.866]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 465, "edu_l1_label": "EDU_O"}, {"txt": "Defnition 2 ((ϵ, δ)-Differential Privacy) A randomized mechanism M : X→R is (ϵ, δ)-differential privacy, if for any two adjacent datasets D, D′ and for any measurable sets S:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9600000, "bbox": [[0.173, 0.877, 0.826, 0.908]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 4, "global_sentence_id": 466, "edu_l1_label": "EDU_O"}, {"txt": "Pr[M(D) ∈ S] ≤ eεPr[M(D′) ∈ S] + ε", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9700000, "bbox": [[0.362, 0.91, 0.639, 0.927]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 5, "global_sentence_id": 467, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.493, 0.95, 0.508, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 468, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9800000, "bbox": [[0.804, 0.912, 0.825, 0.925]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 469, "edu_l1_label": "EDU_O"}, {"txt": "Defnition 3 ((α, ϵ) Re´nyi Differential Privacy) For two probability distributions P and Q, theRe´nyi divergence of order α > 1 defned as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 9900000, "bbox": [[0.176, 0.104, 0.824, 0.119], [0.176, 0.118, 0.515, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 6, "global_sentence_id": 470, "edu_l1_label": "EDU_O"}, {"txt": "R _ { \\alpha } ( P | | Q ) \\triangleq \\frac { 1 } { \\alpha - 1 } \\operatorname { l o g } \\mathbb { E } _ { x \\sim Q } ( \\frac { P ( x ) } { Q ( x ) } ) ^ { \\alpha } .", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10000000, "bbox": [[0.362, 0.137, 0.63, 0.17]]}]}, "tags": ["equation"], "label": "formula", "web_segment_id": 16, "global_sentence_id": 471, "edu_l1_label": "EDU_O"}, {"txt": "(4)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10000000, "bbox": [[0.805, 0.147, 0.824, 0.162]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 11, "global_sentence_id": 472, "edu_l1_label": "EDU_O"}, {"txt": "then, a randomized mechanism M :", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.176, 0.175, 0.422, 0.19]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 473, "edu_l1_label": "EDU_O"}, {"txt": " X →R is (α, ϵ) Re´nyi differential privacy, if for any twoadjacent datasets D, D′ and for any measurable sets S:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10100000, "bbox": [[0.422, 0.175, 0.824, 0.19], [0.176, 0.189, 0.541, 0.204]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 5, "global_sentence_id": 474, "edu_l1_label": "EDU_O"}, {"txt": "R&(M(D)||M(D′))≤≤ e", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.416, 0.226, 0.587, 0.242]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 15, "global_sentence_id": 475, "edu_l1_label": "EDU_O"}, {"txt": "(5)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10200000, "bbox": [[0.805, 0.224, 0.824, 0.24]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 9, "global_sentence_id": 476, "edu_l1_label": "EDU_O"}, {"txt": "Theorem 1 (Mironov, 2017)] showed that if M is (α, ϵ)-RDP guarantee, is also (ϵ +loαg− 11/ δ)-DP.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10300000, "bbox": [[0.176, 0.254, 0.744, 0.268], [0.744, 0.253, 0.82, 0.27]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 12, "global_sentence_id": 477, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we provide more detailed explanation of privacy preserving in PRISM and also present updated results when DP is applied.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.176, 0.283, 0.824, 0.298], [0.176, 0.296, 0.48, 0.312]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 478, "edu_l1_label": "EDU_O"}, {"txt": "To satisfy the (ϵ, δ)-DP, our goal is privatiz2e theϵ2∆2 = maxD,D′ ||M(D) −M(D′)||2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.48, 0.296, 0.824, 0.312], [0.746, 0.324, 0.757, 0.33], [0.176, 0.334, 0.427, 0.347]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 479, "edu_l1_label": "EDU_O"}, {"txt": "When the local training is end, each client has scores s ∈Rdto choose which weight to prune.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.427, 0.334, 0.823, 0.341], [0.176, 0.347, 0.398, 0.362]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 480, "edu_l1_label": "EDU_O"}, {"txt": "Recall that probability θ ∈[0, 1]d can be obtained through sig-moid function.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.398, 0.347, 0.824, 0.362], [0.176, 0.363, 0.275, 0.378]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 481, "edu_l1_label": "EDU_O"}, {"txt": "We inject gaussian noise and clip to θ˜ ∈[c, 1 −c]d, where c is a small value0 < c <12.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.275, 0.363, 0.824, 0.378], [0.176, 0.381, 0.26, 0.394]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 482, "edu_l1_label": "EDU_O"}, {"txt": "In out setup, we fx it at 0.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.26, 0.381, 0.459, 0.394]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 483, "edu_l1_label": "EDU_O"}, {"txt": "Now, we ensure θ˜ is (ϵ, δ)-DP.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.459, 0.381, 0.669, 0.394]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 484, "edu_l1_label": "EDU_O"}, {"txt": "For a fair comparison,we use (ϵ, δ) = (9.8, 10−5) to PRISM and our baselines in all of our experiments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.669, 0.381, 0.824, 0.394], [0.176, 0.395, 0.72, 0.41]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 485, "edu_l1_label": "EDU_O"}, {"txt": "In addition, weregulate the global round to ensure that the overall privacy budget does not exceed ϵ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.72, 0.395, 0.824, 0.41], [0.176, 0.409, 0.739, 0.424]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 486, "edu_l1_label": "EDU_O"}, {"txt": "To track theoverall privacy budget, we employ subsampled moments accountant (Wang et al., 2019)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.739, 0.409, 0.824, 0.424], [0.176, 0.423, 0.749, 0.438]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 487, "edu_l1_label": "EDU_O"}, {"txt": "We refer to the Opacus library which is the user-friendly pytorch framework for differential privacy (Yousefpour et al., 2021)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10400000, "bbox": [[0.749, 0.423, 0.824, 0.438], [0.176, 0.437, 0.824, 0.452], [0.176, 0.45, 0.264, 0.466]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 488, "edu_l1_label": "EDU_O"}, {"txt": "(Imola & Chaudhuri, 2021;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10500000, "bbox": [[0.176, 0.472, 0.353, 0.487]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 489, "edu_l1_label": "EDU_O"}, {"txt": " Isik et al., 2022)] have shown that performing post processing to already privatized vector θ˜ such as Bernoulli sampling enjoys privacy amplifcation under some conditions.By doing so, the overall privacy budget becomes smaller ϵamp ≤min{ϵ, dγα(c)}, where γα(·) isthe binary symmetry Re´nyi divergence as expressed below:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10500000, "bbox": [[0.353, 0.472, 0.824, 0.487], [0.176, 0.487, 0.824, 0.503], [0.176, 0.501, 0.824, 0.516], [0.176, 0.515, 0.562, 0.53]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 490, "edu_l1_label": "EDU_O"}, {"txt": "%(c) = ;", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.336, 0.555, 0.4, 0.572]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 14, "global_sentence_id": 491, "edu_l1_label": "EDU_O"}, {"txt": "(6)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10600000, "bbox": [[0.805, 0.554, 0.824, 0.57]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 10, "global_sentence_id": 492, "edu_l1_label": "EDU_O"}, {"txt": "where α refers to the order of the divergence.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.176, 0.578, 0.474, 0.593]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 493, "edu_l1_label": "EDU_O"}, {"txt": "Note that d limits the privacy amplifcation when themodel size becomes large.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.474, 0.578, 0.824, 0.593], [0.176, 0.592, 0.351, 0.607]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 494, "edu_l1_label": "EDU_O"}, {"txt": "Since PRISM assumes that the model size is large enough due to SLT, we focus on communication effciency rather than privacy amplifcation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10700000, "bbox": [[0.351, 0.592, 0.824, 0.607], [0.176, 0.606, 0.648, 0.621]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 495, "edu_l1_label": "EDU_O"}, {"txt": "BPRIVACY AMPLIFIED SCENARIO", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10800000, "bbox": [[0.177, 0.639, 0.47, 0.657]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 496, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we discuss about potential privacy beneft of PRISM under some conditions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.176, 0.671, 0.771, 0.686]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 497, "edu_l1_label": "EDU_O"}, {"txt": "Typical FL setting, a malicious third party can estimate W tk −W hg from the communicated gradients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.771, 0.671, 0.824, 0.686], [0.176, 0.685, 0.799, 0.7]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 498, "edu_l1_label": "EDU_O"}, {"txt": "Byanalyzing these gradients, an attacker can extract information about the local data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.799, 0.685, 0.824, 0.7], [0.176, 0.699, 0.722, 0.714]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 499, "edu_l1_label": "EDU_O"}, {"txt": "However, with PRISM, only the binary masks M tk and M tg are exchanged, which hinders an attacker’s efforts sincethey do not have access to the synchronized initial weight Winit shared between the client and server.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 10900000, "bbox": [[0.722, 0.699, 0.824, 0.714], [0.176, 0.713, 0.824, 0.728], [0.176, 0.727, 0.824, 0.742]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 500, "edu_l1_label": "EDU_O"}, {"txt": "CTRAINING DETAILS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11000000, "bbox": [[0.177, 0.76, 0.373, 0.778]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 7, "global_sentence_id": 501, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we provide the detailed description of our implementations and experimental settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.176, 0.792, 0.824, 0.807], [0.176, 0.806, 0.212, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 502, "edu_l1_label": "EDU_O"}, {"txt": "In Table 4, we provide the model architectures used in our experiments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.212, 0.806, 0.679, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 503, "edu_l1_label": "EDU_O"}, {"txt": "We use ResNet-based generator and set the local epoch to 100 and learning rate to 0.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.679, 0.806, 0.824, 0.821], [0.176, 0.82, 0.614, 0.835]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 504, "edu_l1_label": "EDU_O"}, {"txt": "In addition, we do not employ training schedulers or learning rate decay.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.614, 0.82, 0.824, 0.835], [0.176, 0.834, 0.458, 0.849]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 505, "edu_l1_label": "EDU_O"}, {"txt": "Our code is based on (Santos et al., 2019; Yeo et al., 2023)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.458, 0.834, 0.824, 0.849], [0.176, 0.848, 0.224, 0.863]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 506, "edu_l1_label": "EDU_O"}, {"txt": "They employ the ImageNet-pretrained VGG19 network for feature matching by minimizing the Eq.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.224, 0.848, 0.824, 0.863], [0.176, 0.861, 0.223, 0.877]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 507, "edu_l1_label": "EDU_O"}, {"txt": "1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.223, 0.861, 0.24, 0.877]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 508, "edu_l1_label": "EDU_O"}, {"txt": "However, calculating the frst and second moments require the large batch size to obtain the accurate statistics.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.24, 0.861, 0.824, 0.877], [0.176, 0.875, 0.319, 0.89]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 509, "edu_l1_label": "EDU_O"}, {"txt": "To address this issue, (Santos et al., 2019)] introduces Adam moving average (AMA).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.319, 0.875, 0.824, 0.89], [0.176, 0.889, 0.229, 0.904]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 510, "edu_l1_label": "EDU_O"}, {"txt": "With a rate λ, the update of AMA m is expressed as follows:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11100000, "bbox": [[0.229, 0.889, 0.63, 0.904]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 511, "edu_l1_label": "EDU_O"}, {"txt": "m ← m - XADAM(m - △),", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11200000, "bbox": [[0.399, 0.912, 0.6, 0.927]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 13, "global_sentence_id": 512, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.493, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 513, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11300000, "bbox": [[0.804, 0.912, 0.825, 0.925]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 514, "edu_l1_label": "EDU_O"}, {"txt": "Table 4: Generator architecture used in our experiments.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11400000, "bbox": [[0.316, 0.102, 0.683, 0.116]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 515, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11500000, "bbox": [[0.208, 0.127, 0.789, 0.436]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 516, "edu_l1_label": "EDU_O"}, {"txt": "where ADAM denotes Adam optimizer (Kingma & Ba, 2014)] and ∆is the discrepancy of themeans of the extracted features. Note that ADAM(m −∆) can be interpreted as gradient descent byminimizing the L2 loss:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11600000, "bbox": [[0.175, 0.456, 0.829, 0.499]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 7, "global_sentence_id": 517, "edu_l1_label": "EDU_O"}, {"txt": "min", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.439, 0.507, 0.474, 0.518]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 10, "global_sentence_id": 518, "edu_l1_label": "EDU_O"}, {"txt": "m", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11700000, "bbox": [[0.442, 0.513, 0.482, 0.528]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 9, "global_sentence_id": 519, "edu_l1_label": "EDU_O"}, {"txt": "This means the difference of statistics (m −∆) is passed through a single MLP layer and updatedusing the Adam optimizer to the direction of minimizing Eq.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.176, 0.527, 0.824, 0.543], [0.176, 0.541, 0.586, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 520, "edu_l1_label": "EDU_O"}, {"txt": "8.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.586, 0.541, 0.604, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 521, "edu_l1_label": "EDU_O"}, {"txt": "By utilizing AMA, Eq.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.604, 0.541, 0.764, 0.556]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 522, "edu_l1_label": "EDU_O"}, {"txt": "1 is for 2  2   Algorithm 1, 2 provides the pseudocode for MADA and PRISM∗correspondingly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.764, 0.541, 0.824, 0.556], [0.555, 0.556, 0.561, 0.565], [0.809, 0.556, 0.815, 0.565], [0.176, 0.58, 0.713, 0.595]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 523, "edu_l1_label": "EDU_O"}, {"txt": "AMA is omittedto simply express the fow of our framework.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.713, 0.58, 0.824, 0.595], [0.176, 0.594, 0.467, 0.609]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 524, "edu_l1_label": "EDU_O"}, {"txt": "See our code for pytorch implementation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.467, 0.594, 0.741, 0.609]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 525, "edu_l1_label": "EDU_O"}, {"txt": "We train the local generator for 100 local iterations with learning rate of 0.1.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.741, 0.594, 0.824, 0.609], [0.176, 0.608, 0.59, 0.623]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 526, "edu_l1_label": "EDU_O"}, {"txt": "For the AMA layer, learning rate is set to 0.005.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.59, 0.608, 0.824, 0.623], [0.176, 0.622, 0.255, 0.637]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 527, "edu_l1_label": "EDU_O"}, {"txt": "In addition, we use the Adam optimizer with β1 = 0.5, β2 = 0.999 to update the scoresof the generators.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.255, 0.622, 0.824, 0.637], [0.176, 0.636, 0.293, 0.651]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 528, "edu_l1_label": "EDU_O"}, {"txt": "After all clients complete their training, communication round is initiated.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.293, 0.636, 0.796, 0.651]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 529, "edu_l1_label": "EDU_O"}, {"txt": "We set the global epoch to 150 for the MNIST dataset and 350 for the CelebA and CIFAR10 datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.796, 0.636, 0.824, 0.651], [0.176, 0.649, 0.824, 0.665]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 530, "edu_l1_label": "EDU_O"}, {"txt": "As we do not adjust the parameters, note that there is room for performance improvements through hyperparameter tuning.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11800000, "bbox": [[0.176, 0.663, 0.824, 0.678], [0.176, 0.677, 0.328, 0.692]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 531, "edu_l1_label": "EDU_O"}, {"txt": "DDATASET PARTITIONING STRATEGIES ACROSS CLIENTS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 11900000, "bbox": [[0.177, 0.711, 0.664, 0.728]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 8, "global_sentence_id": 532, "edu_l1_label": "EDU_O"}, {"txt": "Here, we elaborate on the dataset partitioning strategies used to simulate the IID and non-IID setup.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.176, 0.743, 0.824, 0.758]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 533, "edu_l1_label": "EDU_O"}, {"txt": "For non-IID setup, we used two strategies for partitioning datasets: 1) Shards-partitioned and 2) Dirichlet-partitioned.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12000000, "bbox": [[0.176, 0.756, 0.824, 0.772], [0.176, 0.77, 0.314, 0.785]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 534, "edu_l1_label": "EDU_O"}, {"txt": "IID setup.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.176, 0.791, 0.247, 0.808]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 535, "edu_l1_label": "EDU_O"}, {"txt": "In the case of IID scenario, we assign equal-sized local datasets by uniformly sampling from the entire training set.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.247, 0.791, 0.824, 0.807], [0.176, 0.806, 0.354, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 536, "edu_l1_label": "EDU_O"}, {"txt": "This allows for a balanced distribution of data across the clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12100000, "bbox": [[0.354, 0.806, 0.777, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 537, "edu_l1_label": "EDU_O"}, {"txt": "Non-IID setup (Shards-partitioned).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.176, 0.826, 0.432, 0.843]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 538, "edu_l1_label": "EDU_O"}, {"txt": "In Section 5.2 and Section 5.3, we partition the MNIST, FMNIST, and CIFAR10 datasets into 40 segments based on the sorted class labels and randomly assign four segments to each client.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.432, 0.826, 0.824, 0.842], [0.176, 0.841, 0.824, 0.856], [0.176, 0.855, 0.411, 0.87]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 539, "edu_l1_label": "EDU_O"}, {"txt": "For the CelebA dataset, which contains multiple attributes per image, defning a clear non-IID distribution for splitting is inherently ambiguous.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.411, 0.855, 0.824, 0.87], [0.176, 0.868, 0.711, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 540, "edu_l1_label": "EDU_O"}, {"txt": "In this work, we divide the dataset into tow partitions based on a pivotal attribute (gender, in our case).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.711, 0.868, 0.824, 0.884], [0.176, 0.882, 0.758, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 541, "edu_l1_label": "EDU_O"}, {"txt": "The total number of partitions corresponds to the number of clients, with each client being assigned either the positive or negative subset of images for the pivotal attribute.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.758, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.577, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 542, "edu_l1_label": "EDU_O"}, {"txt": "However, the remaining 39 attributes", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12200000, "bbox": [[0.577, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 543, "edu_l1_label": "EDU_O"}, {"txt": "(8)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12300000, "bbox": [[0.805, 0.504, 0.824, 0.519]]}]}, "tags": ["text"], "label": "formula", "web_segment_id": 6, "global_sentence_id": 544, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12400000, "bbox": [[0.493, 0.949, 0.509, 0.961]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 545, "edu_l1_label": "EDU_O"}, {"txt": "Algorithm 1 MADA", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12500000, "bbox": [[0.176, 0.103, 0.314, 0.119]]}]}, "tags": ["title"], "label": "code_title", "web_segment_id": 25, "global_sentence_id": 546, "edu_l1_label": "EDU_O"}, {"txt": "Kk=1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.331, 0.137, 0.355, 0.154]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 17, "global_sentence_id": 547, "edu_l1_label": "EDU_O"}, {"txt": "Server execute:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.164, 0.311, 0.181]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 20, "global_sentence_id": 548, "edu_l1_label": "EDU_O"}, {"txt": "Initialize a random weight Winit and score vector s, then broadcasts to all clients.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.178, 0.737, 0.194]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 12, "global_sentence_id": 549, "edu_l1_label": "EDU_O"}, {"txt": "for round t = 1, ..., T do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.192, 0.364, 0.208]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 26, "global_sentence_id": 550, "edu_l1_label": "EDU_O"}, {"txt": "Client_side:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.227, 0.206, 0.311, 0.222]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 44, "global_sentence_id": 551, "edu_l1_label": "EDU_O"}, {"txt": "for each client k ∈ [1, K] do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.227, 0.221, 0.419, 0.236]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 43, "global_sentence_id": 552, "edu_l1_label": "EDU_O"}, {"txt": "s↑ = st", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.251, 0.232, 0.307, 0.251]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 42, "global_sentence_id": 553, "edu_l1_label": "EDU_O"}, {"txt": "for local iteration i = 1, , , L do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.25, 0.247, 0.465, 0.264]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 41, "global_sentence_id": 554, "edu_l1_label": "EDU_O"}, {"txt": "Sigmoid(s)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.274, 0.258, 0.405, 0.282]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 40, "global_sentence_id": 555, "edu_l1_label": "EDU_O"}, {"txt": "Mε~Bern(θ)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.275, 0.271, 0.388, 0.295]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 39, "global_sentence_id": 556, "edu_l1_label": "EDU_O"}, {"txt": "Ws ← Winit ⊙ M", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.274, 0.285, 0.415, 0.309]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 38, "global_sentence_id": 557, "edu_l1_label": "EDU_O"}, {"txt": "DI:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.275, 0.301, 0.301, 0.317]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 37, "global_sentence_id": 558, "edu_l1_label": "EDU_O"}, {"txt": "▷Download score vector", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.661, 0.236, 0.824, 0.249]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 10, "global_sentence_id": 559, "edu_l1_label": "EDU_O"}, {"txt": "DEkxftarkaec t← kreal at nd fkake featuresk ψ(Dk)k, ψ(Dfkake)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.277, 0.305, 0.589, 0.335]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 19, "global_sentence_id": 560, "edu_l1_label": "EDU_O"}, {"txt": "st-------nvC雷(Dk),w(D?", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.275, 0.332, 0.562, 0.359]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 36, "global_sentence_id": 561, "edu_l1_label": "EDU_O"}, {"txt": "okc))", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.275, 0.332, 0.562, 0.359]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 36, "global_sentence_id": 562, "edu_l1_label": "EDU_O"}, {"txt": "end for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.252, 0.353, 0.304, 0.367]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 35, "global_sentence_id": 563, "edu_l1_label": "EDU_O"}, {"txt": "Sigmoid(s↑)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.251, 0.365, 0.379, 0.384]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 34, "global_sentence_id": 564, "edu_l1_label": "EDU_O"}, {"txt": "θ′ = θ θ + N(0, Iα2)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.25, 0.382, 0.414, 0.4]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 33, "global_sentence_id": 565, "edu_l1_label": "EDU_O"}, {"txt": "Clip to [c, 1-c]", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.251, 0.398, 0.351, 0.415]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 32, "global_sentence_id": 566, "edu_l1_label": "EDU_O"}, {"txt": "Mε→Bern(倍))", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.25, 0.41, 0.369, 0.431]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 31, "global_sentence_id": 567, "edu_l1_label": "EDU_O"}, {"txt": "Uple", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.249, 0.424, 0.281, 0.443]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 30, "global_sentence_id": 568, "edu_l1_label": "EDU_O"}, {"txt": "end for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.229, 0.439, 0.278, 0.456]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 8, "global_sentence_id": 569, "edu_l1_label": "EDU_O"}, {"txt": "oad binary mask M", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.281, 0.424, 0.504, 0.443]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 29, "global_sentence_id": 570, "edu_l1_label": "EDU_O"}, {"txt": "▷Generate fake images", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.671, 0.305, 0.824, 0.318]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 9, "global_sentence_id": 571, "edu_l1_label": "EDU_O"}, {"txt": "▷Update local score vector", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.646, 0.34, 0.824, 0.353]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 2, "global_sentence_id": 572, "edu_l1_label": "EDU_O"}, {"txt": "▷Aggregate the received binary masks", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.571, 0.47, 0.824, 0.482]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 5, "global_sentence_id": 573, "edu_l1_label": "EDU_O"}, {"txt": "Serverside:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.225, 0.452, 0.313, 0.47]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 28, "global_sentence_id": 574, "edu_l1_label": "EDU_O"}, {"txt": "θˆt+1 ←PKk=1 M tkst+1 ←Sigmoid−1(θˆt+1)λ ←hd(Mt, Bern(θˆt+1))st+1 ←(1 −λ)st + λst+1", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.229, 0.47, 0.353, 0.477], [0.229, 0.487, 0.399, 0.499], [0.229, 0.503, 0.403, 0.516], [0.229, 0.517, 0.406, 0.53]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 14, "global_sentence_id": 575, "edu_l1_label": "EDU_O"}, {"txt": "▷Compute the hamming distance", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.605, 0.503, 0.824, 0.516]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 4, "global_sentence_id": 576, "edu_l1_label": "EDU_O"}, {"txt": "end for", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.528, 0.254, 0.544]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 15, "global_sentence_id": 577, "edu_l1_label": "EDU_O"}, {"txt": "Sample the supermask M ∗∼Bern(θT )Obtain the fnal model W ∗←Winit ⊙M ∗", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.542, 0.464, 0.557], [0.204, 0.556, 0.487, 0.572]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 11, "global_sentence_id": 578, "edu_l1_label": "EDU_O"}, {"txt": "Input:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.176, 0.616, 0.221, 0.633]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 579, "edu_l1_label": "EDU_O"}, {"txt": " ratio of score layer αOutput: probability θtk (100 −α) and binary mask M tk (α)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.221, 0.616, 0.366, 0.632], [0.176, 0.63, 0.565, 0.646]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 6, "global_sentence_id": 580, "edu_l1_label": "EDU_O"}, {"txt": "Client side:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.658, 0.283, 0.674]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 13, "global_sentence_id": 581, "edu_l1_label": "EDU_O"}, {"txt": "for layer l = 1, ..., L do", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.204, 0.672, 0.358, 0.688]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 23, "global_sentence_id": 582, "edu_l1_label": "EDU_O"}, {"txt": "if IsScoreLayer(l,α,L) then", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.229, 0.686, 0.408, 0.702]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 22, "global_sentence_id": 583, "edu_l1_label": "EDU_O"}, {"txt": "else", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.227, 0.715, 0.25, 0.729]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 27, "global_sentence_id": 584, "edu_l1_label": "EDU_O"}, {"txt": "Return binary mask M tk (l)", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12600000, "bbox": [[0.253, 0.728, 0.429, 0.742]]}]}, "tags": ["text"], "label": "code", "web_segment_id": 21, "global_sentence_id": 585, "edu_l1_label": "EDU_O"}, {"txt": "are still shared among clients, resulting in relatively weak heterogeneity.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.176, 0.806, 0.667, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 586, "edu_l1_label": "EDU_O"}, {"txt": "This explains why the performance drop on CelebA is not as signifcant compared to the IID case.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.667, 0.806, 0.824, 0.821], [0.176, 0.819, 0.686, 0.835]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 587, "edu_l1_label": "EDU_O"}, {"txt": "For a more detailed discussion, please refer to Appendix D.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12700000, "bbox": [[0.686, 0.819, 0.824, 0.835], [0.176, 0.833, 0.433, 0.848]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 588, "edu_l1_label": "EDU_O"}, {"txt": "Non-IID setup (Dirichlet-partitioned).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.176, 0.854, 0.442, 0.87]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 589, "edu_l1_label": "EDU_O"}, {"txt": "We further explore a more realistic and challenging nonIID scenario, where datasets are partitioned using Dirichlet distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.442, 0.854, 0.824, 0.87], [0.176, 0.868, 0.636, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 590, "edu_l1_label": "EDU_O"}, {"txt": "Specifcally, we set Dirichlet parameter α = 0.005 to create a more label-skewed distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.636, 0.868, 0.824, 0.884], [0.176, 0.882, 0.613, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 591, "edu_l1_label": "EDU_O"}, {"txt": "For CelebA, we assign data toclients such that each client possesses the pivotal attribute in different proportions using Dirichlet distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.613, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.255, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 592, "edu_l1_label": "EDU_O"}, {"txt": "For example, client 1 has 60% male and 40% female, while client 2 has 20% male and", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12800000, "bbox": [[0.255, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 593, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 12900000, "bbox": [[0.493, 0.949, 0.508, 0.961]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 594, "edu_l1_label": "EDU_O"}, {"txt": "Table 5: Quantitative comparison in Dirichlet non-IID scenario with (ϵ, δ) = (9.8, 10−5). Wecompare FID, P&R, D&C. † indicates that MADA is removed.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13000000, "bbox": [[0.173, 0.102, 0.827, 0.129]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 4, "global_sentence_id": 595, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13100000, "bbox": [[0.239, 0.14, 0.775, 0.339]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 3, "global_sentence_id": 596, "edu_l1_label": "EDU_O"}, {"txt": "Table 6: Quantitative comparison in Dirichlet non-IID scenario without DP. We compare FID, P&R, D&C. † indicates that MADA is removed. We set α = 80 for PRISM∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13200000, "bbox": [[0.175, 0.356, 0.824, 0.385]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 0, "global_sentence_id": 597, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13300000, "bbox": [[0.177, 0.396, 0.825, 0.65]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 598, "edu_l1_label": "EDU_O"}, {"txt": "80% female. To the best of our knowledge, methods for splitting multi-label datasets into extreme non-IID scenarios have not been extensively explored. Table 5 and Table 6 present a quantitative comparison of baselines and our methods under both with and without DP. PRISM demonstrates robust performance even under highly heterogeneity distributions. Note that Multi-FLGAN fails to successfully train generative model under highly heterogeneity.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13400000, "bbox": [[0.173, 0.668, 0.826, 0.74]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 599, "edu_l1_label": "EDU_O"}, {"txt": "EEFFICIENT STORAGE AND QUANTIZATION STRATEGY IN PRISM", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13500000, "bbox": [[0.177, 0.759, 0.743, 0.777]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 600, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we provide further detail description regarding storage effciency of PRISM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.176, 0.792, 0.776, 0.808]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 601, "edu_l1_label": "EDU_O"}, {"txt": "Aforementioned in Section 4.1, the fnal model can be expressed as W ∗= Winit ⊙M ∗.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.776, 0.792, 0.824, 0.808], [0.176, 0.806, 0.729, 0.821]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 602, "edu_l1_label": "EDU_O"}, {"txt": "Since Mfianlis lightweight binary masks, we need to focus on the storage of Winit.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.729, 0.806, 0.823, 0.822], [0.176, 0.82, 0.663, 0.835]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 603, "edu_l1_label": "EDU_O"}, {"txt": "Note that Winit is initialized as signed constant using Kaiming Normal distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.663, 0.82, 0.824, 0.835], [0.176, 0.834, 0.594, 0.849]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 604, "edu_l1_label": "EDU_O"}, {"txt": "In other words, the initialized parameter of layer l becomes {+p2/nl−1, −p2/nl−1, · · · , +p2/nl−1} ∈Rd.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.594, 0.834, 0.824, 0.849], [0.176, 0.85, 0.696, 0.865]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 605, "edu_l1_label": "EDU_O"}, {"txt": "This design makesstoring the scaling factor v =2/nl−1 be enough to represent each parameter, represented asWinit = v ⊙Msign, where Msipgn is defned as {+1, −1, · · · , +1} ∈Rd.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.696, 0.85, 0.824, 0.865], [0.176, 0.866, 0.824, 0.882], [0.176, 0.884, 0.675, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 606, "edu_l1_label": "EDU_O"}, {"txt": "Our memory-effcientapproach is closely related to ternary quantization (Zhu et al., 2016)], which typically compresses neural network weights into {−1, 0, 1} through thresholding and projection.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.675, 0.884, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.675, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 607, "edu_l1_label": "EDU_O"}, {"txt": "However, PRISM dis-", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13600000, "bbox": [[0.675, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 608, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.493, 0.95, 0.508, 0.96]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 609, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13700000, "bbox": [[0.173, 0.099, 0.491, 0.258]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 5, "global_sentence_id": 610, "edu_l1_label": "EDU_O"}, {"txt": "Figure 7: Analysis of PRISM∗using CIFAR10 dataset. The backward path selects α% of scorelayers from deeper layers, closer to the output, while the forward path chooses from the opposite end, nearer to the input. Solid-line demonstrates FID following each direction while dash-line shows communication cost (MB) of each path.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13800000, "bbox": [[0.173, 0.27, 0.827, 0.328]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 1, "global_sentence_id": 611, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 13900000, "bbox": [[0.509, 0.099, 0.825, 0.245]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 4, "global_sentence_id": 612, "edu_l1_label": "EDU_O"}, {"txt": "(b) Non-IID case", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.612, 0.246, 0.722, 0.259]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 11, "global_sentence_id": 613, "edu_l1_label": "EDU_O"}, {"txt": "Less cost Less quality", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14000000, "bbox": [[0.182, 0.345, 0.254, 0.37]]}]}, "tags": ["figure_caption"], "label": "O", "web_segment_id": 12, "global_sentence_id": 614, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14100000, "bbox": [[0.175, 0.37, 0.829, 0.502]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 6, "global_sentence_id": 615, "edu_l1_label": "EDU_O"}, {"txt": "Figure 8: The effect of adjusting α of PRISM∗. Qualitative comparison according to α. Note thatα = 0 is identical to PRISM.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14200000, "bbox": [[0.173, 0.51, 0.827, 0.541]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 8, "global_sentence_id": 616, "edu_l1_label": "EDU_O"}, {"txt": "High cost High quality", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14300000, "bbox": [[0.764, 0.345, 0.816, 0.356], [0.749, 0.357, 0.817, 0.369]]}]}, "tags": ["text"], "label": "title3", "web_segment_id": 10, "global_sentence_id": 617, "edu_l1_label": "EDU_O"}, {"txt": "tinguishes itself by directly initialized frozen weights as signed constants using the Kaiming Normal distribution.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.176, 0.556, 0.824, 0.571], [0.176, 0.569, 0.255, 0.585]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 618, "edu_l1_label": "EDU_O"}, {"txt": "Since storing a signed array is negligible in terms of memory overhead, this design makes the fnal model exceptionally lightweight without extra pruning or quantization.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14400000, "bbox": [[0.255, 0.569, 0.824, 0.585], [0.176, 0.583, 0.741, 0.598]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 619, "edu_l1_label": "EDU_O"}, {"txt": "FANALYSIS OF HYBRID AGGREGATION", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14500000, "bbox": [[0.177, 0.618, 0.517, 0.635]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 620, "edu_l1_label": "IOS"}, {"txt": "In this section, we further analyze PRISM∗, which leverages both binary mask and score communi-cations.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.176, 0.65, 0.824, 0.665], [0.176, 0.664, 0.227, 0.679]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 621, "edu_l1_label": "IOS"}, {"txt": "To explore the trade-off between communication cost and generative capability, we consider two strategies: the backward path and forward path.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.227, 0.664, 0.824, 0.679], [0.176, 0.677, 0.515, 0.693]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 622, "edu_l1_label": "EDU_O"}, {"txt": "The backward path progressively increases the number of score layers from deeper layers to earlier layers.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.515, 0.677, 0.824, 0.693], [0.176, 0.691, 0.577, 0.706]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 623, "edu_l1_label": "EDU_O"}, {"txt": "Conversely, in the forward path, we select score layers from earlier layers to deeper layers.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.577, 0.691, 0.824, 0.706], [0.176, 0.705, 0.533, 0.72]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 624, "edu_l1_label": "EDU_O"}, {"txt": "Figure 7 visually demonstrates the trade-off between communication cost and FID of both strategies.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.533, 0.705, 0.824, 0.72], [0.176, 0.719, 0.54, 0.734]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 625, "edu_l1_label": "EDU_O"}, {"txt": "The FID gradually improves as we increase α values in both cases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.54, 0.719, 0.824, 0.734], [0.176, 0.735, 0.327, 0.748]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 626, "edu_l1_label": "EDU_O"}, {"txt": "Note that the additional communication cost of the backward path tends toincrease more smoothly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.327, 0.735, 0.824, 0.748], [0.176, 0.747, 0.336, 0.762]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 627, "edu_l1_label": "EDU_O"}, {"txt": "Based on these observations, we adopt the backward path for Table 3 and Figure 5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.336, 0.747, 0.824, 0.762], [0.176, 0.76, 0.236, 0.776]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 628, "edu_l1_label": "EDU_O"}, {"txt": "Figure 8 provides a comprehensive comparison across a wide range of α values, showingthat PRISM∗consistently produces high quality images.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.236, 0.76, 0.824, 0.776], [0.176, 0.774, 0.543, 0.789]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 629, "edu_l1_label": "EDU_O"}, {"txt": "Note that all experiments are conducted inprivacy-free scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14600000, "bbox": [[0.543, 0.774, 0.824, 0.789], [0.176, 0.788, 0.317, 0.803]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 630, "edu_l1_label": "EDU_O"}, {"txt": "G COMMUNICATION COST DURING DOWNLINK PROCESS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14700000, "bbox": [[0.177, 0.822, 0.191, 0.841], [0.212, 0.822, 0.657, 0.84]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 9, "global_sentence_id": 631, "edu_l1_label": "IOS"}, {"txt": "In a typical FL setup, the server possesses powerful computational capabilities, whereas the clients do not.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.176, 0.855, 0.824, 0.87], [0.176, 0.868, 0.223, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 632, "edu_l1_label": "IOS"}, {"txt": "As a result, several prior studies primarily focus on addressing the limited bandwidth of clients (Kim et al., 2024a; Hu et al., 2023; Yi et al., 2022)].", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.223, 0.868, 0.824, 0.884], [0.176, 0.882, 0.568, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 633, "edu_l1_label": "IOS"}, {"txt": "Accordingly, we have concentrated on the reducing communication cost during the uplink process, transmitting a foat-type score during downlink to mitigate the information loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.568, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.453, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 634, "edu_l1_label": "IOS"}, {"txt": "Last but not least, binary mask communication can also", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14800000, "bbox": [[0.453, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 635, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.493, 0.95, 0.508, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 636, "edu_l1_label": "EDU_O"}, {"txt": "be employed in the downlink to further enhance communication effciency, despite the occurrence of information loss during the conversion of the global mask into local scores.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 14900000, "bbox": [[0.176, 0.104, 0.824, 0.119], [0.176, 0.118, 0.684, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 7, "global_sentence_id": 637, "edu_l1_label": "EDU_O"}, {"txt": "Table 7:", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.176, 0.144, 0.229, 0.159]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 638, "edu_l1_label": "EDU_O"}, {"txt": " Quantitative results on cross-device environment.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.229, 0.144, 0.573, 0.16]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 639, "edu_l1_label": "EDU_O"}, {"txt": "We set the number of clients to 50 and participant ratio to 20%.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15000000, "bbox": [[0.573, 0.144, 0.824, 0.159], [0.176, 0.158, 0.335, 0.173]]}]}, "tags": ["text"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 640, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15100000, "bbox": [[0.176, 0.19, 0.826, 0.303]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 3, "global_sentence_id": 641, "edu_l1_label": "EDU_O"}, {"txt": "HCROSS-DEVICE ENVIRONMENTS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15200000, "bbox": [[0.177, 0.333, 0.477, 0.35]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 642, "edu_l1_label": "IOS"}, {"txt": "PRISM is designed to provide effcient communication cost, making it ideal for cross-device environments with numerous clients and limited bandwidth.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.176, 0.364, 0.824, 0.379], [0.176, 0.378, 0.553, 0.393]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 643, "edu_l1_label": "IOS"}, {"txt": "Typically, cross-device setting involves a large number of clients and an unreliable network, such as those encountered in iPhone and wearable devices.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.553, 0.378, 0.824, 0.393], [0.176, 0.392, 0.824, 0.407], [0.176, 0.406, 0.229, 0.421]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 644, "edu_l1_label": "IOS"}, {"txt": "This complexity makes it diffcult to ensure that all clients in every communication round.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.229, 0.406, 0.824, 0.421]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 645, "edu_l1_label": "IOS"}, {"txt": "To simulate this cross-device environment, we set the number of clients to 50 and the participant ratio as 20%.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.176, 0.42, 0.824, 0.435], [0.176, 0.433, 0.261, 0.449]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 646, "edu_l1_label": "IOS"}, {"txt": "Table 7 shows the quantitative results using MNIST dataset, both with and without DP.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.261, 0.433, 0.824, 0.449]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 647, "edu_l1_label": "IOS"}, {"txt": "PRISM demonstrates a clear performance advantage over other baselines and is well-suited for the cross-device environment.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15300000, "bbox": [[0.176, 0.447, 0.824, 0.462], [0.176, 0.461, 0.347, 0.476]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 2, "global_sentence_id": 648, "edu_l1_label": "IOS"}, {"txt": "IANALYSIS OF CONVERGENCE OF λ IN MADA", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15400000, "bbox": [[0.177, 0.495, 0.581, 0.513]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 6, "global_sentence_id": 649, "edu_l1_label": "IOS"}, {"txt": "In this section, we provide a detailed discussion on how the reduction in λ, which measures thesimilarity between two consecutive global masks, helps mitigate client drift.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15500000, "bbox": [[0.176, 0.526, 0.824, 0.541], [0.176, 0.54, 0.672, 0.555]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 650, "edu_l1_label": "IOS"}, {"txt": "One possible explanation is that MADA implicitly modulates the learning rate for global updates on the server-side without requiring an explicit learning rate scheduler.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.176, 0.561, 0.824, 0.576], [0.176, 0.575, 0.617, 0.59]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 651, "edu_l1_label": "IOS"}, {"txt": "This behavior reduces the need for client-side learning rate scheduling to regularize local objectives.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.617, 0.575, 0.824, 0.59], [0.176, 0.589, 0.633, 0.604]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 652, "edu_l1_label": "EDU_O"}, {"txt": "To illustrate this, consider a local mask update represented as wkt = wkt−∆tk is local model update from client k. In stan−dard  ·FedkAvg, the global update can be expressedas wFtedAvg = wkt−introducest a1 mask-aw−atre  ·aggregattion metch1anism, and the global model is formulated as wMtAGA =−η · ∆tk.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.633, 0.589, 0.824, 0.604], [0.176, 0.603, 0.482, 0.617], [0.176, 0.619, 0.824, 0.632], [0.176, 0.632, 0.298, 0.647], [0.176, 0.649, 0.824, 0.664], [0.466, 0.667, 0.53, 0.68]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 653, "edu_l1_label": "EDU_O"}, {"txt": "Substituting the local mask update into this−η · λΣk∆tk, indicating that λeffectively scales the g l−obal upkdates st e·p sikze.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.53, 0.667, 0.824, 0.68], [0.621, 0.683, 0.824, 0.696], [0.176, 0.695, 0.473, 0.71]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 654, "edu_l1_label": "EDU_O"}, {"txt": "W−he ·n λ is klarge, thke global model closely follows theclient updates, while a small λ reduces the infuence of client updates, acting as a dampening factor.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15600000, "bbox": [[0.473, 0.695, 0.824, 0.71], [0.176, 0.709, 0.824, 0.724]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 655, "edu_l1_label": "EDU_O"}, {"txt": "The intuition behind this behavior is rooted in the nature or early-stage training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.176, 0.73, 0.705, 0.745]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 656, "edu_l1_label": "IOS"}, {"txt": "During the initial rounds, the model is far from convergence, resulting in large gradient norms and substantial parameter changes.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.705, 0.73, 0.824, 0.745], [0.176, 0.744, 0.824, 0.759], [0.176, 0.758, 0.263, 0.773]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 657, "edu_l1_label": "EDU_O"}, {"txt": "Consequently, consecutive global masks exhibit signifcant differences, leading to a large λ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.263, 0.758, 0.824, 0.773], [0.176, 0.772, 0.228, 0.787]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 658, "edu_l1_label": "EDU_O"}, {"txt": "As training progresses and the model stabilizes, mask updates become more similar, de-creasing λ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.228, 0.772, 0.824, 0.787], [0.176, 0.785, 0.249, 0.801]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 659, "edu_l1_label": "EDU_O"}, {"txt": "This gradual reduction in λ prevents excessive global updates, stabilizing convergenceand mitigating client drift.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.249, 0.785, 0.824, 0.801], [0.176, 0.799, 0.352, 0.814]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 660, "edu_l1_label": "EDU_O"}, {"txt": "We elaborate on the behavior of client drift.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.352, 0.799, 0.652, 0.814]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 661, "edu_l1_label": "EDU_O"}, {"txt": "Even when client-drift is biased toward a specifc dominant client, the update difference for the dominant client naturally decreases from the perspective of global support, whereas the difference from the local solutions of other clients increases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.652, 0.799, 0.824, 0.814], [0.176, 0.813, 0.824, 0.828], [0.176, 0.827, 0.824, 0.842], [0.176, 0.841, 0.324, 0.856]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 662, "edu_l1_label": "EDU_O"}, {"txt": "This pattern is refected in the aggregated model, and the difference between consecutive global models can partially capture this behavior (e.g., indicated by a large λ).", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.324, 0.841, 0.824, 0.856], [0.176, 0.855, 0.763, 0.87]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 663, "edu_l1_label": "EDU_O"}, {"txt": "As train-ing progresses and the model approaches the global optimum across local objectives, the differences between consecutive global models gradually decrease, leading to a reduction in λ. Conversely, inthe absence of client drift, such as in IID scenarios, the model learns fairly across all clients and naturally converges without notable fuctuations in global model differences.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15700000, "bbox": [[0.763, 0.855, 0.824, 0.87], [0.176, 0.868, 0.824, 0.884], [0.176, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.675, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 664, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15800000, "bbox": [[0.492, 0.949, 0.508, 0.961]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 665, "edu_l1_label": "EDU_O"}, {"txt": "In summary, MADA updates the global model in a way that avoids local divergence, gradually reducing global model differences even without learning rate decay.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.176, 0.104, 0.824, 0.119], [0.176, 0.118, 0.6, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 666, "edu_l1_label": "EDU_O"}, {"txt": "As shown in Figure 6, λ decreasessharply in the early stages of training due to signifcant local divergence but gradually converges over time.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.6, 0.118, 0.824, 0.133], [0.176, 0.132, 0.824, 0.147], [0.176, 0.146, 0.209, 0.161]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 667, "edu_l1_label": "EDU_O"}, {"txt": "This result suggests that our understanding aligns with the behavior of λ.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 15900000, "bbox": [[0.209, 0.146, 0.688, 0.161]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 4, "global_sentence_id": 668, "edu_l1_label": "EDU_O"}, {"txt": "Table 8: Comparison of FLOPS between PRISM and other baselines.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16000000, "bbox": [[0.273, 0.173, 0.724, 0.188]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 669, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16100000, "bbox": [[0.319, 0.204, 0.676, 0.25]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 670, "edu_l1_label": "EDU_O"}, {"txt": "JCOMPUTATION COST ANALYSIS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16200000, "bbox": [[0.177, 0.279, 0.464, 0.297]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 5, "global_sentence_id": 671, "edu_l1_label": "BOS"}, {"txt": "One might say that SLT process within PRISM increases computational complexity, particularly in large networks.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.176, 0.312, 0.824, 0.327], [0.176, 0.325, 0.296, 0.341]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 672, "edu_l1_label": "IOS"}, {"txt": "As the number of model parameters grows, the required score parameters also scale linearly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.296, 0.325, 0.824, 0.341], [0.176, 0.339, 0.266, 0.354]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 673, "edu_l1_label": "IOS"}, {"txt": "However, in SLT, weight parameters are not optimized via gradient descent.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.266, 0.339, 0.761, 0.354]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 674, "edu_l1_label": "IOS"}, {"txt": "The only additional computations stem from the sigmoid and Bernoulli processes introduced during the binary masking procedure.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.761, 0.339, 0.824, 0.354], [0.176, 0.353, 0.824, 0.368], [0.176, 0.367, 0.337, 0.382]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 675, "edu_l1_label": "IOS"}, {"txt": "While these operations introduce some additional computational overhead, they remain negligible in the context of GPU-accelerated gradient descent.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.337, 0.367, 0.824, 0.382], [0.176, 0.381, 0.665, 0.396]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 676, "edu_l1_label": "IOS"}, {"txt": "To demonstrate that the additional computational cost is not substantial, we compare foating-point-operations per second (FLOPS) in Table 8.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.665, 0.381, 0.824, 0.396], [0.176, 0.395, 0.824, 0.41], [0.176, 0.408, 0.314, 0.424]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 677, "edu_l1_label": "IOS"}, {"txt": "GS-WGAN exhibits increased FLOPS due to spectral normalization, which is applied to stabilize GAN training.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.314, 0.408, 0.824, 0.424], [0.176, 0.422, 0.421, 0.437]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 678, "edu_l1_label": "IOS"}, {"txt": "In contrast, PRISM, despite incorporating the SLT process, maintains a suffciently low FLOPS, ensuring its computational effciency.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16300000, "bbox": [[0.421, 0.422, 0.824, 0.437], [0.176, 0.436, 0.66, 0.451]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 679, "edu_l1_label": "IOS"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16400000, "bbox": [[0.398, 0.461, 0.604, 0.618]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 10, "global_sentence_id": 680, "edu_l1_label": "EDU_O"}, {"txt": "Figure 9: Generated images of PRISM where ddpm is used instead of MMD loss.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16500000, "bbox": [[0.232, 0.625, 0.768, 0.642]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 9, "global_sentence_id": 681, "edu_l1_label": "EDU_O"}, {"txt": "KSCALABILITY OF PRISM", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16600000, "bbox": [[0.177, 0.668, 0.421, 0.687]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 682, "edu_l1_label": "BOS"}, {"txt": "Due to the challenges associated with training generative models in FL setups, research on federated generative models has primarily focused on simple architectures (e.g., GANs) and datasets, such as MNIST and FMNIST level generation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.176, 0.7, 0.824, 0.716], [0.176, 0.714, 0.824, 0.729], [0.176, 0.728, 0.434, 0.743]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 683, "edu_l1_label": "IOS"}, {"txt": "However, it is important to note that the feld of federated generative models remains far behind the current trend of using more complex and powerful models, such as diffusion models, and large-scale datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.434, 0.728, 0.824, 0.743], [0.176, 0.742, 0.824, 0.757], [0.176, 0.756, 0.516, 0.771]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 684, "edu_l1_label": "IOS"}, {"txt": "This gap arises from the inherent diffculties in adapting these advanced models to FL setups.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.516, 0.756, 0.824, 0.771], [0.176, 0.77, 0.501, 0.785]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 685, "edu_l1_label": "IOS"}, {"txt": "Existing works have struggled to achieve stable performance in challenging setups, such as non-IID and privacy-preserving cases, even with MNISTlevel datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.501, 0.77, 0.824, 0.785], [0.176, 0.783, 0.824, 0.799], [0.176, 0.797, 0.268, 0.812]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 686, "edu_l1_label": "IOS"}, {"txt": "Even for centralized differential privacy generative models (Dockhorn et al., 2022; Jiang et al., 2024)],training generative models is highly challenging and often unstable.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.268, 0.797, 0.824, 0.812], [0.176, 0.811, 0.739, 0.826]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 687, "edu_l1_label": "IOS"}, {"txt": "Due to these obstacles, prior studies have primarily focused on simpler datasets, such as MNIST, and have been largely restricted to GANs.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16700000, "bbox": [[0.739, 0.811, 0.824, 0.826], [0.176, 0.825, 0.824, 0.84], [0.176, 0.839, 0.352, 0.854]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 688, "edu_l1_label": "IOS"}, {"txt": "K.1EXTEND TO DIFFUSION MODEL", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16800000, "bbox": [[0.177, 0.87, 0.436, 0.885]]}]}, "tags": ["title"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 689, "edu_l1_label": "IOS"}, {"txt": "We further investigate the applicability of PRISM to larger and mode powerful generative models, such as diffusion models.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.345, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 690, "edu_l1_label": "IOS"}, {"txt": "Specifcally, we retain the core concept of identifying SLT but replaced", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 16900000, "bbox": [[0.345, 0.91, 0.824, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 691, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.492, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 692, "edu_l1_label": "EDU_O"}, {"txt": "the MMD loss with DDPM loss (Ho et al., 2020)] to guide score updates.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.176, 0.104, 0.669, 0.119]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 693, "edu_l1_label": "IOS"}, {"txt": "Notably, all other confgurations remain unchanged.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.669, 0.104, 0.824, 0.119], [0.176, 0.118, 0.378, 0.133]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 694, "edu_l1_label": "IOS"}, {"txt": "We conduct experiments on the MNIST dataset under a non-IID, privacy-free scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.378, 0.118, 0.824, 0.133], [0.176, 0.132, 0.317, 0.147]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 695, "edu_l1_label": "IOS"}, {"txt": "Figure 9 presents the generated images on the server-side, which exhibit slight noisiness.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.317, 0.132, 0.824, 0.147], [0.176, 0.146, 0.24, 0.161]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 696, "edu_l1_label": "IOS"}, {"txt": "We anticipate that by improving the approach for identifying SLT within diffusion models and further curating the experimental setup, we can achieve more promising results.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.24, 0.146, 0.824, 0.161], [0.176, 0.16, 0.735, 0.175]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 697, "edu_l1_label": "IOS"}, {"txt": "On the other hand, directly applying the MMD loss-based approach used in PRISM, which requires generating a set of samples for each update, poses practical challenges due to the well-known limitations of DDPM, including slow sampling and convergence rates.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.735, 0.16, 0.824, 0.175], [0.176, 0.174, 0.824, 0.189], [0.176, 0.187, 0.824, 0.202], [0.176, 0.201, 0.546, 0.216]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 698, "edu_l1_label": "IOS"}, {"txt": "These limitations lead to signifcant computational costs, making the approach practically infeasible.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.546, 0.201, 0.824, 0.216], [0.176, 0.215, 0.568, 0.23]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 699, "edu_l1_label": "IOS"}, {"txt": "Since our primary focus is lightweight communication cost and stable performance, we leave this future work, effciently incorporating the powerful and diffusion model in federated settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17000000, "bbox": [[0.568, 0.215, 0.824, 0.23], [0.176, 0.229, 0.824, 0.244], [0.176, 0.243, 0.509, 0.258]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 700, "edu_l1_label": "IOS"}, {"txt": "K.2EVALUATION ON LARGE-SCALE DATASETS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17100000, "bbox": [[0.175, 0.287, 0.513, 0.302]]}]}, "tags": ["figure_caption"], "label": "title2", "web_segment_id": 5, "global_sentence_id": 701, "edu_l1_label": "EDU_O"}, {"txt": "figure", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17200000, "bbox": [[0.173, 0.327, 0.831, 0.563]]}]}, "tags": ["figure"], "label": "figure", "web_segment_id": 3, "global_sentence_id": 702, "edu_l1_label": "EDU_O"}, {"txt": "Figure 10: Generated images of CelebA 128x128 dataset and CIFAR100 dataset under non-IID and privacy-free scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17300000, "bbox": [[0.173, 0.57, 0.827, 0.599]]}]}, "tags": ["figure_caption"], "label": "figure_title", "web_segment_id": 4, "global_sentence_id": 703, "edu_l1_label": "EDU_O"}, {"txt": "Table 9: Performance of PRISM using CelebA 128x128 and CIAFR100 datasets under non-IID and privacy-free scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17400000, "bbox": [[0.174, 0.643, 0.827, 0.673]]}]}, "tags": ["figure_caption"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 704, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17500000, "bbox": [[0.303, 0.689, 0.691, 0.75]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 705, "edu_l1_label": "EDU_O"}, {"txt": "Note that the feld of federated generative models have struggled to achieve stable performance in challenging setups, such as non-IID and privacy-preserving cases, even with MNIST-level datasets.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.176, 0.785, 0.824, 0.801], [0.176, 0.799, 0.824, 0.814]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 706, "edu_l1_label": "IOS"}, {"txt": "However, even under these conditions, PRISM is the frst to consistently demonstrate stable performance under these conditions.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.176, 0.813, 0.824, 0.828], [0.176, 0.827, 0.395, 0.842]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 707, "edu_l1_label": "IOS"}, {"txt": "Despite the poor conditions of federated generative models, it is important to address whether PRISM can be extended to large-scale datasets under limited resources.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.395, 0.827, 0.824, 0.842], [0.176, 0.841, 0.824, 0.856]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 708, "edu_l1_label": "IOS"}, {"txt": "We provide quantitative and qualitative results on large-scale datasets in Table 9 and Figure 10, such as CelebA 128x128 and CIFAR100 datasets, under non-IID and privacy-free scenario.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.176, 0.855, 0.824, 0.87], [0.176, 0.868, 0.738, 0.884]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 709, "edu_l1_label": "IOS"}, {"txt": "Both experiments are conducted under exactly same settings as in Section 5.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.738, 0.868, 0.824, 0.884], [0.176, 0.882, 0.601, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 710, "edu_l1_label": "IOS"}, {"txt": "As a result, PRISM demonstrates acceptable performance on high-resolution images; however, it reveals certain limitations when applied to large-scale datasets with diverse classes, such as CIFAR100.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17600000, "bbox": [[0.601, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.623, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 1, "global_sentence_id": 711, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17700000, "bbox": [[0.492, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 712, "edu_l1_label": "EDU_O"}, {"txt": "Table 10: FID comparison where distance metric is cosine similarity for MADA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17800000, "bbox": [[0.235, 0.101, 0.762, 0.118]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 6, "global_sentence_id": 713, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 17900000, "bbox": [[0.241, 0.127, 0.755, 0.235]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 0, "global_sentence_id": 714, "edu_l1_label": "EDU_O"}, {"txt": "L SELECTION OF OTHER DISTANCE METRIC", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18000000, "bbox": [[0.177, 0.264, 0.189, 0.282], [0.209, 0.264, 0.55, 0.281]]}]}, "tags": ["text"], "label": "O", "web_segment_id": 11, "global_sentence_id": 715, "edu_l1_label": "EDU_O"}, {"txt": "Table 10 provide the FID performance of PRISM when using cosine similarity instead of hamming distance for MADA.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.176, 0.297, 0.824, 0.313], [0.176, 0.311, 0.311, 0.326]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 716, "edu_l1_label": "IOS"}, {"txt": "PRISM with cosine similarity achieves competitive performance compared to the scheme using Hamming distance in most cases.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.311, 0.311, 0.824, 0.326], [0.176, 0.325, 0.51, 0.34]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 717, "edu_l1_label": "EDU_O"}, {"txt": "Although the performance degrades for CelebA in the non-IID scenario, it still outperforms all baseline methods, further demonstrating the robustness of the approach.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18100000, "bbox": [[0.51, 0.325, 0.824, 0.34], [0.176, 0.339, 0.824, 0.354], [0.176, 0.353, 0.314, 0.368]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 3, "global_sentence_id": 718, "edu_l1_label": "EDU_O"}, {"txt": "M THE EFFECT OF MASKING MECHANISM", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18200000, "bbox": [[0.177, 0.389, 0.194, 0.407], [0.215, 0.389, 0.54, 0.406]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 4, "global_sentence_id": 719, "edu_l1_label": "IOS"}, {"txt": "Table 11: Effect of mask selection on MNIST.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18300000, "bbox": [[0.35, 0.428, 0.65, 0.441]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 5, "global_sentence_id": 720, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18400000, "bbox": [[0.236, 0.459, 0.757, 0.523]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 721, "edu_l1_label": "EDU_O"}, {"txt": "In Table 11, we compare the effect of the mask extraction algorithm on generative model training. For this, we relied on prior studies on the existence of SLT and the convergence of MMD with a characteristic kernel [46, 14]. Our results empirically show SLT’s convergence under mask aver-aging generative FL setting. We also explore the Random and Top-k% algorithms. The Bernoulli method employed in PRISM outperformed top-k%, while Random and Weight represent the lower and upper bounds of performance achievable by SLT, respectively.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18500000, "bbox": [[0.173, 0.545, 0.827, 0.632]]}]}, "tags": ["table_caption"], "label": "content", "web_segment_id": 9, "global_sentence_id": 722, "edu_l1_label": "IOS"}, {"txt": "NEVALUATION ON THE CENTRALIZED SETTING", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18600000, "bbox": [[0.177, 0.65, 0.592, 0.667]]}]}, "tags": ["title"], "label": "O", "web_segment_id": 7, "global_sentence_id": 723, "edu_l1_label": "EDU_O"}, {"txt": "Table 12: Comparison of PRISM with centralized setting.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18700000, "bbox": [[0.311, 0.69, 0.689, 0.702]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 8, "global_sentence_id": 724, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18800000, "bbox": [[0.239, 0.719, 0.757, 0.833]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 725, "edu_l1_label": "EDU_O"}, {"txt": "In Table 12, we report the results of models trained under FL vs. vanilla (centralized data) setups.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18900000, "bbox": [[0.176, 0.855, 0.824, 0.87]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 726, "edu_l1_label": "IOS"}, {"txt": "PRISM shows performance degradation due to privacy, data heterogeneity, and communication overhead in FL settings.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18900000, "bbox": [[0.176, 0.868, 0.824, 0.884], [0.176, 0.882, 0.308, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 727, "edu_l1_label": "IOS"}, {"txt": "However, centralized training is not applicable in scenarios where distributed samples cannot be shared, which is the primary focus of our work.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18900000, "bbox": [[0.308, 0.882, 0.824, 0.897], [0.176, 0.896, 0.613, 0.911]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 728, "edu_l1_label": "IOS"}, {"txt": "Therefore, our original submission did not include this comparison, consistent with existing FL research.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 18900000, "bbox": [[0.613, 0.896, 0.824, 0.911], [0.176, 0.91, 0.66, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 10, "global_sentence_id": 729, "edu_l1_label": "IOS"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19000000, "bbox": [[0.492, 0.95, 0.507, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 730, "edu_l1_label": "EDU_O"}, {"txt": "OADDITIONAL RESULTS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19100000, "bbox": [[0.177, 0.102, 0.398, 0.119]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 5, "global_sentence_id": 731, "edu_l1_label": "BOS"}, {"txt": "O.1DIFFERENT RANDOM INITIALIZATION OR SEED", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19200000, "bbox": [[0.177, 0.133, 0.542, 0.148]]}]}, "tags": ["text"], "label": "title2", "web_segment_id": 6, "global_sentence_id": 732, "edu_l1_label": "EDU_O"}, {"txt": "Table 13: Quantitative performance of PRISM across different random seeds on the MNIST dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19300000, "bbox": [[0.171, 0.161, 0.82, 0.176]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 7, "global_sentence_id": 733, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19400000, "bbox": [[0.27, 0.192, 0.727, 0.33]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 2, "global_sentence_id": 734, "edu_l1_label": "EDU_O"}, {"txt": "PRISM learns a binary mask over frozen, randomly initialized weights. One might argue that the way model parameters are initialized can signifcantly impact performance. To validate the robustness of PRISM, we present quantitative performance across various random seeds using the MNIST dataset in Table 13. Note that the random seed 30 in the frst row of each table corresponds to the default confgurations described in Section 5 and all of experimental setup is identical to Section 5.1 and Section 5.2.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19500000, "bbox": [[0.175, 0.348, 0.829, 0.432]]}]}, "tags": ["figure_caption"], "label": "content", "web_segment_id": 9, "global_sentence_id": 735, "edu_l1_label": "EDU_O"}, {"txt": "O.2ABLATION STUDY ON THE LEARNING RATE FOR BASELINES", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19600000, "bbox": [[0.176, 0.448, 0.634, 0.462]]}]}, "tags": ["table_caption"], "label": "title2", "web_segment_id": 10, "global_sentence_id": 736, "edu_l1_label": "EDU_O"}, {"txt": "Table 14: Baseline’s FID performance for adjusting learning rate using MNIST dataset.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19700000, "bbox": [[0.212, 0.476, 0.785, 0.491]]}]}, "tags": ["table_caption"], "label": "table_title", "web_segment_id": 3, "global_sentence_id": 737, "edu_l1_label": "EDU_O"}, {"txt": "table", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19800000, "bbox": [[0.207, 0.507, 0.794, 0.593]]}]}, "tags": ["table"], "label": "table", "web_segment_id": 1, "global_sentence_id": 738, "edu_l1_label": "EDU_O"}, {"txt": "In this section, we present the additional experiments to validate the fairness of our entire experiments. In Table 14, we sweep learning rates for GS-WGAN and DP-FedAvgGAN and a FID performance on MNIST dataset with Non-IID and privacy-preserving scenario. Since they utilize GANs, there are separate learning rates for the generator and discriminator, and we examined these combinations. In the tables below, the top row represents the discriminator’s lr / generator’s lr, while thesubsequent row reports the FID scores. Due to the notorious instability of GAN training caused by the coupled learning dynamics between the generator and discriminator, we observed divergence in some settings. For GS-WGAN, the best performance was achieved with 1e-4 / 1e-5, but it still failed to generate MNIST images properly.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 19900000, "bbox": [[0.174, 0.61, 0.828, 0.737]]}]}, "tags": ["table_caption"], "label": "content", "web_segment_id": 8, "global_sentence_id": 739, "edu_l1_label": "IOS"}, {"txt": "PCONTEXTUALIZE THE PRACTICAL SCENARIOS", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20000000, "bbox": [[0.177, 0.754, 0.589, 0.771]]}]}, "tags": ["title"], "label": "title1", "web_segment_id": 4, "global_sentence_id": 740, "edu_l1_label": "BOS"}, {"txt": "Federated learning (FL) is commonly motivated by real-world scenarios such as healthcare, fnance, and mobile devices, where direct data sharing is infeasible due to privacy concerns.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.176, 0.785, 0.824, 0.801], [0.176, 0.799, 0.74, 0.814]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 741, "edu_l1_label": "EDU_O"}, {"txt": "In this context, our work does not aim to generate private data directly but instead focuses on enabling stable and effective generative modeling under these constraints.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.74, 0.799, 0.824, 0.814], [0.176, 0.813, 0.824, 0.828], [0.176, 0.827, 0.549, 0.842]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 742, "edu_l1_label": "EDU_O"}, {"txt": "For instance, in healthcare, PRISM allows multiple institutions to collaboratively train generative models without sharing sensitive patient data.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.549, 0.827, 0.824, 0.842], [0.176, 0.841, 0.824, 0.856]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 743, "edu_l1_label": "EDU_O"}, {"txt": "The resulting models can facilitate privacy-preserving synthetic data generation, imputation of missing medical records, data augmentation for low-data scenarios, and solving inverse problems such as denoising and reconstruction.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.176, 0.855, 0.824, 0.87], [0.176, 0.868, 0.824, 0.884], [0.176, 0.882, 0.389, 0.897]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 744, "edu_l1_label": "EDU_O"}, {"txt": "Similarly, in mobile devices, PRISM enables decentralized training of generative models while preserving user privacy, which can enhance personalized content generation and user-specifc data augmentation.", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20100000, "bbox": [[0.389, 0.882, 0.824, 0.897], [0.176, 0.896, 0.824, 0.911], [0.176, 0.91, 0.487, 0.925]]}]}, "tags": ["text"], "label": "content", "web_segment_id": 0, "global_sentence_id": 745, "edu_l1_label": "EDU_O"}, {"txt": "footer", "language": "english", "position": {"pdf_position": [{"page_number": 0, "shape": [1650, 1275], "position_id": 20200000, "bbox": [[0.492, 0.95, 0.508, 0.959]]}]}, "tags": ["footer"], "label": "O", "web_segment_id": -1, "global_sentence_id": 746, "edu_l1_label": "EDU_O"}], "type": "PDF"}
# PRISM:PRIVACY-PRESERVING IMPROVED STOCHASTICMASKINGFORFEDERATED GENERATIVE MODELS
## ABSTRACT
## 1INTRODUCTION
## 2RELATED WORK
### Federated learning for classifcation models.
### Federated learning for generative models with privacy consideration.
## 3BACKGROUND
### 3.1STRONG LOTTERY TICKETS
### 3.2DIFFERENTIAL PRIVACY
## 4METHOD
### 4.1PRISM : PRIVACY-PRESERVING IMPROVED STOCHASTIC MASKING
#### Local score updates with MMD loss.
#### Binary mask generation and aggregation.
#### Model initialization for storage effciency.
### 4.2PRIVACY
### 4.3MASK-AWARE DYNAMIC MOVING AVERAGE AGGREGATION
### 4.4TRADING-OFF BETWEEN PERFORMANCE AND COMMUNICATION COST
## 5EXPERIMENTS
### 5.1IID CASE
### 5.2NON-IID CASE
### 5.3 PERFORMANCE WITHOUT DIFFERENTIAL PRIVACY
### 5.4  EFFECT OF DYNAMIC MOVING AVERAGE AGGREGATION
### 5.5RESOURCE EFFICIENCY AT INFERENCE TIME
## 6 CONCLUSION
## AAPPENDIX
### APRIVACY
### BPRIVACY AMPLIFIED SCENARIO
### CTRAINING DETAILS
### DDATASET PARTITIONING STRATEGIES ACROSS CLIENTS
#### IID setup.
#### Non-IID setup (Shards-partitioned).
#### Non-IID setup (Dirichlet-partitioned).
### EEFFICIENT STORAGE AND QUANTIZATION STRATEGY IN PRISM
### FANALYSIS OF HYBRID AGGREGATION
### G COMMUNICATION COST DURING DOWNLINK PROCESS
### HCROSS-DEVICE ENVIRONMENTS
### IANALYSIS OF CONVERGENCE OF λ IN MADA
### JCOMPUTATION COST ANALYSIS
### KSCALABILITY OF PRISM
#### K.1EXTEND TO DIFFUSION MODEL
#### K.2EVALUATION ON LARGE-SCALE DATASETS
### L SELECTION OF OTHER DISTANCE METRIC
### M THE EFFECT OF MASKING MECHANISM
### NEVALUATION ON THE CENTRALIZED SETTING
### OADDITIONAL RESULTS
#### O.1DIFFERENT RANDOM INITIALIZATION OR SEED
#### O.2ABLATION STUDY ON THE LEARNING RATE FOR BASELINES
### PCONTEXTUALIZE THE PRACTICAL SCENARIOS
