Distributed, Parallel, and Cluster Computing

• New submissions
• Cross-lists
• Replacements

See recent articles

Showing new listings for Friday, 18 April 2025

New submissions (showing 2 of 2 entries)

[1] arXiv:2504.12766 [pdf, html, other]

Title: Falcon: Advancing Asynchronous BFT Consensus for Lower Latency and Enhanced Throughput

Xiaohai Dai, Chaozheng Ding, Wei Li, Jiang Xiao, Bolin Zhang, Chen Yu, Albert Y. Zomaya, Hai Jin

Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)

Asynchronous Byzantine Fault Tolerant (BFT) consensus protocols have garnered significant attention with the rise of blockchain technology. A typical asynchronous protocol is designed by executing sequential instances of the Asynchronous Common Sub-seQuence (ACSQ). The ACSQ protocol consists of two primary components: the Asynchronous Common Subset (ACS) protocol and a block sorting mechanism, with the ACS protocol comprising two stages: broadcast and agreement. However, current protocols encounter three critical issues: high latency arising from the execution of the agreement stage, latency instability due to the integral-sorting mechanism, and reduced throughput caused by block discarding. To address these issues,we propose Falcon, an asynchronous BFT protocol that achieves low latency and enhanced throughput. Falcon introduces a novel broadcast protocol, Graded Broadcast (GBC), which enables a block to be included in the ACS set directly, bypassing the agreement stage and thereby reducing latency. To ensure safety, Falcon incorporates a new binary agreement protocol called Asymmetrical Asynchronous Binary Agreement (AABA), designed to complement GBC. Additionally, Falcon employs a partial-sorting mechanism, allowing continuous rather than simultaneous block committing, enhancing latency stability. Finally, we incorporate an agreement trigger that, before its activation, enables nodes to wait for more blocks to be delivered and committed, thereby boosting throughput. We conduct a series of experiments to evaluate Falcon, demonstrating its superior performance.

[2] arXiv:2504.13141 [pdf, html, other]

Title: Complexity at Scale: A Quantitative Analysis of an Alibaba Microservice Deployment

Giles Winchester, George Parisis, Luc Berthouze

Comments: 19 pages, 24 figures, 3 tables

Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Software Engineering (cs.SE)

Microservice architectures are increasingly prevalent in organisations providing online applications. Recent studies have begun to explore the characteristics of real-world large-scale microservice deployments; however, their operational complexities, and the degree to which this complexities are consistent across different deployments, remains under-explored. In this paper, we analyse a microservice dataset released by Alibaba along three dimensions of complexity: scale, heterogeneity, and dynamicity. We find that large-scale deployments can consist of tens of thousands of microservices, that support an even broader array of front-end functionality. Moreover, our analysis shows wide-spread long-tailed distributions of characteristics between microservices, such as share of workload and dependencies, highlighting inequality across the deployment. This diversity is also reflected in call graphs, where we find that whilst front-end services produce dominant call graphs, rarer non-dominant call graphs are prevalent and could involve dissimilar microservice calls. We also find that runtime dependencies between microservices deviate from the static view of system dependencies, and that the deployment undergoes daily changes to microservices. We discuss the implications of our findings for state-of-the-art research in microservice management and research testbed realism, and compare our results to previous descriptions of large-scale microservice deployments to begin to build an understanding of their commonalities.

Cross submissions (showing 4 of 4 entries)

[3] arXiv:2504.12471 (cross-list from cs.LG) [pdf, html, other]

Title: You Don't Need All Attentions: Distributed Dynamic Fine-Tuning for Foundation Models

Shiwei Ding, Lan Zhang, Zhenlin Wang, Giuseppe Ateniese, Xiaoyong Yuan

Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF)

Fine-tuning plays a crucial role in adapting models to downstream tasks with minimal training efforts. However, the rapidly increasing size of foundation models poses a daunting challenge for accommodating foundation model fine-tuning in most commercial devices, which often have limited memory bandwidth. Techniques like model sharding and tensor parallelism address this issue by distributing computation across multiple devices to meet memory requirements. Nevertheless, these methods do not fully leverage their foundation nature in facilitating the fine-tuning process, resulting in high computational costs and imbalanced workloads. We introduce a novel Distributed Dynamic Fine-Tuning (D2FT) framework that strategically orchestrates operations across attention modules based on our observation that not all attention modules are necessary for forward and backward propagation in fine-tuning foundation models. Through three innovative selection strategies, D2FT significantly reduces the computational workload required for fine-tuning foundation models. Furthermore, D2FT addresses workload imbalances in distributed computing environments by optimizing these selection strategies via multiple knapsack optimization. Our experimental results demonstrate that the proposed D2FT framework reduces the training computational costs by 40% and training communication costs by 50% with only 1% to 2% accuracy drops on the CIFAR-10, CIFAR-100, and Stanford Cars datasets. Moreover, the results show that D2FT can be effectively extended to recent LoRA, a state-of-the-art parameter-efficient fine-tuning technique. By reducing 40% computational cost or 50% communication cost, D2FT LoRA top-1 accuracy only drops 4% to 6% on Stanford Cars dataset.

[4] arXiv:2504.12733 (cross-list from cs.CR) [pdf, other]

Title: Adversary-Augmented Simulation for Fairness Evaluation and Defense in Hyperledger Fabric

Erwan Mahe, Rouwaida Abdallah, Pierre-Yves Piriou, Sara Tucci-Piergiovanni

Comments: 20 pages, 14 figures. arXiv admin note: text overlap with arXiv:2403.14342

Subjects: Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC); Multiagent Systems (cs.MA)

This paper presents an adversary model and a simulation framework specifically tailored for analyzing attacks on distributed systems composed of multiple distributed protocols, with a focus on assessing the security of blockchain networks. Our model classifies and constrains adversarial actions based on the assumptions of the target protocols, defined by failure models, communication models, and the fault tolerance thresholds of Byzantine Fault Tolerant (BFT) protocols. The goal is to study not only the intended effects of adversarial strategies but also their unintended side effects on critical system properties. We apply this framework to analyze fairness properties in a Hyperledger Fabric (HF) blockchain network. Our focus is on novel fairness attacks that involve coordinated adversarial actions across various HF services. Simulations show that even a constrained adversary can violate fairness with respect to specific clients (client fairness) and impact related guarantees (order fairness), which relate the reception order of transactions to their final order in the blockchain. This paper significantly extends our previous work by introducing and evaluating a mitigation mechanism specifically designed to counter transaction reordering attacks. We implement and integrate this defense into our simulation environment, demonstrating its effectiveness under diverse conditions.

[5] arXiv:2504.12742 (cross-list from cs.LG) [pdf, html, other]

Title: Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum

Yuan Zhou, Xinli Shi, Xuelong Li, Jiachen Zhong, Guanghui Wen, Jinde Cao

Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Optimization and Control (math.OC)

Decentralized Federated Learning (DFL) eliminates the reliance on the server-client architecture inherent in traditional federated learning, attracting significant research interest in recent years. Simultaneously, the objective functions in machine learning tasks are often nonconvex and frequently incorporate additional, potentially nonsmooth regularization terms to satisfy practical requirements, thereby forming nonconvex composite optimization problems. Employing DFL methods to solve such general optimization problems leads to the formulation of Decentralized Nonconvex Composite Federated Learning (DNCFL), a topic that remains largely underexplored. In this paper, we propose a novel DNCFL algorithm, termed \bf{DEPOSITUM}. Built upon proximal stochastic gradient tracking, DEPOSITUM mitigates the impact of data heterogeneity by enabling clients to approximate the global gradient. The introduction of momentums in the proximal gradient descent step, replacing tracking variables, reduces the variance introduced by stochastic gradients. Additionally, DEPOSITUM supports local updates of client variables, significantly reducing communication costs. Theoretical analysis demonstrates that DEPOSITUM achieves an expected $\epsilon$-stationary point with an iteration complexity of $\mathcal{O}(1/\epsilon^2)$. The proximal gradient, consensus errors, and gradient estimation errors decrease at a sublinear rate of $\mathcal{O}(1/T)$. With appropriate parameter selection, the algorithm achieves network-independent linear speedup without requiring mega-batch sampling. Finally, we apply DEPOSITUM to the training of neural networks on real-world datasets, systematically examining the influence of various hyperparameters on its performance. Comparisons with other federated composite optimization algorithms validate the effectiveness of the proposed method.

[6] arXiv:2504.13003 (cross-list from cs.DS) [pdf, html, other]

Title: Towards Optimal Distributed Edge Coloring with Small Palettes

Manuel Jakob, Yannic Maus, Florian Schager

Subjects: Data Structures and Algorithms (cs.DS); Distributed, Parallel, and Cluster Computing (cs.DC)

We design a deterministic distributed $\mathcal{O}(\log n)$-round reduction from the $(2\Delta-2)$-edge coloring problem to the much easier $(2\Delta-1)$-edge coloring problem. This is almost optimal, as the $(2\Delta-2)$-edge coloring problem admits an $\Omega(\log_\Delta n)$ lower bound. Further, we also obtain an optimal $\mathcal{O}(\log_\Delta n)$-round reduction, albeit to the harder maximal independent set (MIS) problem.
The current state-of-the-art for $(2\Delta - 1)$-edge coloring actually comes from an MIS algorithm by [Ghaffari \& Grunau, FOCS'24], which runs in $\widetilde{\mathcal{O}}(\log^{5/3} n)$ rounds. With our new reduction, this round complexity now carries over to the $(2\Delta - 2)$-edge coloring problem as well. Alternatively, one can also plug in the $(\mathrm{poly} \log \Delta + \mathcal{O}(\log^{\ast} n))$-round $(2\Delta - 1)$-edge coloring algorithm from [Balliu, Brandt, Kuhn \& Olivetti, PODC'22], which yields an optimal runtime of $\mathcal{O}(\log n)$ rounds for $\Delta \leq \mathrm{poly} \log n$. Previously, the fastest deterministic algorithm using less than $2\Delta - 1$ colors for general graphs by [Brandt, Maus, Narayanan, Schager \& Uitto, SODA'25] ran in $\widetilde{\mathcal{O}}(\log^3 n)$ rounds. In addition, we also obtain a $\mathcal{O}(\log \log n)$-round randomized reduction of $(2\Delta - 2)$-edge coloring to $(2\Delta - 1)$-edge coloring. This improves upon the (very recent) best randomized algorithm using less than $2\Delta - 1$ colors from [Bourreau, Brandt \& Nolin, STOC'25] by reducing the round complexity from $\widetilde{\mathcal{O}}(\log^{8/3}\log n)$ down to $\widetilde{\mathcal{O}}(\log^{5/3} \log n)$.

Replacement submissions (showing 4 of 4 entries)

[7] arXiv:2412.08534 (replaced) [pdf, html, other]

Title: Protecting Confidentiality, Privacy and Integrity in Collaborative Learning

Dong Chen, Alice Dethise, Istemi Ekin Akkus, Ivica Rimac, Klaus Satzke, Antti Koskela, Marco Canini, Wei Wang, Ruichuan Chen

Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Cryptography and Security (cs.CR); Machine Learning (cs.LG)

A collaboration between dataset owners and model owners is needed to facilitate effective machine learning (ML) training. During this collaboration, however, dataset owners and model owners want to protect the confidentiality of their respective assets (i.e., datasets, models and training code), with the dataset owners also caring about the privacy of individual users whose data is in their datasets. Existing solutions either provide limited confidentiality for models and training code, or suffer from privacy issues due to collusion.
We present Citadel++, a collaborative ML training system designed to simultaneously protect the confidentiality of datasets, models and training code as well as the privacy of individual users. Citadel++ enhances differential privacy mechanisms to safeguard the privacy of individual user data while maintaining model utility. By employing Virtual Machine-level Trusted Execution Environments (TEEs) as well as the improved sandboxing and integrity mechanisms through OS-level techniques, Citadel++ effectively preserves the confidentiality of datasets, models and training code, and enforces our privacy mechanisms even when the models and training code have been maliciously designed. Our experiments show that Citadel++ provides model utility and performance while adhering to the confidentiality and privacy requirements of dataset owners and model owners, outperforming the state-of-the-art privacy-preserving training systems by up to 543x on CPU and 113x on GPU TEEs.

[8] arXiv:2501.18447 (replaced) [pdf, html, other]

Title: Semaphores Augmented with a Waiting Array

Dave Dice, Alex Kogan

Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)

Semaphores are a widely used and foundational synchronization and coordination construct used for shared memory multithreaded programming. They are a keystone concept, in the sense that most other synchronization constructs can be implemented in terms of semaphores, although the converse does not generally hold. Semaphores and the quality of their implementation are of consequence as they remain heavily used in the Linux kernel and are also available for application programming via the pthreads programming interface.
We first show that semaphores can be implemented by borrowing ideas from the classic ticket lock algorithm. The resulting "ticket-semaphore" algorithm is simple and compact (space efficient) but does not scale well because of the detrimental impact of global spinning. We then transform "ticket-semaphore" into the "TWA-semaphore" by the applying techniques derived from the "TWA - Ticket Locks Augmented with a Waiting Array" algorithm, yielding a scalable semaphore that remains compact and has extremely low latency.

[9] arXiv:2407.11061 (replaced) [pdf, html, other]

Title: Exploring the Boundaries of On-Device Inference: When Tiny Falls Short, Go Hierarchical

Adarsh Prasad Behera, Paulius Daubaris, Iñaki Bravo, José Gallego, Roberto Morabito, Joerg Widmer, Jaya Prakash Varma Champati

Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)

On-device inference holds great potential for increased energy efficiency, responsiveness, and privacy in edge ML systems. However, due to less capable ML models that can be embedded in resource-limited devices, use cases are limited to simple inference tasks such as visual keyword spotting, gesture recognition, and predictive analytics. In this context, the Hierarchical Inference (HI) system has emerged as a promising solution that augments the capabilities of the local ML by offloading selected samples to an edge server or cloud for remote ML inference. Existing works demonstrate through simulation that HI improves accuracy. However, they do not account for the latency and energy consumption on the device, nor do they consider three key heterogeneous dimensions that characterize ML systems: hardware, network connectivity, and models. In contrast, this paper systematically compares the performance of HI with on-device inference based on measurements of accuracy, latency, and energy for running embedded ML models on five devices with different capabilities and three image classification datasets. For a given accuracy requirement, the HI systems we designed achieved up to 73% lower latency and up to 77% lower device energy consumption than an on-device inference system. The key to building an efficient HI system is the availability of small-size, reasonably accurate on-device models whose outputs can be effectively differentiated for samples that require remote inference. Despite the performance gains, HI requires on-device inference for all samples, which adds a fixed overhead to its latency and energy consumption. Therefore, we design a hybrid system, Early Exit with HI (EE-HI), and demonstrate that compared to HI, EE-HI reduces the latency by up to 59.7% and lowers the device's energy consumption by up to 60.4%.

[10] arXiv:2503.00170 (replaced) [pdf, other]

Title: Elastic Restaking Networks

Roi Bar-Zur, Ittay Eyal

Subjects: Computer Science and Game Theory (cs.GT); Distributed, Parallel, and Cluster Computing (cs.DC)

Many blockchain-based decentralized services require their validators (operators) to deposit stake (collateral), which is forfeited (slashed) if they misbehave. Restaking networks let validators secure multiple services by reusing stake. These networks have quickly gained traction, leveraging over~\$20 billion in stake. However, restaking introduces a new attack vector where validators can coordinate to misbehave across multiple services simultaneously, extracting digital assets while forfeiting their stake only once.
Previous work focused either on preventing coordinated misbehavior or on protecting services if all other services are Byzantine and might unjustly cause slashing due to bugs or malice. The first model overlooks how a single Byzantine service can collapse the network, while the second ignores shared-stake benefits.
To bridge the gap, we analyze the system as a strategic game of coordinated misbehavior, when a given fraction of the services are Byzantine. We introduce elastic restaking networks, where validators can allocate portions of their stake that may cumulatively exceed their total stake, and when allocations are lost, the remaining stake stretches to cover remaining allocations. We show that elastic networks exhibit superior robustness compared to previous approaches, and demonstrate a synergistic effect where an elastic restaking network enhances its blockchain's security, contrary to community concerns of an opposite effect in existing networks. We then design incentives for tuning validators' allocations.
Our elastic restaking system and incentive design have immediate practical implications for deployed restaking networks.