A field test of a federated learning/federated analytic blockchain network implementation in an HPC environment - Short, James E; Miyachi, Ken; Toouli, Christian; Todd, Steve (Frontiers in Blockchain, 2022)
The rapid upswing in interest in federated learning (FL) and federated analytics (FA) architectures has corresponded with the rapid increase in commercial AI software products, ranging from face detection and language translation to connected IOT devices, smartphones and autonomous vehicles equipped with high-resolution sensors. However, the typical mobile-edge computing paradigm assumes all data resources are transferred from the IoT client to a computational platform through a cellular network. The problem is this data architecture is not appropriate for all human centered activity. As it does not readily address questions of data ownership, privacy and data location in the context of the multiple datasets required for machine learning. In this paper, we report on a pilot distributed ledger and smart contracts network model, designed to track analytic jobs in an HPC supercomputing environment. The technical design and test system integrates the FL/FA model into a blockchain-based network architecture, wherein the test system records interactions with the global server and blockchain network. The design goal is to create a secure audit trail of supercomputer analytic operations, and the ability to securely federate those operations across multiple supercomputer deployments. As there are still relatively few real-world applications of FL/FA models and blockchain networks, our test system design and deployment is intended to help spark further research into the integration of both technologies.