Evaluating multi-agent reinforcement learning on heterogeneous platforms

Samuel Wiggins; Yuan Meng; Rajgopal Kannan; Viktor Prasanna

doi:10.1117/12.2668689

12 June 2023 Evaluating multi-agent reinforcement learning on heterogeneous platforms

Samuel Wiggins, Yuan Meng, Rajgopal Kannan, Viktor Prasanna

Proceedings Volume 12538, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V; 125381S (2023) https://doi.org/10.1117/12.2668689
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Conference Poster

Abstract

Multi-Agent Reinforcement Learning (MARL) is a key technology in Artificial Intelligence Systems. The development of MARL is a “Training-in-Simulation” process and involves time-consuming computation and communication. Existing implementations of MARL using homogeneous architecture fail to satisfy the above requirements simultaneously. In this work, using a state-of-the-art MARL algorithm - Multi-Agent Deep Deterministic Policy Gradient (MADDPG) - as a benchmark, we provide a systematic analysis of the performance bottleneck on multi-core CPU platforms. We show that (a). the training throughput of large neural network models can benefit from data parallel architecture such as GPU; (b) Memory-intensive components including replay management and all-to-all communication leads to large memory overheads on multi-level cache memory hierarchy on CPU and GPU, thus should be accelerated using custom hardware coupled with large on-chip SRAM with specialized data layout optimizations on FPGA.

Citation Download Citation

Samuel Wiggins, Yuan Meng, Rajgopal Kannan, and Viktor Prasanna "Evaluating multi-agent reinforcement learning on heterogeneous platforms", Proc. SPIE 12538, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V, 125381S (12 June 2023); https://doi.org/10.1117/12.2668689

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available