This paper details the lessons learned from designing a specialized competition focused on network attack simulation, specifically emphasizing lateral movement in networks. Our experience in planning the competition reveals several critical design principles that could guide the development of future competitions. Among these principles are the need for a carefully curated simulation environment that mimics real-world network behaviors and vulnerabilities without neglecting the limitations of current capabilities, the necessity for defining a clear scoring system that accurately reflects the effectiveness of different strategies while allowing for interpretability and style, and the importance of ensuring that the competition has clear roles for participants with varied levels of expertise in both cybersecurity and reinforcement learning. This paper offers lessons learned for those interested in organizing similar competitions or developing benchmarks in the area of cyber reinforcement learning. We strongly advocate for the use of network attack simulation competitions as an effective means to accelerate research, foster community collaboration, and identify emergent strategies and techniques. By outlining our design process, the challenges faced, and the solutions implemented, we provide actionable insights to assist practitioners, researchers, and government sponsors.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.