DeepSec 2024 Talk: From Dungeon Crawling to Cyber Defense Drill: Using RPG Principles and LLM for Operational Team Dev – Aurélien Denis & Charles Garang
Continuous improvement/training is in the DNA of cybersecurity professionals, specifically for incident responders, which are always searching for new ways to learn and practice their technical and analytical crafts. This is even more the case in mature environments where Incident response teams may find themselves in a situation with few high stakes incidents, preventing them from applying their technical and thinking skills, thus lowering their readiness when a crisis occur.
LLMs based conversational agents are becoming mainstream, and applications are countless.
In the meantime, Tabletop Role-Playing Games (TTRPG) are found to be a great breeding ground for creativity and fun. To achieve the benefits of this game, preparation is needed and a game master must be present to keep the players engaged.
So we leveraged the power of AI, mixed automation and past experiences or lessons learned with the fun of TTRPG to provide a new tool for incident responders to practice live sessions… In this talk, we will present our new Mattermost-enabled game that allows players to be confronted with dire situations.
We asked Aurélien and Charles a few more questions about their talk.
Please tell us the top 5 facts about your talk.
- A good way to write good LLM prompts is to have it written by an LLM.
- There’s no limit to what a lazy dev can do.
- Coding is hard, but no-coding is too.
- Using RPG DM principles helped us create a generative system without training a model but we did have to replace dice-rolling with LLM prompting.
- Using a centralized messaging platform is a good vector to play games and you don’t need to break the bank to start prototyping.
How did you come up with it? Was there something like an initial spark that set your mind on creating this talk?
We’re cybersecurity incident handlers, and we noticed we do not often have “jeopardy” incident engagements. So we wanted to come up with a system to keep our mind sharp and to find a way to keep training without building labs and maintaining them.
We also see ourselves as “nerds” and we do love a good tabletop RPG. This kind of game is fun, engaging (as we’re directly influencing the story) and is a great way to learn about yourself. Also, when Aurelien attended DeepSec last year he tried out “HackBack” which was a kind of Incident Response RPG but found it a bit too rules-heavy.
So we thought it would be interesting to come up with a system that takes root in the narrative storytelling from those games to drill cybersecurity incidents.
Why do you think this is an important topic?
We believe there are other cybersecurity teams which are in the same situation as us and could benefit from practicing these kinds of drills. We wanted to build a solution that could be easily deployed and leveraged by other teams. LLMs are one way to stimulate creativity and to off-load time-consuming tasks (those who have ran tabletop RPG definitely see what we mean: world building, improvising on the spot, keeping the game challenging and engaging, etc.).
We know that Incident Response is about asking the right questions and then finding the data to get suitable answers. So we think that creating those kind of “ready-to-play” systems is a good way to make players focus on asking the right questions (finding the data is something that they should be able to do, given enough time).
Is there something you want everybody to know – some good advice for our readers, maybe?
This system is very new, and there are a lot of evolutions that we would love to implement. It’s far from perfect, but we hope it will spark some interest and creativity and allow people to experiment with LLM in a fun way.
Also, we wanted to show that no-code helped us prototype quickly a MVP and iterate on it. While it’s not “great” code, it’s “good” enough for our needs and we can improve it later.
In our talk, we’ll talk about the system, but also about how we iterated to get to where we are today.
A prediction for the future – what do you think will be the next innovations or future downfalls when it comes to your field of expertise / the topic of your talk in particular?
Big LLM models are expensive (in terms of resources if self-hosted, or money if cloud-hosted).
New models are being published (and some have been during the development of our project) which can open up new use cases or make them less error-prone regarding their prompts.
LLM do have some use cases where it can help an investigator sift through data more quickly (by using summarization chains for example) and we think it could also help as a chatbot for rubber-duck investigating.
Aurelien is a cybersecurity professional who has been active in the industry for over five years. He began his journey as an intern Incident Handler at CERT-W, where he gained valuable hands-on experience in incident response and digital forensics. Aurelien then went on to join CERT-XMCO as a full-time consultant. Over two and a half years, he honed his skills and used them to maintain a cybersecurity watch for his clients, helping them to manage their external attack surface and also engaging in complex security incidents. Aurelien is now working as a Cybersecurity analyst at CERT Societe Generale.
With over seven years of experience in cybersecurity, Charles is a seasoned analyst currently helping to protect one of France’s leading financial institutions within CERT Societe Generale. He brings a wealth of knowledge from his diverse background, having served as a SOC analyst at a French Managed Security Service Provider (MSSP) and a Threat Intelligence Analyst at a French Ministry.
Charles and Aurelien are active members of cybersecurity communities, such as InterCERT France or FIRST where they both held talks regarding to automation, threat intelligence and artificial intelligence. Aurelien also shared his expertise through publications in XMCO’s ActuSecu, a leading French cybersecurity newspaper.