DeepSec 2024 Talk: A Practical Approach to Generative AI Security – Florian Grunow & Hannes Mohr
The rise of applications based on AI (mostly generative AI) forces us to think about the security and privacy implications of these systems. We will try to make sense about the attack surface of generative AI applications, what practitioners in the field need to consider in development and operations, and how they can derive security measures for these systems.
We will first dive into the range of generative AI applications using examples of the OpenAI ecosystem. This will give the audience an understanding about the fundamental problem of AI from a security perspective. We then offer an insight into the attack surface that those applications have. This will help understand what needs to be secured and what can be secured. Many times, good old security best practices will be a good start, although AI security brings new challenges that we will discuss. In addition, we will talk briefly about privacy issues related to AI that we need to consider in the future.
All the aspects mentioned above will be supported by examples we have prepared. The aim is for the audience to gain an understanding of the issues associated with generative AI applications, and for security practitioners to derive security measures that can be applied in this area.
We asked Florian and Hannes a few more questions about their talk.
Please tell us the top 5 facts about your talk.
- AI poses new threats that we need to solve.
- We can solve those issues right now by applying certain best practices and we need to understand that others will not work any more.
- AI needs to be considered insecure by design.
- The non-determinism that comes with AI is a security nightmare.
- Remember, your attackers are basically talking to a 6-year-old with excellent conversational skills that oversees your data but will overlook your intentions.
How did you come up with it? Was there something like an initial spark that set your mind on creating this talk?
AI is the new thing, and customers are implementing it in the weirdest places. Understanding this technology is key to protecting the systems out there.
Why do you think this is an important topic?
It’s AI, and everybody wants to have one.
Is there something you want everybody to know – some good advice for our readers, maybe?
Remember, attackers targeting AI are basically talking to a 6-year-old with excellent conversational skills that oversees your data but will overlook the intentions and will happily hand out any information for a lollipop.
A prediction for the future – what do you think will be the next innovations or future downfalls for your field of expertise / the topic of your talk in particular?
We will see people crossing the red lines by not respecting the inherent dangers of AI for profit. This will lead to severe incidents and hopefully a learning curve. AI security and how to do it right will be a dedicated element for holistic IT security in the future.
Florian holds a Bachelor’s degree in Medical Computer Sciences and a Master’s degree in Software Engineering. He works as a Security Analyst and team lead at ERNW.
Hannes holds a Bachelor’s and a Master’s degree in Physics. He works as a Security Analyst and Team Lead at ERNW and takes care of application and infrastructure assessments. His topics include mobile, IoT and telco security.