DeepSec Scuttlebutt: Fun with Fuzzing, LLMs, and Backdoors
[This is the blog version of our monthly DeepSec Scuttlebutt musings. You can subscribe to the DeepSec Scuttlebug mailing list, if you want to read the content directly in your email client.]
Dear readers,
the Summer temperatures are rising. The year 2023 features the highest measured temperatures in measurement history. This is no surprise. The models predicting what we see and feel now have been created in the 1970s by Exxon. So far, the model has been quite accurate. What has this to do with information security? Well, infosec also uses models for attack and defence, too. The principles of information security has stayed the same, despite the various trends. These are the building blocks of our security models. They can be adapted, but the overall principles have little changed from two-hosts-networks to the cloud platforms. Enter security fashion, because sometimes the trends are a threat to security models. Adding security early to projects has gained a wider acceptance, but adding defences is prone to wild promises by various vendors. We have seen router access lists evolving to stateful packet filtering, reputation-based address databases, attack signatures, and overall next-generation pattern matching. If every incarnation of a basic security principle was really different, we would see fewer attacks. So has the technology gotten worse? No, the attackers just have learned to adapt.
Speaking of adapting, I have some more updates on the evolution of large language models. LLMs have been turned into a black box, mainly because of their size, the training data, and fine-tuning of algorithms. The first consequences already show. The Federal Trade Commission (FTC) is looking into OpenAI for data leaks and ChatGPT’s inaccurate responses. They have compiled a set of questions showing a lack of security. The inquiry is a reminder that LLMs are still prototypes. Contrary to rumours, the language models are not sentient. “Dumb Meaning: Machine Learning and Artificial Semantics” is an article that details the algorithm design. The author, Hannes Bajohr, argues that models such as ChatGPT and others are dumb and have no consciousness. They just can handle complex artificial semantic. A Google engineer claimed that the LaMDA model had gained a level of intelligence and should be regarded as a person. If this were true, what about the more complex models capable of creating artificial images? The Google engineer’s claim is highly biased. Can a firewall or spam filter be intelligent even if it doesn’t write coherent texts? Most people would not describe modern security filters as sentient. Snort/Suricata or YARA rules are just not the language most people understand.
Apart from the basic principles of information security, there is something else that has changed little over the years: the urge to put backdoors into otherwise secure systems. DeepSec has been an avid supporter of not using backdoors or weakening cryptographic algorithms. If you wonder how this relates to AI, then I recommend reading the interview of Signal’s Meredith Whittaker. After that, read the publication “Universal and Transferable Adversarial Attacks on Aligned Language Models” (link is at the bottom). Who would have thought that fuzzing can be used to attack LLMs? 😀
If you have some thoughts on what current technology is lacking or how attackers adapt, then today is the last chance to tell us. The call for papers of DeepSec and DeepINTEL officially close today at midnight (CEST).
Best regards,
René.
P.S.: We have left Twitter/X a long time ago. Please have a look at our Mastodon account https://social.tchncs.de/@deepsec for updates.
Sources:
- Climate Change predicted by Exxon 50 years ago
- Article „Dumb Meaning: Machine Learning and Artificial Semantics“
- FTC vs. OpenAI (Washington Post)
- Wall Street Journal about the dark side of LLM training
- Signal’s Meredith Whittaker: ‘These are the people who could actually pause AI if they wanted to’ (The Guardian)