DeepSec 2022 Talk: Towards the Automation of Highly Targeted Phishing Attacks with Adversarial Artificial Intelligence – Francesco Morano and Enrico Frumento
The work we will present aims to develop a Proof of Concept (PoC) of an attack scenario that uses Artificial Intelligence (i.e., AI) to create a semi-automatic phishing attack. The AI-based PoC used different network types to automatically compose highly targeted phishing emails with information derived from the initial OSINT analysis of the potential victims. The study approaches the problem from a cybercriminal point of view to understand the feasibility of such an attack tactic and prepare for possible defences. Phishing is a popular way to perform social engineering attacks. According to the Verizon 2022 Data Breach Investigations Report, 82% of data breaches involve human elements and belong to several categories, including phishing, the most common. Using AI tools, this study implements a complete attack chain: (i) initial collection of victims’ data through OSINT, (ii) generation of phishing email body using a GPT-2 and (iii) creation of the graphic mimicking the real organisation brand identity (i.e., logo and stylistic features) through other models. The paper presents the steps needed to prepare an effective phishing strategy and discusses whether and how AI can automate it. This study helps penetration testers and red teams build targeted phishing simulations more rapidly. We discussed the result in terms of the simulated attack’s efficiency.
The aim is to provide red and purple teams with a methodological approach to social engineering attacks by continuing the work started by one of the authors in a previous study. The study aim is to explore the AI’s potentialities in a full OpSec attack stack: wearing the attackers’ hat and performing a full attack. A semi-automatic attack vector created the phishing email.
We asked Enrico and Francesco a few more questions about their talk.
Please tell us the top 5 facts about your talk
- We build on top of a solid line of research developed in several years.
- In 2014 I presented at DeepSec the paper “An innovative and comprehensive framework for Social Driven Vulnerability Assessment” where we introduced how to perform a simulated phishing campaign, years before ProofPoint, Knowbe4 etc. Since then, we have tested 150.000 persons in approximately 40 enterprises.
- In 2017 we published the paper “Victim Communication Stack (VCS): A flexible model to select the Human Attack Vector” (DOI: 10.1145/3098954. 3103156), which first introduced a standardised way to create compelling social engineering attacks (not only phishing), decomposing the human attack model. This is the result of the European Project DOGANA (www.dogana-project.eu) of which I was the scientific coordinator. The difference of this model was to concretely give a methodological approach to create social engineering attacks for red teams and not to explain from a psychological point of view why and how humans fall into error. This research starting from those results, explored the automation with an AI.
- We introduced the concept of Social Engineering 2.0 in Davide Ariu, Enrico Frumento, and Giorgio Fumera. “Social Engineering 2.0: A Foundational Work: Invited Paper” (DOI: 10.1145/3075564.3076260)
- This paper aims to develop a Proof of Concept (PoC) of an attack scenario that uses Artificial Intelligence (i.e., AI) to create a semi-automatic phishing attack. The AI-based PoC used different network types to automatically compose highly targeted phishing emails with information derived from the initial OSINT analysis of the potential victims.
- The study approaches the problem from a cybercriminal point of view to understand the feasibility of such an attack tactic and prepare for possible defences.
- Building on the Victim Communication Stack model, we aim to create a methodological approach for red teams to craft social engineering attacks.
- At Blackhat 2021, a similar paper (here) used GPT-3 with slightly better results. However, that paper did not explore the automation of the entire attack scenario and was not actively exploring attackers’ potentialities. They also executed trials with individuals, which we still plan to do. However, we have a solid base of social engineering tests with companies we will exploit.
How did you come up with it? Was there something like an initial spark that set your mind on creating this talk?
In 2017 we published the paper “Victim Communication Stack (VCS): A flexible model to select the Human Attack Vector” (DOI: 10.1145/3098954. 3103156), which first introduced a standardised way to create compelling social engineering attacks (not only phishing), decomposing the human attack model. This result comes from the European Project DOGANA (www.dogana-project.eu) of which I (Frumento Enrico) was the scientific coordinator. The difference of this model was to concretely give a methodological approach to create social engineering attacks for red teams and not to explain from a psychological point of view why and how humans fall into error. This research, starting from those results, explored the automation with an AI.
Why do you think this is an important topic?
The topic is essential because automation of attacks is on the rise. Social Engineering is the first in class among the attack tactics and one of the most rewarded competencies in cybercrime. Automating this attack would extend the attackers’ surface to small and microenterprises. At the same time, most red teams still rely on the manual creation of attacks and miss a methodological way to create simulations, which is still based mainly on the experience of single talents. Moreover, in cyber ranges, or CTF, the social engineering simulated attacks are rarely considered (except for the SECTF at DEFCON) and rarely involve dedicated roles (e.g., psychologists or cognitive scientists) in the red or purple teams.
Is there something you want everybody to know – some good advice for our readers maybe?
Social engineering is evolving. It is not only the best and most used attack tactic today (WEF reports that approx. 95% of the attacks are caused by a human error) but also the most disrupting in terms of methodology and competencies required to perform the attack. When you attack humans, you need human sciences specialists with a hacking or cybercrime mindset. We come from 12 years of experience performing social engineering simulations (we presented our first simulation at DeepSec in 2010 and then in 2014) and a 5 million funded European project (www.dogana-project.eu) dedicated to understanding the limits of social engineering simulations. We are now adding AI to the basket of tools the attackers or red teams can use, and what we present is our early results. It is essential to come to discuss the evolutions of social engineering into social engineering 2.0, at the talk or after with a beer in hands.
A prediction for the future – what do you think will be the next innovations or future downfalls when it comes to your field of expertise / the topic of your talk in particular?
Social engineering is evolving into social engineering 2.0, which has one crucial difference: leveraging the value of data. The conjunction of data analysis and social engineering is the most significant development. We experimented with AI and data to see how far an attacker or a red team could go in automating social engineering attacks.
Dr Enrico Frumento works as a senior domain specialist at Cefriel (www.cefriel.com) in European and private-funded innovation projects on ICT Security. His research focuses on unconventional security, cybercrime and social engineering. He is the author of subject-related publications and books. He is the scientific coordinator of the project DOGANA (www.dogana-project.eu), which focuses on the contrast to modern social engineering and the technical coordinator of the project HERMENEUT (www.hermeneut.eu), which focuses on developing an innovative methodology for the dynamic assessment of organisation’s vulnerabilities and corresponding tangible and intangible assets at risk.
Dr. Francesco Morano is a scientific researcher and technical consultant in the Cybersecurity team at Cefriel. He is a member of the Order of Engineers and began his scientific research career by participating in several European projects. During his undergraduate and early professional years, he devoted himself to researching the most innovative technologies applied to various fields, including image processing and cybersecurity.