DeepSec 2016 Talk: Machine Duping – Pwning Deep Learning Systems – Clarence Chio
Give a man a computer, and you 0wn him for a day. Teach a man to employ machine learning, and he will have to battle Skynet for a lifetime. This quote might not be the exact copy of the original, but it will do. Machine now learn stuff. Hence the are of machine learning is the new playground for start-ups, old school companies, researchers, and hackers, of course. A new era of sapiosexual attraction to artificial minds has begun. Information security is not spared. Algorithms have long been a part of defence. Now they are being used with machine learning. Since algorithms and machines run on networked computers, they can be attacked. At DeepSec 2016 Clarence Chio will explain to you how it can be done.
Deep learning and neural networks have gained incredible popularity in recent years. But most deep learning systems are not designed with security and resiliency in mind and can be duped by any attacker with a good understanding of the system. In this talk, we will dive into popular deep learning software and show how it can be tampered with to do what you want it do, while avoiding detection by system administrators. Besides giving a high level overview of deep learning and its inherent shortcomings in an adversarial setting, we will focus on tampering real systems to show real weaknesses in critical systems built with it. In particular, this demo-driven session will be focused on manipulating an image recognition and captcha solving system built with deep learning at the core.
The systems we use today are already very complex. By adding more complexity we will increase the attack surface. Algorithms from the field of deep learning and neural networks are no exception. Keep this in mind when you evaluate new products containing these features. Bear in mind that complexity also makes testing very difficult. By definition any system that learns, alters its state. This means your security component is constantly changing. This will very probably have an impact on your security defence, one way or another. We hope that this presentation will open your mind to this challenge.
Clarence Chio graduated with a B.S. and M.S. in Computer Science from Stanford, specializing in data mining and artificial intelligence. He currently works as a Security Research Engineer at Shape Security, building a product that protects high valued web assets from automated attacks. At Shape, he works on the data analysis systems used to tackle this problem.
Clarence spoke on Machine Learning and Security at DEF CON 24, GeekPwn Shanghai, PHDays Moscow, BSides Las Vegas and NYC, Code Blue Tokyo, SecTor Toronto, Hack in Paris, and QCon San Francisco (2015-2016). He had been a community speaker with Intel, and is also the founder and organizer of the ‘Data Mining for Cyber Security’ meetup group, the largest gathering of security data scientists in the San Francisco Bay Area.