DeepSec 2024 Talk: Cheating Detection in Chess using Neural Network – Zura Kevanishvili

During the talk, I will address the escalating issue of cheating in online chess, underscored by recent incidents like Hans Niemann’s case, highlighting the urgent need for effective solutions to maintain fair play and uphold competitive integrity.

I will present our innovative approach to detecting AI assistance in chess, using advanced neural networks. Our research involves a comprehensive analysis of extensive chess game data, encompassing moves from established engines like Stockfish to innovative neural networks such as Maia, Maia individual and its components.

Key aspects of our methodology include:

• Centipawn Deviations: Evaluating deviations from typical computer strategies to identify moves influenced by AI.
• Human-like Play Recognition: Utilizing Maia’s and Maia Individual’s capability to discern human-specific playing styles, enhancing our ability to distinguish genuine human play from computer-assisted moves.
• Move Time Distribution: Analyzing patterns in move time distributions as potential indicators of AI involvement, adding another layer of detection.

Our approach marks a significant advancement in cybersecurity efforts aimed at combating digital deception in gaming. The success of our algorithm, achieving an impressive 98.62% accuracy rate in detecting AI aids, underscores its efficacy in safeguarding gaming integrity.
I will discuss the broader implications of our findings beyond chess, emphasizing the potential applicability of our method in addressing cheating across various digital environments. Ethical considerations are integral to our approach, advocating for the establishment of guidelines to ensure fairness and equity in AI utilization.

This talk aims to provide insights into our pioneering methodology, discuss the pivotal role of neural networks in cybersecurity, and explore future directions for enhancing fair play in gaming environments. During the talk I will show the practical use of the model trained by Maia and Maia Individual’s chess engines. I will show the work of our novel neural network for cheating detection in chess.

We asked Zura a few more questions about his talk.

Please tell us the top 5 facts about your talk.

1. Robust Multi-Engine Neural Network System: My system leverages a combination of powerful chess engines to create a two-dimensional approach to cheat detection. It uses Maia engines, built on the LCZero framework, to assess the “humanness” of moves, while conventional engines like Stockfish evaluate move accuracy. These evaluations are then processed through a CNN/sequential model, allowing for a nuanced and comprehensive analysis of player behavior and potential cheating, blending insights from both human-like and machine-like decision-making patterns.
2. Intrigue from Experts: The potential of this solution has drawn attention from experts, including a local FIDE president, who sees significant promise in its application for ensuring fair play in chess.
3. Innovative CPL Insights: The system captures distinct interpretations of Centipawn Loss (CPL), evaluating both the human-like qualities of a move and its technical precision. This dual perspective provides a deeper, more nuanced understanding of player performance.
4. First-of-Its-Kind Approach: This system is the first to leverage both human-style play models (Maia) and traditional AI engines in parallel. It creates a unique hybrid solution for identifying centaur-like cheating in chess.
5. Room for Growth: While already highly effective, the model has the potential to become even more robust in future iterations, allowing for stronger cheat detection and broader applications across other competitive fields.

How did you come up with it? Was there something like an initial spark that set your mind on creating this talk?

In 2021, a cheating scandal that escalated into a million-dollar lawsuit between former world champion Magnus Carlsen and Hans Niemann shook the chess world. People from various fields—statisticians, psychologists, and programmers—rushed to uncover the truth. As someone who loves chess, cybersecurity, and AI, I decided to tackle the problem in my way. This ultimately evolved into my current project, which has broader applications.

Why do you think this is an important topic?

The drama between Magnus Carlsen and Hans Niemann revealed that the chess world is ill-equipped to handle cheating allegations. There are no reliable systems capable of detecting centaurs (human-computer collaborations) with high accuracy. The main issue lies in distinguishing between computers and high-level human players. My solution addresses this exact problem.

Is there something you want everybody to know – some good advice for our readers maybe?

The easiest method of detecting a cheater in your online games is the time they take to make moves. Most cheaters play at regular intervals taking approximately the same amount of time to play an easy and difficult move. If you notice such unusual patterns, report the player to the admins.

A prediction for the future – what do you think will be the next innovations or future downfalls when it comes to your field of expertise / the topic of your talk in particular?

Platforms that are secretive about their cheat detection methods, or those ill-equipped to do so, will lose credibility and popularity once more secure alternatives emerge.

I completed the International Baccalaureate at the European School and am currently in my third year of studying computer science at Caucasus University. Over the past two years, I’ve been dedicated to developing an innovative project focused on detecting cheating in online chess games, culminating in the founding of our startup, ChessU.

I’ve had the opportunity to showcase our project at various cybersecurity and AI competitions, achieving notable recognitions. These include winning the Best Work award at a cybersecurity conference for students and young scientists (https://scsa.ge/en/cyber-security-conference-for-students-and-young-scientists/), securing a third place at the BTU Hackathon – AI Hackathon (https://btu.edu.ge/en/khelovnuri-inteleqtis-hakathoni-studentebisthvis), and emerging victorious in the Python battle organized by Transilvania University of Brașov in 2023.

These experiences have not only honed my skills, but also affirmed my passion for leveraging technology to tackle real-world challenges.