Online Edition

September 28-30


Phil Winder


Winder Research, UK


Dr. Phil Winder is a multidisciplinary software engineer and data scientist. As the CEO of Winder Research, a Cloud-Native data science consultancy, he helps startups and enterprises improve their data-based processes, platforms, and products. Phil specializes in implementing production-grade cloud-native machine learning and was an early champion of the MLOps movement. More recently, Phil has authored a book on Reinforcement Learning (RL) ( which provides an in-depth introduction of industrial RL to engineers.

He has thrilled thousands of engineers with his data science training courses in public, private, and on the O’Reilly online learning platform. Phil’s courses focus on using data science in industry and cover a wide range of hot yet practical topics, from cleaning data to deep reinforcement learning. He is a regular speaker and is active in the data science community.

Phil holds a Ph.D. and M.Eng. in electronic engineering from the University of Hull and lives in Yorkshire, U.K., with his brewing equipment and family.


A Code-Driven Introduction to Reinforcement Learning

The best way to improve the security of any system is to detect all vulnerabilities and patch them. Unfortunately, this is rarely possible due to the extreme complexity of modern systems. One primary threat is payloads arriving from the public internet, with the attacker using them to discover and exploit vulnerabilities. For this reason, web application firewalls (WAF) are introduced to detect suspicious behaviour. These are often rules-based and when they detect nefarious activities they significantly reduce the overall damage.

However, the overall effectiveness entirely depends on the ability to detect whether payloads are harmful or harmless. This represents a moving goal post, where attackers are constantly trying to find new patterns that evade detection. WAFs are particularly vulnerable to attack because of the sheer complexity of highly expressive languages like SQL and HTML.

A solution to this problem is to build an autonomous agent that is capable of proactively attacking a WAF until it becomes exploitable. Such an agent can generate malicious payloads and learn the weaknesses of the current WAF configuration. This presentation investigates the state of the art in the cyber-security space, specifically focussing on how reinforcement learning is helping beat the hackers.”

Session Keywords

🔑 Reinforcement Learning
🔑 Cyber Security

« Back