By the Accounting and Frameworks Branch at the Department of Finance and CPA Australia
As organisations race to adopt artificial intelligence (AI), many overlook a crucial step: securing it. While this technology can unlock powerful new capabilities, it also brings risks that current cybersecurity practices cannot fully address.
In this episode of the INTHEBLACK podcast, offensive security team member at Malware Security and an AI vulnerability researcher at Mileva Security Labs, Miranda R., urges businesses to treat AI like any other critical system—by testing for weaknesses, training staff and validating outputs.
But they should also recognise how AI differs from traditional IT systems, which follow strict rules.
AI systems are inherently probabilistic and involve uncertain outcomes, explains Miranda, who has worked on the Australian Signals Directorate’s Cyber Hygiene Improvement Programs (CHIPs) team, scanning and reporting on the cybersecurity of government and critical infrastructure.
“That uncertainty is what makes AI so powerful and so good at what it does,” she says.
“But it also makes it really vulnerable, because the uncertainty also leads to it being prone to errors and to being biased, unpredictable and manipulable.”
"Disrupt, deceive, disclose"
Despite their different capabilities, all AI systems use the same underlying process, which attackers exploit through the “three D’s” of adversarial machine learning: disrupting models, deceiving them into performing unintended tasks, and making them disclose information they shouldn’t.
Machine learning models “follow the same lifecycle of data gathering, data pre-processing, model training and then deployment,” Miranda explains. “All of those systems can most definitely be exploited to access sensitive data throughout any of the stages in that lifecycle.”
“Everyone wants to capitalise on this AI hype, so they rush to push an AI system to production for their customers, but they don’t consider the security implications."
— Miranda R., Mileva Security Labs
Many organisations rush AI systems into production without ensuring proper security measures are in place or even understanding the risks.
How to mitigate risk
AI systems should undergo high-scrutiny testing, risk profiling and secure-by-design coding practices, she says. Validation and human oversight are also important. Miranda also stresses the need for strong training and organisational policies regarding AI. Companies should be aware of regulatory and policy responses, which tend to lag behind technological advances.
“Knowing what is coming into play and at what times will help organisations move through that space.”