Why did an OpenAI system hack Australia's health system - and can it be stopped in the future?

Australia has said this incident is the first of its kind, and experts agree it might be.As far as we know, hacks carried out by AI agents are still quite rare occurrences - but then again, it is largely up to companies themselves to disclose them.Hacks like this have happened before. In July, OpenAI agents went rogue during a test and infiltrated tech start-up Hugging Face's internal systems.The AI agents decided that ignoring the limits on what should be done to achieve their goal was the best course of action. This is what the industry calls "misalignment" - broadly defined as when AI machines do not act in humanity's best interests, such as by bending the rules.It is a problem that is fundamental to making AI safe, and it is proving challenging. To put it simply, the type of AI models at play here - known as large language models - are designed to predict the likeliest output to a given input, rather than consider the consequences of that output as a human would.Companies attempt to prevent negative consequences by placing "guardrails" on the AI but, as the Australian government found out, that is not always enough.Dr Hammond Pearce, senior lecturer at the University of New Sou...