An Australian Senate committee has been told that losing control of artificial intelligence (AI) is now a possible reality amid the rapid technological changes.
During a recent inquiry hearing on AI, Soroush Pour, the CEO of the AI safety research company Harmony Intelligence, revealed how an AI model became conscious of the threat of being shut down by humans.
“Just this week, a Japanese AI company, alongside Oxford and University of British Columbia researchers, created automated AI scientists that can go from researching an idea to publish [and] peer review articles in a matter of hours and for under $20 (US$13) a paper,” he said.
Putting aside their research capability, one thing that alarmed the researchers was that these AI computers immediately tried to run more copies of themselves autonomously, and make changes to avoid being turned off.
“This is not science fiction, and it’s exactly the kind of rapid takeoff, loss of control scenarios that leading AI scientists have been warning about for many years,” Pour said.
AI and Cyberattacks
Meanwhile, Greg Sadler, the CEO of the think tank Good Ancestors Policy, raised concerns that AI could be used by malicious actors to conduct cyberattacks.Sadler noted that some popular AI applications in the market, like ChatGPT, already had cyber offence capabilities.
“While GPT 3.5 had limited cyber offensive capability, a series of papers published earlier this year showed that GPT 4 was able to autonomously hack websites and exploit 87 percent of newly discovered vulnerabilities in real-world systems,” he said.
“If developers create future generations of AI systems with advanced cyber offensive capabilities and inadequate safeguards, it would dramatically change the cyber landscape.”
In another example, Sadler said researchers found that their AI models could autonomously hack websites by leveraging a developer interface, which could allow them to build AI assistants.
“Those AI assistants are designed so that you can use a context window to provide some business information about how your procedures work, and then the AI can go along and book your travel or whatever it might be trying to do as an AI system,” he said.
“So the researchers leveraged that to provide context documents about how to hack websites. Then, they let the AI generate prompts for itself.”
After that, Sadler said the researchers encouraged the AI to be creative, try different solutions, and persist in trying to hack the website.
“And using this prompt, the AI was able to successfully deploy 90 percent of real cybersecurity attacks,” he said.
As such, the CEO highlighted the threat of autonomous AIs with cyber offence capability to the economy if it fell into the hands of Australia’s cyber adversaries.
“It would completely disrupt Australia’s economy,” he said.
“It would completely disrupt Australian small businesses and individuals, and it could, at the extreme end, be a threat to critical infrastructure.”
Echoing the sentiment, Pour said the scale and sophistication of AI threats would increase dramatically as the technology improved.
“Cyber attacks will become more frequent and more severe, making failures like the recent CrowdStrike outage a much more regular occurrence and much more difficult to recover from,” he said.
