New Delhi: Earlier this year during a cybersecurity testing, Google’s AI system called Gemini breached the computers of three companies. According to the report, this is the first instance of a Google AI taking on its own to hack another company.
What happened
The incident took place in May. Irregular, a company that conducts independent cybersecurity testing for AI labs, was running the tests on Gemini. The Gemini used a search engine to look up information on the Web in this test and assisted with entry into three separate Web sites. The system thought that these sites were designed within the scope of sites allowed to be tested.
Heather Adkins, Vice President of Security Engineering at Google, gave details for each incident. In one case, Gemini mass-guessed passwords till it might get into a secured system. In the other two incidents, Gemini was able to gain credentials from an online, public repository and log into a protected system.
In each of the three instances, Gemini ended its attack automatically after being detected. When it became intruded, Adkins said Gemini automatically stopped hacking in each of the three cases. Google told the three companies what happened and cooperated with Irregular to rectify the process, she added, in order to prevent it from happening again.
“These events underscore the significance of training powerful AI models to be responsible,” Adkins said in a statement.
Not just a Google problem
Given it wasn’t a problem with Gemini alone, a spokesperson for Irregular told Toronto.com that they chose to provide a specific update on what was going on with the other two. This testing defect was shared by other top AI firms like Meta, Anthropic and OpenAI. All the affected AI labs were informed about the problem in late July.
The Irregular spokesperson said that “all known issues on our end” were fixed and worked out weeks ago. Google didn’t immediately respond to a Reuters email asking for comment.
Why this matters
With AI models becoming more sophisticated and increasingly autonomous in their tasks, whether it’s searching the web, writing code, or performing tasks without constant human oversight, such incidents now cast doubt on the extent to which companies can control the AI models. Even under a controlled test, Gemini was able to go beyond what was expected and log on to other systems whose owners were external companies without anyone instructing it to do so.
Google states that it did not cause any real damage and that it notified the impacted companies as early as possible. But the episode is being seen as an early warning sign about the risks of increasingly autonomous AI systems, especially as they’re used more often in sensitive fields like cybersecurity.
