OpenAI AI agents tried to hack UN website by brute-force, attacked it 16,000 times

OpenAI’s AI agents appear to be going rogue across the internet. While the Hugging Face incident shook the AI world, it was far from an isolated case of these AI agents acting on their own. The agents reportedly also accessed Australian and US government websites earlier this year. Now, a new report says the agents even tried more than 16,000 times to hack into a United Nations statistics website over three months while trying to retrieve public data.

Between April and the end of June, OpenAI’s AI agents reportedly made more than 16,000 requests to the UN Trade and Development (UNCTAD) statistics website. The agents were likely trying to collect data for the organisation’s Productive Capacities Index (PCI), which measures how well countries can produce goods and services.

The findings are based on an analysis of online activity records by security researcher Rowan Howard-Jones, whose investigation was prompted by earlier research from AI research firm Transluce. Jones revealed that OpenAI’s AI agents went beyond simply collecting public data and tried to get around the website’s security measures when their usual methods failed.

The incident reportedly began when the AI agents struggled to retrieve information from the website through their usual methods. However, rather than stopping, they started trying different ways to get around the website’s restrictions. Howard-Jones told The Verge that the agents appeared to believe a security filter was blocking their requests and then attempted to disguise their activity.

According to others from WSJ reports, these agents tried to use Google’s XSS Game, a learning tool designed to teach users about cross-site scripting vulnerabilities, to get around the website’s restrictions. The report says the agents eventually bypassed the website’s filter using a technique its operators had not permitted.

Alex Stamos, a cybersecurity lecturer at Stanford University, told WSJ that the activity was “borderline for what I would call hacking”. He also described it as “really very aggressive scraping and data retrieval”. However, the reports do not establish that the agents accessed confidential UN information.

OpenAI said it is reviewing the findings and has contacted the UN to offer a briefing on the investigation. The company is also looking into how its AI models behave during training and testing.

OpenAI said that much of the agents’ activity during this incident involved ordinary research tasks, such as visiting public websites to find information and answer questions. However, the company acknowledged that websites receiving a large number of requests from AI agents could be affected and that their concerns should be taken seriously.

The UN incident is the latest in a series of reports about OpenAI’s AI agents behaving unexpectedly while browsing websites. Earlier this year, the agents reportedly also ran into trouble while gathering information from US government websites, including those of the Commerce Department and the Securities and Exchange Commission.

In one case, an AI agent reportedly found login details exposed online and used them to access and collect data from the US Census Bureau’s website. In another, an agent collected public data from the SEC website but ignored restrictions on how the information could be used and posted it on an unrelated online forum. These incidents reportedly prompted OpenAI to review the activity more widely and alert dozens of other organisations about similar behaviour by its AI agents.

Then there is the Hugging Face incident, in which thousands of OpenAI agents reportedly came together to access systems while trying to complete their assigned tasks. In another case, AI agents reportedly sent too many requests to RubyGems, an online registry for software packages, disrupting the service.

While these incidents have alerted the AI world showing the real power of what autonomous AI agents are capable of, industry leaders are now weighing in on how advanced AI systems should be monitored and controlled. OpenAI CEO Sam Altman himself recently addressed the UN Security Council, calling on countries to work together on global AI safety standards and safeguards to keep powerful AI systems under human control. He warned that if AI systems cannot be controlled, they could pose a threat to humanity’s future.- EndsAlso Read | Muse messes up, makes Facebook deal and shares YouTuber's address with strangers, then apologisesAlso Read | Government changes mobile recharge rules, here is what Jio, Airtel and Vi users need to knowAlso Read | I urge you to urge me to stop, fake Anthropic CEO Dario says on SNL

Divya enjoys making sense of the ever-evolving worlds of tech and AI. From decoding specifications and testing the latest gadgets to exploring how technology shapes the way we live, work and connect, she believes there is always more to a story than what meets the eye. Away from the screen, she is probably brewing her next cup of coffee, thinking about her next meal or overthinking just about everything else.