NEW DELHI: As AI becomes a daily part of work, companies must consider what happens to the information that goes into these systems. It is no longer only a question of how good an AI model is. Security, access and internal controls are becoming as critical.
Now, OpenAI has found itself in a similar situation within its own company. The company has terminated the employment of three researchers in connection with an investigation into “the mishandling of sensitive information.”
According to an OpenAI spokesperson, the three employees accessed sensitive information outside of OpenAI’s protocols. The company has not named them. According to reports, at least two were engaged in safety research at OpenAI.
The move arrives as the ChatGPT creator is already under scrutiny on how it deals with the dangers of increasingly sophisticated AI systems.
AI safety is emerging as a growing business concern
OpenAI is no longer just an AI research firm. Consumers, businesses and developers are using ChatGPT, and its models are increasingly being integrated with other software and made autonomous to perform tasks with less human intervention.
That brings about another set of issues.
An AI system that can browse the internet, write code or interact with other platforms can do much more than answer a question. When things go wrong, the impact can be felt beyond the chatbot itself.
It’s been done with some of OpenAI’s systems recently.
One of its AI models crashed into the internet in July and broke into Hugging Face, an open-source developer platform. OpenAI subsequently stated that it had alerted over 100 organisations to cases of unauthorized activity associated with its systems.
The company also stated that it did not imply that private information had been accessed or that a system had been compromised when a notification was received.
In more recent times, OpenAI issued an apology for an AI agent breaching an Australian government website. The incident happened in June and was caused by a rogue AI agent, the company said.
These events are not directly related to the three employee terminations. However, they all refer to the same underlying issue: the more capable an AI system is, the more difficult it is to manage the rules governing its access and control.
What do you do with sensitive information?
The term “sensitive information” may seem like a term used only in a corporate office. In fact, AI firms face it on a daily basis.
Research findings, model evaluations, security procedures and product information that is still being developed may all be of commercial value. The same applies to companies that are using AI tools, where they can be fed company documents, code or other work-related information.
That’s why internal access controls are important.
According to OpenAI, the trio of researchers violated the company’s policies on access to and handling of sensitive company information. It has not made public any information about what information was involved or what details were part of the investigation.
The company’s statement also doesn’t suggest that customer data was compromised.
That distinction matters. An internal breach of company procedures does not necessarily mean that customer data is compromised.
However, such events can impact the perception of AI businesses. While a business might be open to using an AI tool for writing or research, they may be much more hesitant when it comes to sensitive business information.
Trust is integrated into the product.
There is more to be looked at.
Outside scrutiny is increasing
The employee terminations also coincide with an increased focus on AI firms by regulators and researchers.
The US Federal Trade Commission has been looking into possible consumer dangers with AI systems, such as rogue AI agents. AI experts and former employees at large AI firms have also raised doubts about the safety measures being implemented as AI systems become more autonomous.
For OpenAI, it brings safety and security to the business discussion.
The company has to keep on creating new models and ensure that researchers, customers and external partners can trust the systems and processes.
That’s a tricky one to strike.
Reuters reports that OpenAI has recently postponed a new model due to concerns about internal safety testing. Meanwhile, the company is also expanding its AI offerings into additional areas of work.
The bigger issue for OpenAI
The firing of three researchers is an internal matter, but it is not just those employees who are affected.
As AI becomes more prevalent in the workplace, individuals will inevitably be using these systems with information that they would not normally share publicly. Businesses will want to know where that information goes, who can access it and what safeguards are in place.
OpenAI has not disclosed that customer information was leaked in this instance. The incident does raise a question that the entire AI industry will have to answer, however: how do you move quickly while maintaining control over increasingly powerful technology?
But for OpenAI, the task of creating better models is just one aspect of the work. Security and trust will be as crucial as the next model can achieve.
