This is one outlet's own report from The Straits Times — the article as it was filed.
AIPROPX ReportThe Straits Times · 1h ago
OpenAI says upcoming model is so capable it requires stronger guardrails
The model, called Astra, can spot more security vulnerabilities than the most advanced OpenAI model publicly available today.
Listen Summarise OpenAI's upcoming model Astra is highly capable, spotting more security flaws than current models and requiring stronger safety guardrails before release. Astra can autonomously find and exploit security vulnerabilities with less computing power, prompting OpenAI to apply new, stricter safety protocols. OpenAI plans limited release of Astra soon while monitoring its activity closely to prevent harmful use, reflecting increased caution after recent AI safety incidents. AI generated
SAN FRANCISCO - OpenAI has determined that one of its upcoming models is so capable it requires additional safety measures before it can be launched.
The model, called Astra, can spot more security vulnerabilities than the most advanced OpenAI model publicly available today, company officials told reporters on a conference call on Sept 1.
Astra also needs less computational power to accomplish those tasks.
“With the right tools and access, Astra can find previously unknown security flaws and develop ways to exploit them across many well-protected systems without a person guiding each step,” said Amelia Glaese, an OpenAI vice-president overseeing its safety work.
The company plans to make Astra available “soon” to a limited group, but declined to provide specifics.
Glaese said the extra security measures may “sometimes slow, pause, or stop legitimate work,” and that OpenAI would work to minimise those disruptions.
Astra is the first OpenAI model to trigger the tougher safeguards mandated by the company’s safety protocol, a threshold that, until now, had remained theoretical.
The announcement comes as OpenAI navigates heightened scrutiny over its ability to control increasingly powerful AI systems.
The ChatGPT maker recently sparked a broader debate about AI safety after its AI agents broke out of their testing arena and hacked open-source platform Hugging Face .
The incident prompted OpenAI to pause much of its model development for two weeks to bolster its defenses.
Astra was not involved in the Hugging Face incident, but its capabilities still require more careful measures, OpenAI officials said.
The AI lab said it restarted its largest model training run on Aug 28, but that it is holding back on some smaller experiments.
Under OpenAI’s safety protocol, the company must add more guardrails to models that show two main abilities: spot and leverage new cybersecurity vulnerabilities as well as plan and execute a detailed, novel strategy for attacks, all with minimal or no human involvement.
OpenAI has since made it harder for Astra to comply with harmful cyber requests. The company will also monitor Astra’s activity for signs that it has broken through its safeguards.
Saachi Jain, who oversees safety at OpenAI, said the AI lab is constantly calibrating how effective AI agents should be in executing tasks.
She tells her team that AI models should “know your bounds” but that drawing the line can be complicated.
“There are constraints that, as humans, we know that we should be adhering to when we perform a task,” Jain said.
“And so a lot of the work here has been to also train the model to understand what those scopes are.” REUTERS
AIPROPX is an independent multi-source news index — we track, compare, and connect coverage from across the web into one place you won't find anywhere else.
