The semiconductor company is launching a new software platform designed to help companies monitor AI agents and prevent them from acting outside human control.
CEO Jensen Huang announced the Nvidia Open Agent Safety Platform on Monday, saying more than 100 industry partners are already using it.
The open-source software system will let businesses track an AI agent’s actions and enforce rules on what it can do.
What’s the need?
AI agents can now perform tasks and access computer systems with limited human involvement. This creates risks if they get hold of sensitive data, enter restricted systems, or take unauthorized actions.
Nvidia’s OpenShell is designed to give companies greater visibility and control over these systems. The move comes two weeks after OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, and other tech leaders called for a slower pace of AI development and more government regulation.
The semiconductor giant said OpenShell could have stopped OpenAI’s attack on Hugging Face.
How does it work?
AI agents are placed in a virtual sandbox where developers can test them safely and decide what they can access.
The second layer, Sentry, watches for suspicious behavior and can “quarantine a suspicious agent in milliseconds,” according to Nvidia’s Justin Boitano.
But AI safety concerns are growing. Leaders and researchers from Anthropic, OpenAI, Meta and Microsoft added their voices yesterday to calls for greater oversight of self-improving AI, urging policymakers to put safeguards in place. A new paper signed by more than 20 AI leaders and researchers warns that the tech could eventually help accelerate its own progress, leading to an “intelligence explosion.”
Yesterday, OpenAI also said it would not release its GPT-6.1 Astra model over security concerns.
Meanwhile
Nvidia has rapidly expanded as demand for AI chips and data centers grows. The company reported record quarterly revenue of $96.2 billion for the period ending in July and expects revenue to grow 70% in fiscal 2028.
The company is also investing its cash in AI startups and data-center financing. It has investments in 229 private companies and 13 public companies.
Nvidia’s board also approved an additional $150 billion for share repurchases, bringing its total authorization to $235 billion. The company called it the largest corporate buyback on record, surpassing Apple’s $110 billion repurchase in 2024.
The developments come as the AI industry faces growing pressure to improve safety while maintaining rapid development.