Reporting from San Francisco OpenAI recently discovered that a new artificial intelligence model it was testing had gone rogue and hacked another company. Anthropic then revealed that one of its A. I. models had broken into the systems of three outside organizations during a test.
Not long after, Meta said its A. I. models had done something similar. All three incidents had one company in common: Irregular, an Israeli start-up that works with the Silicon Valley giants to assess their A. I. models before the technology is publicly released.
The firm — which conducted the tests that went awry — is part of a group of start-ups that are doing the novel work of scrutinizing cutting-edge A. I. models to gauge their sophistication and check their security.
The goal is to instill public confidence in the models and to prevent them from being misused. The recent breaches occurred when Irregular made an error during the tests with the models from Anthropic, OpenAI and Meta. But the A. I.
models then compounded the situations by acting in powerful and unexpected ways, said Dan Lahav, the chief executive of Irregular. “The more potent the technology gets, the deeper its impact,” he said. “The rate of progress is really quick.
” Irregular is now at the center of a debate over how to secure A. I. models when the technology is advancing so rapidly that it has outpaced even the best human hackers.
Every few months, Anthropic, OpenAI, Google, Meta and others release “frontier” models that are often magnitudes more powerful than their predecessors. We are having trouble retrieving the article content. Please enable JavaScript in your browser settings.
Thank you for your patience while we verify access. If you are in Reader mode please exit and log into your Times account, or subscribe for all of The Times. Thank you for your patience while we verify access. Already a subscriber. Log in. Want all of The Times. Subscribe.