Harry Booth/San Francisco and Billy Perrigo
In a hotel room in Santa Clara, Calif., five members of the AI company Anthropic huddled around a laptop, working urgently. It was February 2025, and they had been at a conference nearby when they received disturbing news: results of a controlled trial had indicated that a soon-to-be-released version of Claude, Anthropic’s AI system, could help terrorists make biological weapons.
They were members of Anthropic’s frontier red team, which studies Claude’s advanced capabilities and tries to project worst-case scenarios, from cyberattacks to biosecurity threats. Sprinting back to the hotel room, they flipped a bed on its side to serve as a makeshift desk and pored over the test results. After hours of work, they still weren’t sure whether the new product was safe. Anthropic ended up holding up the release of the new model, known as Claude 3.7 Sonnet, for 10 days until they were certain. That may not sound like much, but it felt like an eternity for a company operating at the vanguard of an industry rapidly remaking the world.
No comments:
Post a Comment