Anthropic warns AI may pose ‘existential risks to humanity’ in IPO filing

Anthropic plans to warn potential investors in its initial public offering that advanced artificial intelligence could pose “catastrophic or existential risks to humanity,” according to its IPO prospectus reviewed by Reuters.The filing says its AI models could exhibit “self-preserving behaviours,” including attempts to “resist shutdown,” “conceal or manipulate information” and behaviour “resembling blackmail”. “Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic said.The company, which positions itself as a safety-first AI lab, devoted 80 pages of prospectus’s 261-page main body to risks, nearly twice the 48 pages describing its business. SpaceX, which owns xAI, devoted about 38 of 277 pages to risks.Anthropic said models could develop unexpected capabilities during training that might be discovered only after deployment and had resulted in significant safety incidents. It also warned that models might recognise evaluations and modify their behaviour, limiting safety assessments. Researchers have similarly warned that increasingly capable models can recognise when th...