AI nearly had US troops attack a Chinese ship. Nobody could check its work.

AI Summary

A report warns that an AI system nearly led U.S. troops to attack a Chinese ship, while personnel lacked the means to independently verify its work. It argues that AI deployments in national security and other high-stakes fields need independent oversight and verifiable records, potentially modeled on aviation flight recorders.

The lack of independent oversight and verifiable records in artificial intelligence deployment poses significant risks to national security, finance and healthcare, necessitating a solution similar to aviation's flight recorders.