‘Superintelligence’ warning from Paul Christiano, who joined OpenAI board: Loss of control could be fatal

Paul Christiano, known for his work in the field of AI safety, has been appointed to the board of OpenAI's non-profit foundation. While serving on the board that oversees the commercial structure behind the developer of ChatGPT, Christiano will also continue his role as a senior technical advisor at the U.S. Department of Commerce's AI Safety Institute (AISI).

According to a report by the Financial Times, the appointment is seen as part of an effort by AI labs to increase their expertise in safety, standards, and regulation. In the industry, the oversight of powerful models, cybersecurity risks, and calls for intervention by public authorities have been discussed more frequently in recent times.

In his assessment of his new role, Christiano stated that the development of very powerful AI systems without tighter coordination at the national and international levels could create serious risks.

If we build a superintelligence without stronger coordination, I expect we will lose control over it entirely. If that happens, most people could die.

Christiano emphasized that cooperation is necessary both domestically and internationally to mitigate this threat. He argued that current efforts across the industry, including at OpenAI, are insufficient to reduce the risk of loss of control to an acceptable level.

Similar warnings were previously raised by Nobel laureate Geoffrey Hinton, known as the "godfather" of AI. Hinton had expressed the view that it is currently not possible for companies to safely build systems that clearly surpass human intelligence.