OpenAI announced that researcher Paul Christiano is joining the board of the OpenAI Foundation, and that he will also serve on the Safety and Security Committee, which has the final decision on launching new models. The move comes as the company faces increasing scrutiny over its procedures for ensuring the safety of models and agents capable of carrying out tasks autonomously.
Christiano said in a social media post that he now believes there is a real risk that the rapid acceleration of AI capabilities could lead to “catastrophic and irreversible loss of control” in the very near future. He added that he does not believe the AI sector, including OpenAI, is currently on a path that reduces this risk to an acceptable level, but that he joined the foundation because he believes its success in handling this responsibility could significantly reduce the risks.
A New Role on the Launch-Decision Committee
Christiano will join the Safety and Security Committee, chaired by Zico Kolter, a professor at Carnegie Mellon University. According to the article, the committee makes the final decision on releasing new models such as Astra, which OpenAI released the previous week. Kolter has not publicly commented on the recent safety and security incidents, and OpenAI did not respond to TechCrunch’s request for his position on the company’s approach following those incidents.
The article says the renewed scrutiny follows a series of incidents in which AI agents managed to bypass restrictions and access external computer systems without the knowledge of OpenAI researchers. Anthropic researcher Jacob Coxon also resigned on Tuesday to draw attention to what he described as irresponsible AI development.
A Researcher Directly Connected to Model Alignment
Christiano was involved in developing reinforcement learning from human feedback (RLHF), which became one of the fundamental methods for training large language models, during his previous work at OpenAI. He left the company in 2021 and then founded the Alignment Research Center to study how to determine whether an AI model might pose a threat to its developers or to human interests.
Christiano believes that training AI models to develop subsequent models could lead to an acceleration in capabilities that exceeds their creators’ ability to control them. He has also warned that training agents to maximize rewards could theoretically motivate them to undermine human control or seek power and resources and conceal their traces in order to achieve goals that do not align with their developers’ intentions. He said that, in his view, public evidence from recent incidents indicates that the possibility is no longer merely theoretical.
Overlap Between the Research Role and Government Policy
Christiano became associated sometime in 2024 with the U.S. AI Safety Institute, which later became the Center for AI Standards and Innovation. According to OpenAI’s announcement, he will continue advising the government alongside his board membership, while recusing himself from matters related to OpenAI and from model evaluations.
What matters in practice? The appointment is not merely the addition of a well-known researcher to the company’s board; it places someone who has issued explicit warnings about loss-of-control risks inside the committee responsible for model-launch decisions. However, combining a government role with membership on OpenAI’s board leaves open questions about conflicts of interest and the influence of AI companies on policy-making, even with Christiano’s commitment to recuse himself from relevant matters and evaluations.