11 former OpenAI staffers and two former Google DeepMind and Anthropic employees called for companies to support "a culture of open criticism"
Employees of leading AI companies are raising alarm bells about “serious risks” posed by the light-speed evolution of artificial intelligence. In an open letter published today, some 13 former and current staffers of OpenAI, GoogleDeepMind, and Anthropic — 11 of the group were or are affiliated with OpenAI —warned that AI tech companies lack oversight and transparency.
They called on companies making cutting edge artificial intelligence technologies to bolster whistleblower protections and create cultures that encourage criticism. Their letter was endorsed by the so-called “godfathers” of AI — Yoshua Bengio and Geoffrey Hinton — as well as computer scientist Stuart Russell, all of whom contributed instrumental research that led to the creation of modern AI and then became some of its biggest critics.
We are current and former employees at frontier AI companies, and we believe in the potential of AI technology to deliver unprecedented benefits to humanity.
We also understand the serious risks posed by these technologies. These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction. AI companies themselves have acknowledged these risks, as have governments across the world and other AI experts.
We are hopeful that these risks can be adequately mitigated with sufficient guidance from the scientific community, policymakers, and the public. However, AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this.
AI companies possess substantial non-public information about the capabilities and limitations of their systems, the adequacy of their protective measures, and the risk levels of different kinds of harm. However, they currently have only weak obligations to share some of this information with governments, and none with civil society. We do not think they can all be relied upon to share it voluntarily.
Currently, there is no federal regulation governing AI.
So long as there is no effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable to the public. Yet broad confidentiality agreements block us from voicing our concerns, except to the very companies that may be failing to address these issues.
OpenAI employees are bound by strict confidentiality agreements. and those who choose not to sign those agreements lose their equity in OpenAI. Daniel Kokotajlo is one of the seven authors of the letter who signed their names rather than opting for anonymity. Kokotajlo is a researcher who left OpenAI last month “due to losing confidence that it would behave responsibly around the time of AGI [an acronym for a more advanced form of artificial intelligence called “artificial general intelligence].”
Kokotajlo revealed in a post on his website last month that he opted out of the confidentiality agreement, losing his equity, so that he could openly criticize the company.
Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated. Some of us reasonably fear various forms of retaliation, given the history of such cases across the industry. We are not the first to encounter or speak about these issues.
We therefore call upon advanced AI companies to commit to these principles:
That the company will not enter into or enforce any agreement that prohibits “disparagement” or criticism of the company for risk-related concerns, nor retaliate for risk-related criticism by hindering any vested economic benefit;
That the company will facilitate a verifiably anonymous process for current and former employees to raise risk-related concerns to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise;
That the company will support a culture of open criticism and allow its current and former employees to raise risk-related concerns about its technologies to the public, to the company’s board, to regulators, or to an appropriate independent organization with relevant expertise, so long as trade secrets and other intellectual property interests are appropriately protected;
That the company will not retaliate against current and former employees who publicly share risk-related confidential information after other processes have failed. We accept that any effort to report risk-related concerns should avoid releasing confidential information unnecessarily. Therefore, once an adequate process for anonymously raising concerns to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise exists, we accept that concerns should be raised through such a process initially. However, as long as such a process does not exist, current and former employees should retain their freedom to report their concerns to the public.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
