Foundation models have forever changed AI research. In the future, they need to be released responsibly.

Percy Liang is director of the Center for Research on Foundation Models, a faculty affiliate at the Stanford Institute for Human-Centered AI and an associate professor of Computer Science at Stanford University.

Humans are not very good at forecasting the future, especially when it comes to technology.

Foundation models are a new class of large-scale neural networks with the ability to generate text, audio, video and images. These models will anchor all kinds of applications and hold the power to influence many aspects of society. It’s difficult for anyone, even experts, to imagine where this technology will lead in the coming years.

Foundation models are trained on broad data using self-supervision at scale so they can be adapted to a wide range of tasks. This breakthrough approach to AI represents a dramatic improvement in accuracy and opens up new possibilities, given that organizations no longer need to start training a model from scratch for every new AI application. It also poses clear risks as the downstream consequences are hard to predict, let alone control. If not governed effectively, foundation models such as GPT-3, PaLM and DALL-E 2 could cause significant harm to individuals and society, whether it’s intended or not.

One of the key pieces of governance is establishing community norms for the release of foundation models so that a diverse group of researchers has the opportunity to closely analyze them. Currently, companies like Microsoft, Google, OpenAI, Meta and DeepMind each take a different position on how to release their models. Some embrace a broadly open release, while others prefer a closed release or one that’s limited to a small set of researchers.

While we do not expect consensus, we believe it is problematic for each foundation model developer to determine its release policy independently. A single actor releasing an unsafe, powerful technology could knowingly or unknowingly cause significant harm to individuals and society. Moreover, developers would benefit from sharing best practices, rather than incurring the economic and social costs of rediscovering certain harms over and over again.

Fortunately, releasing new foundation models doesn’t have to be an all or nothing proposition. A multidimensional framework of policies would consider four key questions.

1. What to release: Papers, models, code and data can be released separately; each has an impact on expanding scientific knowledge and decreasing the potential risk of harm.
2. Who gets access to the release: Given the risks to releasing models, the sequencing of who has access matters. For example, there may be an inner circle of trusted colleagues, a middle circle of researchers who apply for access and the general public.
3. When to release the model: The timing of a release should depend on both intrinsic properties, such as the results of safety evaluations, and external conditions, such as what other models exist and how much time has elapsed.
4. How to release the model: The process of releasing new assets must include a bidirectional means for communication between developers and researchers so that the release is maintained over time.

To help developers make better-informed decisions with input from the broader community, we at the Center for Research on Foundation Models at the Stanford Institute for Human-Centered AI have proposed creating a foundation models review board. The role of the board would be to facilitate the process of developers releasing foundation models to external researchers. This approach will expand the group of researchers who can study and improve foundation models, while helping to manage the risks of release.

The basic workflow of the review board would look something like this:

• A developer posts a call for proposals describing the available foundation model(s) and what the developer believes to be the most critical areas of research on these models.
• A researcher submits a research proposal specifying the research goals, the type of access needed to accomplish those goals and a plan for managing any ethical and security risks.
• The board reviews the research proposal and deliberates, potentially with additional input from the researcher.
• Based on the board’s recommendation, the foundation model developer makes a final decision to approve, reject or defer the proposal.
• If the proposal is approved, the foundation model developer releases the desired assets to the researcher.

A review board like this would ensure that release decisions are made in a highly contextual way: for a particular researcher, for a particular purpose, for a specific foundation model with a specific form of access, and at a certain point in time. This concreteness makes it much easier to reason about the benefits and risks of any given decision. Community norms on model releases would emerge through a series of these decisions.

We need to recognize that foundation models are evolving rapidly and require norms for governance. The models we see in five years may be unrecognizable to us today, much like the models today would be inconceivable five years ago. Those developing foundation models should engage with the community to develop best practices around releasing new models. Downstream users, including application developers and researchers, should be more aware of the models they are using, what data was used to train those models and how the models were built — and if that information is not available, demand to know.

An important hallmark of human-centered AI is transparency and openness, providing collective governance, marked by fair procedures and aspiring toward superior outcomes. Given the immense uncertainty and our poor ability to forecast the future, we cannot make decisions solely based on anticipated outcomes. We need to focus on developing a resilient process that will allow us to be prepared for whatever lies ahead.

As foundation models research is still in its early stages, input and feedback are extremely valuable. For those who work with foundation models, whether it be through research or development, we’d love to hear from you at contact-crfm@stanford.edu.

Additional contributions to this report come from Rob Reich, professor of political science and, by courtesy, professor of philosophy at Stanford University; Rishi Bommasani, a Ph.D. student in the computer science department at Stanford; and Kathleen Creel, HAI-EIS Embedded EthiCS Fellow at Stanford.