AI may become one of the most important technologies shaping the future of sentient life. It could transform medicine, scientific research, economic productivity, and more. It could even reshape the kinds of minds that exist and the experiences they are capable of having.

AI could help us substantially reduce suffering. But it could also increase suffering, potentially even on astronomical scales, by enabling new forms of misuse, intensifying conflict, amplifying harmful institutions, concentrating political or economic power, or accelerating dangerous technologies.

Current AI governance has focused on many important issues, including safety, security, economic disruption, democratic institutions, international competition, fairness, and existential risk. Yet the issue of suffering—and how AI can be governed to reduce it—has received comparatively little attention.

SFAIG

We believe this may deserve to become a more explicit area of inquiry. We might call it Suffering-Focused AI Governance (SFAIG). If current AI governance addresses a wide range of moral concerns, SFAIG focuses on suffering, though not necessarily exclusively. In brief, it asks how the institutions, policies, and norms that shape AI can be designed to reduce suffering as effectively as possible.

SFAIG brings together questions that are currently scattered across public policy, ethics, philosophy of mind, animal welfare, economics, technical AI safety, and related fields. The goal is to provide a coherent framework for thinking about AI and suffering: identifying neglected research questions, prioritizing interventions, developing shared conceptual frameworks, and fostering collaboration across disciplines.

Without such a framework, we are concerned that suffering may remain neglected within AI governance, increasing the risk of futures containing vast amounts of suffering.

Approach

SFAIG spans the many cause areas that fall under AI governance, including public health, animal welfare, warfare and conflict, democracy and concentration of power, AI safety and alignment, and AI welfare. To prioritize among these areas, it employs an Importance, Tractability, and Neglectedness (ITN) framework. SFAIG asks where AI is likely to have the greatest impact on suffering, which problems are most solvable, and where attention and resources are most lacking.

This framework helps prioritize interventions, many of which overlap with proposals in mainstream AI governance. These may include, for instance, better evaluation and monitoring of AI systems, restrictions and guardrails on the use of AI, international cooperation, and broader efforts to prepare for the future.

A key feature of SFAIG is examining what mainstream AI governance is already trying to achieve and why. Existing proposals are often motivated by concerns such as national security, economic stability, cyberattacks, bioweapons, or existential risk. SFAIG asks whether current proposals are robust from the perspective of reducing suffering and, where appropriate, how they might be modified to better achieve that objective. It also draws attention to questions that may otherwise receive insufficient consideration, particularly those concerning nonhuman animals, the long-term future, and the possibility of artificial suffering.

Key Questions

Among the questions SFAIG seeks to answer are:

• How can AI governance reduce risks while preserving AI’s potential to alleviate suffering?
• How can AI governance empower those seeking to reduce suffering while preventing excessive concentrations of political or economic power?
• How should uncertainty about AI capabilities, timelines, artificial sentience, and future populations affect our priorities?
• Which interventions are most robust across different technological, social, and geopolitical futures?
• How should we think about tradeoffs between the near-term and the long-term future when prioritizing efforts to reduce suffering?

Conclusion

SFAIG will appeal most strongly to those who already prioritize reducing suffering. But even if we are uncertain whether reducing suffering should be our highest moral priority in theory, virtually every plausible moral theory regards suffering as deeply important. Yet because this issue has received little attention in AI governance, there is a risk that suffering will be neglected in practice. It thus seems worthwhile to dedicate substantial research to AI governance with an explicit focus on reducing suffering, even if it is not our only concern.