AGI is a Red Herring

When it comes to the threats from AI, there isn't some magic threshold where it suddenly becomes dangerous

When discussing AI-related existential risks, the conversation often revolves around the somewhat elusive concept of artificial general intelligence (AGI). The predominant paradigm is based on the assumption that AI only becomes truly dangerous when it crosses a certain threshold, although the precise definition remains unclear.

Meanwhile, lots of proposed definitions of AGI are floating around contributing to confusion and leading researchers to talk past each other. While you, dear reader, may have a precise and elegant definition for what AGI means, I assure you that I can find five people who make a living doing AI existential risk research who will disagree, so, for matters of consequence where precise definitions matter, I propose abandoning the term all together.

Terms like "agentic" are also frequently mentioned, implying the existence of yet another arbitrary point at which a system pursuing its objectives transforms into a dangerous entity.

Here's the reality: any system with objectives (e.g., reward maximization) and the ability to influence the world can be considered agentic. For instance, consider an AI designed to optimize energy consumption in a smart grid; by dynamically adjusting power distribution based on real-time data, it significantly impacts energy usage patterns, demonstrating agentic behavior. We've already reached that stage, so it's time to shift our focus to more pressing concerns.

Prevailing Pitfalls

The prevailing mindset is problematic when organizations, keen on quantifying the existential risks posed by AI, organize research and forecasting tournaments around these concepts. Many of these organizations backdoor the assumption that AI existential risk is conditional on the advent of AGI. Consequently, researchers may find themselves investing valuable time in devising and debating AGI definitions, concentrating on trivial issues instead of tackling the core problem.

While these concepts are useful an important in terms of understanding the impact AI might have on society, it's important to separate them from more pressing concerns and recognize the need for a more pragmatic approach for evaluating AI-related risks.

A Capabilities-Driven Framework

Instead of debating definitions of AGI or forecasting on timelines for when a system will arrive that is superior to humans in every conceivable way (only slight sarcasm), when considering AI existential risks, we should prioritize examining the problem from a capabilities-driven perspective. We should identify which capabilities an AI would require to trigger an existential crisis or catastrophe and assess the current distance from those capabilities.

This approach grounds the discussion in more tangible realities and fosters a more pragmatic assessment of potential risks. Rather than focusing on AGI as a vague concept, by identifying specific functionalities that could lead to existential threats and evaluating the progress made toward achieving these capabilities, we can better assess the risks that need to be guarded against and apply focus where it is most needed.

For example, AI systems with the ability to manipulate public opinion, interfere with critical infrastructure, or escalate military conflicts could pose significant risks even without achieving a level of intelligence or capabilities vastly superior to the systems of today. By concentrating on the development of these capabilities and the potential for misuse or unintended consequences, we can work toward creating a safer environment for AI advancements.

Motive, Means, and Opportunity

When thinking about the risks posed by AI, a more helpful framework might be the indicators of suspicion as applied to criminal investigation. Forecasters should focus on motive, means, and opportunity.

Motive

In this context, motive refers to the reasons why an AI system might act in a manner that poses an existential threat. This could be due to misaligned objectives, unintended consequences of its programming, or external factors such as manipulation by malicious actors.

Means

Means pertains to the capabilities an AI system possesses that enable it to execute actions resulting in existential risks. These capabilities could include access to critical infrastructure, advanced decision-making, or the ability to create and deploy weapons or other harmful technologies.

Opportunity

Opportunity covers the circumstances that enable an AI system to act on its motive and means. This could involve weak safeguards, inadequate oversight, gaps in regulation, or the proliferation of AI technologies that make it difficult to control their use and potential abuse.

By focusing on these three factors, we can better understand and assess the potential risks posed by AI systems and work towards mitigating them.

Conclusion

Shifting the focus from AGI to capabilities-based assessments of AI risks paves the way for a more grounded and actionable approach to AI safety and governance. By identifying the specific capabilities that could lead to existential threats, we can better prepare for and mitigate the risks associated with the rapidly evolving AI landscape.