Meta CEO Mark Zuckerberg has been a vocal proponent of AI, describing his vision for a future where everyone has a “personal superintelligence” integrated into their daily lives through devices like smart glasses. While he touts the potential for personal empowerment and creativity, his optimistic view is being sharply challenged by AI safety experts who warn of the technology’s potentially catastrophic dangers.
Zuckerberg’s vision is centered on a decentralized model, where AI assistants help individuals achieve personal goals, rather than a centralized, all-powerful system. He argues this approach will empower people and lead to a new era of human progress. However, critics from the AI safety community believe he is either misusing the term “superintelligence” or is not fully grasping the radical, society-altering consequences of such technology.
A Matter of Definition and Danger
David Krueger, an assistant professor of AI policy and safety at the University of Montreal, has been particularly blunt in his criticism. He told the Daily Mail that Zuckerberg’s talk of “superintelligence” is either incoherent or a “branding term.” Krueger warns that true advanced AI “poses an existential risk… is going to completely reshape the world… is going to, at a minimum, take everyone’s jobs and end up running society and lead to an unprecedented concentration of power.” He concluded with a chilling warning that it could “literally lead to the end of humanity.”
Similarly, Nick Bostrom, the author of the seminal book Superintelligence and a leading figure in the field of existential risk, defines superintelligence as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” Bostrom suggests that if such an intelligence is developed, it will be “more radically transformative than just another cool consumer appliance.” He implies that Zuckerberg’s vision for a consumer-friendly AI assistant does not fully capture the profound and uncontrollable nature of a truly superintelligent system.
The Broader Debate
Zuckerberg’s vision for “personal superintelligence” is part of a larger debate within the AI community. He has positioned Meta’s approach against what he sees as a more dystopian alternative where a centralized superintelligence automates all valuable work, leaving humanity to “live on a dole of its output.”
However, experts counter that this is a false dichotomy. They argue that even a “personal” superintelligence, if widely available, would inevitably be used by profit-maximizing individuals and businesses to automate labor, leading to mass unemployment and societal disruption, regardless of the developer’s initial intent. The core of the problem, according to critics, is not the intended use but the fundamental challenge of ensuring a superintelligence’s goals align with human values and safety—a problem that has not yet been solved.