Apple co-founder joins top scientists, leaders calling for pause on AI superintelligence development

Many prominent figures, including Apple co-founder Steve Wozniak and Virgin Group founder Richard Branson, have called for an interim ban on the development of AI superintelligence. The list, which includes more than 1,000 personalities, also includes the names of computer scientists Yoshua Bengio and Geoff Hinton, who are considered the ‘Godfathers of AI’. Other leading figures who signed the letter also include author Yuval Noah Harari, former US national security adviser Susan Rice, actor Stephen Fry and Prince Harry. The signatories argue that the development of superintelligence poses significant risks to human freedom, civil liberty, dignity, control and even extinction. They demanded an interim ban on the development of the technology until it could be developed safely and controllably with strong public buy-in. ​”Innovative AI tools could bring unprecedented health and prosperity. In addition to these tools, however, many leading AI companies have the stated goal of building superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks. This has raised concerns ranging from human economic aging and disempowerment, freedom, liberty and national security, risk and even potential human extinction,” the signatories argue. “We call for a ban on the development of superintelligence, which is not lifted until there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in,” they added. What is Superintelligence? Superintelligence, or Artificial General Intelligence (AGI), is a hypothetical stage in the development of an AI system when the technology surpasses humans in almost all cognitive tasks. However, there is no clear definition of Superintelligence in the field, and each technology company approaches the subject with a slightly different lens. However, there has been a push from almost all the frontier AI labs for the development of Superintelligence. Sam Altman, CEO of OpenAI, predicts the rise of superintelligence by 2030, while Mark Zuckerberg, CEO of Meta, said that Superintelligence is ‘now in sight’. ​Some technology leaders are optimistic about the arrival of this new technology, while others worry that its development could lead to the extinction of humanity. In a recent interaction, Bengio warned about the rise of AI with agency, with the ability to form and execute its own plans. He noted that such agents may engage in deceptive or manipulative behavior to achieve their goals and may turn against people.