Pope steps into AI debate, warns about growing reliance on algorithms - IOL

Pope Leo XIV has warned against allowing AI algorithms to make decisions that should remain the responsibility of human conscience.

Pope Leo XIV on Wednesday warned against delegating to algorithms decisions that should be taken by humans alone, amid growing concern over the risks associated with artificial intelligence (AI).

"Technological development, the spread of mass media and social networks, and the ever-increasing use of artificial intelligence open up new possibilities by breaking down barriers and distances", the US pope said at his weekly general audience in the Vatican.

"On the other hand, relationships tend to become less and less personal and increasingly virtual, and there is a risk of becoming accustomed to delegating to algorithms decisions that belong solely to human conscience," he warned, addressing thousands of faithful gathered in St Peter's Square.

"Ensuring that social relationships have real substance and personal depth is, in this regard, the contribution that the Christian community seeks to offer," he said.

Concern over AI has grown in recent weeks following several security incidents and the emerging ability of artificial intelligence models to improve themselves without human intervention. The debate now also concerns the sector's ability to regulate itself.

In his first major document, the encyclical "Magnifica Humanitas" (Magnificent Humanity), published in May, the head of the Catholic Church issued a powerful call to "disarm" AI in order to "prevent it from dominating humanity".

In recent months, the Holy See has also stepped up its interventions and initiatives in support of international AI governance based on human dignity, particularly in the fields of nuclear technology, culture and artistic creation.

The world's biggest AI companies are working on creating a standards body, ChatGPT-maker OpenAI said on Tuesday, as fears rise about the dangers of AI and pressure is on to create a safer version of the technology.

OpenAI said it was working with Anthropic and Google on an idea that originated from a proposal by Google DeepMind's Demis Hassabis for a self-regulatory body modeled after the Financial Industry Regulatory Authority that oversees US brokers and investment firms.

In July, Hassabis called for the United States to create an organization that would test the most powerful AI systems before they are released to the public.

"Funding would need to be substantial and likely mostly come from industry, in order to attract world-class technical talent and provide the necessary compute resources for large-scale testing," Hassabis wrote.

OpenAI CEO Sam Altman on Tuesday voiced support for industry-wide collaboration and acknowledged the public's fear that competition between companies and countries might sideline safety.

"I think it's great for our industry to say we want to come together and we want to be able to coordinate and make sure we have enough time to do this safely," Altman said at a conference hosted by Salesforce in San Francisco.

"When there's any implication..." that some in the industry might "not do the right thing, I think that's when people get very scared," Altman added.

Some tech figures fear cooperation among the AI giants will push out smaller rivals.

The chief of Canadian AI company Cohere recently accused the dominant American AI labs of forming a "cartel" under the guise of safety.

"AI needs guardrails. That is not the dispute and never has been. The dispute is over who writes them, who gets to participate and whose interests the rules are protecting," Cohere co-founder and CEO Aidan Gomez wrote in a blog post on Sunday.

Concerns about AI safety have escalated in recent weeks, including warnings from workers who resigned from major AI companies amid concerns about the industry's risks.

Last week, researcher Jacob Coxon left Anthropic and warned that AI labs were "gambling with our lives." He also previously worked at OpenAI.

On Monday, former Google DeepMind researcher Bilal Chughtai echoed the concerns raised by Coxon and others.

"I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome," Chughtai wrote on social media.

On Saturday, Anthropic CEO Dario Amodei called for a slowdown in the pace of AI development, a sentiment that OpenAI CEO Sam Altman, Elon Musk and Hassabis all supported.

US Senator Bernie Sanders, who fears catastrophic consequences of AI, said that tech leaders couldn't be trusted to honor pledges to slow down development or set safety standards.

"When the future of humanity is at stake, we need binding international safety rules, not voluntary standards from the industry," Sanders said at an AI safety event in Washington.

Altman warned that AI accidents, small and large, might be inevitable.

"Accidents with any new technology are unavoidable, and we should have a great culture of transparent reporting about them," Altman said.

"Learning from them quickly and adapting as they happen I think will serve our industry well, and maybe we can avoid" the bigger, potentially catastrophic incidents, Altman continued.

US President Donald Trump on Monday dismissed warnings about the risks of AI as "hoaxes."

Nvidia CEO Jensen Huang has also pushed back, signaling a belief that the warnings on AI were part of a coordinated campaign.

"The confluence of these words and then the prediction is alarming and troubling and it shouldn't be done. It's irresponsible," Huang said Monday at a conference in Los Angeles, where Trump called him while he was on stage.

Huang, in the call, thanked Trump for "seeing through" the warnings on AI.

Nvidia sits at the center of the AI revolution, with its chips powering the vast majority of the systems, and Huang has become an informal advisor to Trump.