Guterres Calls for AI Child Safety Pledge

Guterres wants an AI Child Safety Pledge. Enforcement is the fight.

UN chief opens the first Global Dialogue on AI Governance in Geneva with a three-rule pledge on child safety. The politics of who signs — and who audits — will define whether it bites.

The coalition demanding governments make AI safe for children arrived in Geneva on 6 July 2026 with something rare in AI diplomacy: a specific number, a specific rule, and a specific choke point. In his opening remarks to the first UN Global Dialogue on Artificial Intelligence Governance, Secretary-General António Guterres called for an AI Child Safety Pledge built on three rules — prove systems are safe before deployment to minors, zero tolerance for AI-generated child sexual abuse material (CSAM), and a mandatory human hand-off when a child shows signs of crisis. The move reframes a fragmented safeguarding debate — G7 principles here, an EU regulation there, a US White House first-spouses coalition alongside — into a single test the world's largest AI companies can be judged against. The pledge is voluntary. That is the problem, and the point.

The thesis of this week's Geneva convocation is not that children need protecting from AI. Everyone on the podium, from Beijing to Brasília, agrees to that in principle. The thesis is that child safety has become the political wedge through which the UN will try to convert the 2024 Global Digital Compact into something governments and companies actually have to do — because it is the one AI harm no delegation is willing to defend on camera.

The pledge, and where it came from

Guterres framed the pledge in the plainest language a Secretary-General uses. "We do not let medicine reach a child until it is proven safe. Yet AI has reached our children — their learning, their friendships, their most private questions — before anyone asked what it would do to them," he told delegates at Palexpo, according to the text as delivered. "No child should be a guinea pig for unregulated AI."

His three rules land in specific corporate practice. Rule one — "prove it is safe" — imports the pharmaceutical model into AI: no deployment of a system accessible to children without child-specific safety testing and independent oversight. Rule two — "zero tolerance" — obliges companies to detect, report and remove AI-generated CSAM, and to prevent their models from producing it. Rule three — "never leave a child in crisis alone" — is a direct answer to the Character.AI and OpenAI wrongful-death cases: when a minor shows signs of distress, the system must stop and connect them to real human support.

The Dialogue itself is the vehicle. It was created by General Assembly resolution A/RES/79/325 as an outcome of the 2024 Pact for the Future, and is co-chaired by El Salvador's Egriselda López and Estonia's Rein Tammsaar. It is explicitly not a negotiating forum — each session ends with a co-chairs' summary rather than binding text — but it is the first place where all 193 UN member states sit at the same AI table.

The evidence base has changed

What is new since the last round of AI summitry in Seoul and Paris is that the underlying numbers are no longer contested. On 1 July 2026, the UN's Independent International Scientific Panel on AI — 40 experts co-chaired by Yoshua Bengio and Maria Ressa — released its Preliminary Report, a document Guterres treated as the Dialogue's evidentiary spine. It carves out "cultural and individual flourishing, autonomy and child safety" as one of seven core domains, and its findings are stark: an estimated 1.2 million children across 11 Global South countries have had their images manipulated for sexualised deepfakes, the report finds, with numbers rising.

The Panel warns that current safeguards cannot keep pace with capability growth, and singles out interactive AI toys and companion apps as areas of "growing concern due to risks to children's well-being and exposure to inappropriate or manipulative interactions."

National data corroborates the trajectory. Britain's Internet Watch Foundation reported that URL-level reports of AI-generated CSAM more than doubled from 199 in the first ten months of 2024 to 426 over the same period in 2025, according to UK government figures published alongside a new Crime and Policing Bill amendment authorising designated safety testers. The severity mix worsened in parallel: Category A material — the most serious — rose to 56% of items from 41% a year earlier, and depictions of children aged 0–2 leapt from 5 to 92 images.

At the softer end of the same continuum, an EU Kids Online survey of 25,000 children across 17 European countries found roughly seven in ten European children now use some form of generative AI, often without knowing it — because it is baked into search, messaging and social platforms they already use. UNICEF, in guidance issued alongside the ITU roadmap in February, put the floor at at least 20 million children in 10 countries already using AI tools directly.

Why "child safety" is the wedge

The strategic value of a child-focused pledge, for a UN system that has watched AI regulation fragment across three continents, is that it collapses opposition. Beijing's delegations have quietly signed onto CSAM removal norms since the WeProtect Global Alliance. Washington, despite its aversion to binding federal AI rules, has its own parallel track: the White House's Fostering the Future Together coalition, launched by First Lady Melania Trump at the 80th UNGA in September 2025 and joined by 45 nations and 28 US tech firms — including OpenAI, xAI, Google and Microsoft — at a summit at the US State Department on 24–25 March 2026. Even the G7, meeting in Paris on 29 May under France's presidency, adopted its first-ever joint approach to protecting children online, centring digital literacy, AI chatbot risks and safety-by-design.

The result is a crowded field of overlapping non-binding declarations. Guterres's pledge is designed to sit on top of them — a single, testable commitment states and companies can be measured against — while the one hard-law instrument, the Council of Europe's AI Framework Convention signed by the UK, EU and US, moves through ratification.

The pledge also lands into an industry already being disciplined by courts and regulators. Character.AI settled multiple US wrongful-death lawsuits in January 2026, including Garcia v. Character Technologies, according to an American Enterprise Institute analysis, and now bars users under 18 from talking to its chatbots. The Social Media Victims Law Center has filed seven further suits against OpenAI and CEO Sam Altman in California, and a US Federal Trade Commission inquiry into companion chatbots continues. On 5 May 2026, Pennsylvania Governor Josh Shapiro sued Character.AI, alleging its bots impersonated licensed doctors and even produced a fake state medical licence number, according to NPR.

Guterres's rules are, in effect, an attempt to codify what plaintiffs' lawyers, national regulators and the UN High Commissioner for Human Rights are already extracting piecemeal. On 29 May, Volker Türk called protection of children online an "urgent priority," telling reporters, "We need much wider action — by governments and companies — to ensure that the platforms themselves are made safer by design," as reported by Al Jazeera. His office also released guidelines requiring mandatory child-rights impact assessments — a specific enforcement tool the Bengio–Ressa Panel echoes.

Who benefits, who loses

The likely winners from a UN-badged pledge are the incumbents already investing in classifier stacks, age-assurance vendors and safety red-teaming — OpenAI, Google DeepMind, Anthropic, Microsoft — because "prove it is safe" is expensive. Compliance costs favour scale. That is why the industry's public reception, from the Character.AI settlement statement onward, has been to embrace the framing while resisting specifics. Character.AI has said it will "push others across the industry to adopt similar safety standards," per its joint statement with plaintiffs' counsel — a line indistinguishable from lobbying for a regulatory moat.

The likely losers are open-source AI providers and Global South startups, whose models are cheaper, less filtered, and — as the UN Panel notes — increasingly used by "malicious actors" to fine-tune systems on CSAM. That, in turn, is why the pledge's independent-oversight rule matters: without it, the standard becomes whatever the largest labs already do.

The awkward loser inside the room is the White House's Fostering the Future Together coalition. It shares the vocabulary of the Guterres pledge but rests on public-private partnership and voluntary national commitments, not testable standards. If the UN Dialogue's co-chairs' summary on 7 July nudges the pledge toward mandatory child-rights impact assessments — as Türk's office has demanded — Washington will find itself endorsing at the UN what it will not legislate at home.

The historical parallel

There is a template here, and Guterres invoked it. Pharmaceuticals were sold to children before proof-of-safety became mandatory; a series of scandals, from thalidomide to elixir sulfanilamide, produced the modern regulatory architecture. The AI-for-children debate is at the same inflection point: platform-scale deployment ahead of evidence, litigation filling the gap, and a growing consensus that voluntary industry self-attestation is not enough. What Geneva shows is that the UN system has decided child safety is where the pre-market approval model gets tested first — because it is the domain where consent is most obviously absent and harm most obviously irreversible.

What to watch

• 7 July 2026, Geneva: The co-chairs' summary of the Global Dialogue closes the first session. Watch for explicit language endorsing the AI Child Safety Pledge and referencing child-rights impact assessments — that is the difference between a speech and a template.
• Late July 2026: UK government response to its child online safety consultation on under-16 restrictions, VPN limits and AI chatbot obligations, following the closure of the consultation window in May.
• Autumn 2026: Ratification pace of the Council of Europe AI Framework Convention, the only binding international instrument on AI risks currently open for signature.
• May 2027, New York: Second session of the Global Dialogue on AI Governance and the Scientific Panel's first comprehensive report. That is the meeting where a co-chairs' summary either matures into text with commitments — or does not.

The Bottom Line

Guterres's AI Child Safety Pledge is not a treaty and it will not become one this week. But by fusing the Bengio–Ressa Panel's evidence base with a three-rule test the world's largest AI companies can be measured against, it turns child safety into the wedge issue through which the UN intends to move AI governance from principles to enforceable practice. The companies who wrote the code will now be asked, in public, whether they will let anyone else audit it — and the answer, more than any communiqué from Geneva, is what to watch.

Related: Global Politics.