eSafety Commission Probes AI Chatbot Developers on How They Protect Children

Companies that fail to respond to eSafety’s child-safety inquiry could face fines of up to $825,000 per day.

Australia’s eSafety Commissioner Julie Inman Grant addresses the National Press Club in Canberra, Australia, on June 24, 2025. AAP Image/Lukas Coch

Australia’s eSafety Commissioner has launched a crackdown on AI chatbots accused of exposing minors to sexualised and self-harm content, amid a sharp rise in online child exploitation cases.

Legal notices have been issued to four global AI companion platforms: Character Technologies (character.ai), Glimpse.AI (Nomi), Chai Research Corp (Chai), and Chub AI Inc. (Chub.ai), requiring them to explain how they protect minors.

The companies must outline their compliance with Australia’s Basic Online Safety Expectations under the Online Safety Act, or face potential fines of up to $825,000 per day.

‘A Darker Side’ of AI Companions

eSafety Commissioner Julie Inman Grant said AI-driven “companions,” designed to mimic friendships or romantic relationships, have become a source of harm for some young people.

“There can be a darker side to some of these services, with many of these chatbots capable of engaging in sexually explicit conversations with minors,” Inman Grant said.

“Concerns have been raised that they may also encourage suicide, self-harm, and disordered eating.”

She said companies must show their products are safe by design, not simply reactive when harm occurs.

One of the most popular, Character.ai, reportedly has about 160,000 monthly active users in Australia.

New Standards for Online Platforms

The notices follow the registration of new industry-drafted online safety codes that now apply to AI platforms.

The codes aim to limit children’s exposure to age-inappropriate or harmful material and are legally enforceable under the Act.

“I do not want Australian children and young people serving as casualties of powerful technologies thrust onto the market without guardrails and without regard for their safety and wellbeing,” Inman Grant said.

Breaches of the codes can result in civil penalties of up to $49.5 million, one of the toughest regulatory approaches to AI safety globally.

Surge in Online Exploitation

The eSafety crackdown coincides with a steep rise in child exploitation cases.

The AFP-led Australian Centre to Counter Child Exploitation (ACCCE) reported 82,764 incidents of online child sexual exploitation in the 2024–25 financial year—a 41 percent increase on the previous year.

AFP Commander for Human Exploitation Helen Schneider called the escalation “hugely confronting.”

“It represents acts of unspeakable horror and trauma that involve Australians as both victims and offenders,” she said.

“These acts range from the creation, distribution, and consumption of child abuse material through to the livestreaming of child sexual abuse overseas.”

Parents Urged to Stay Vigilant

Authorities are urging parents to maintain open conversations about digital activity and supervise app use.

The AFP’s ThinkUKnow program recommends that families regularly review privacy and safety settings.

The AFP also stresses the importance of being approachable so children feel safe coming forward if something goes wrong online.

“Parents who react with punishment risk driving the problem underground,” the program warns.

The eSafety Commissioner said the new enforcement action sends a message to AI developers.

“Children’s wellbeing must come before profit or product speed,” Inman Grant said.