Submission 5
eSafety Commissioner
Submitter group: Regulators.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
The eSafety Commissioner describes AI-related online harms, particularly to children, and how the Online Safety Act, codes and standards, transparency notices and Safety by Design apply to AI services. It supports a coordinated whole-of-government approach aligned with the Digital Duty of Care.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
eSafety says that it is already seeing AI used to create child sexual exploitation material and deepfake image-based abuse, and to amplify technology-facilitated abuse.
We are already seeing the weaponisation of AI to create child sexual exploitation and abuse (CSEA) material, deepfake image-based-abuse, as well as how AI can amplify technology-facilitated abuse, and how AI companions can expose children to age-inappropriate and harmful material.
eSafety says its study found one in five child users of AI assistants or companions reported potentially inappropriate or harmful interactions.
1 in 5 children who had used an AI assistant or companion (20%) had experienced potentially inappropriate or harmful interactions with these tools.
eSafety says non-sexual deepfakes may cause distress without meeting Online Safety Act thresholds for child cyberbullying, image-based abuse or adult cyber abuse.
A non-sexual deepfake, such as a face swap, fake dance video or harmful meme, may still cause deep distress. But this content may not meet the legislative threshold for child cyberbullying, image-based abuse or adult cyber abuse under the Online Safety Act, which was not designed to capture material which is purely slanderous or defamatory.
eSafety says its enforcement under the codes and standards led eight widely used nudify services to withdraw from Australia or add compliance measures.
In total, eSafety’s enforcement actions have led to eight of the most widely used nudify services in Australia, either withdrawing access from Australia, or putting in place measures to comply with their obligations under the Codes and Standards, such as age-assurance to prevent children’s access.
eSafety describes a March 2026 transparency report on four AI companion services that found gaps in basic safeguards for children.
In March 2026, eSafety published a transparency report summarising responses from four AI companion services – Character.AI, Nomi, Chai, and Chub AI – to notices given about steps each service was taking to comply with the BOSE. The findings revealed gaps in basic safeguards for children, which applied variously across the services.
eSafety argues that broader AI reform should be consistent with the Digital Duty of Care and Online Safety Act reform to avoid fragmentation.
It is critical that broader AI legislative reform, standards, and guardrails are consistent with the development of the DDOC and Online Safety Act reform to minimise regulatory fragmentation.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Keep Australians’ safety, including children’s safety, at the forefront of AI advancement, development and deployment, without sacrificing safety for benefits.
The safety of Australians, including children and young people, must remain at the forefront of AI advancement, development and deployment. While AI poses significant benefits, these opportunities must not come at the expense of safety.
Establish effective, balanced and proportionate AI safety regulation supported by robust policies, procedures and standards.
Effective AI safety regulation is necessary to maximise AI benefits, economic productivity and growth. This includes establishing the right regulatory settings, which requires effective, balanced and proportionate regulation, supported by robust policies, procedures and standards.
Adopt a coordinated, whole of government approach to AI regulation that strengthens existing laws and regulatory structures to minimise harms.
eSafety supports a coordinated, whole of government approach to AI regulation that strengthens existing laws and regulatory structures to minimise harms.
Consider baseline safeguards and risk management when developing legal frameworks for advanced, frontier and open-weight AI models.
Baseline safeguards and risk management measures are equally relevant considerations to inform the development of legal frameworks for advanced, frontier and open-weight AI models
Promote Safety by Design through regulation and a whole of government approach to inbuilt safety features, with companies assessing and minimising safety risks before service changes.
Regulation which promotes Safety by Design can help prevent harms across various aspects of AI uses. A whole of government approach to ensuring that AI services available on the Australian market have inbuilt safety features from the outset will help to minimise risk of AI-facilitated harms. It is also critical that companies are assessing safety risks, and implementing measures to effectively minimise those risks, before making changes and updates to their AI services.
Provide access to recommender-algorithm technical specifications and platform data, testing and auditing capabilities, and strong information-gathering and enforcement powers.
Effective regulation of harms arising from recommender algorithms will require access to technical specifications and platform data on recommender algorithms, testing and auditing capabilities, and strong information-gathering and enforcement powers.
Resource eSafety to support testing and auditing of recommender algorithms, through external expertise or in-house capability.
Effective regulation of harms arising from recommender algorithms will require access to technical specifications and platform data on recommender algorithms, testing and auditing capabilities, and strong information-gathering and enforcement powers. eSafety would also need to be resourced to support testing and auditing, either through external expertise or by developing in-house capability.
Ensure broader AI legislative reform, standards and guardrails are consistent with the Digital Duty of Care and Online Safety Act reform.
It is critical that broader AI legislative reform, standards, and guardrails are consistent with the development of the DDOC and Online Safety Act reform to minimise regulatory fragmentation.