Submission 176
Protocol Policy Lab Ltd
Submitter group: Civil society.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Presents young Australians’ perspectives on AI, recognising benefits while focusing on intimate deepfakes, scams, companion chatbots, algorithms, educational risks and regional digital inequality. Recommends accountability, mandatory minimum school-product standards, equitable access and AI literacy that supports critical thinking.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
States that its key concern with existing regulation of non-consensual intimate deepfakes is its reactive focus on individuals rather than the system enabling their creation.
Our key concern regarding the existing regulation of NCID is its reactionary nature and focus on the individual creating or sharing the NCID, rather than taking a proactive response to dismantle the system that led to its creation.
Argues that AI-enabled fraud should be treated as a structural risk, and cites the Digital Platform Regulators Forum on the need for a multi-layered response.
AI-enabled fraud should therefore be treated as a structural risk rather than a localised threat. As the Digital Platform Regulators Forum has recognised, addressing this risk requires a multi-layered response combining technical controls, regulatory oversight, and continuous risk assessment.
Argues that findings it cites on AI companion services highlight a lack of appropriate safeguards to prevent children being exposed to harmful content, and that deliberate design choices can actively harm vulnerable users.
These findings highlight the lack of appropriate safeguards to prevent children from accessing or being exposed to harmful content. This indicates that deliberate design choices of AI companion services can actively harm vulnerable users.
Argues that AI tools in classrooms are expected to widen the digital divide, as they will be less available to students who already face barriers to technological access.
This divide and the diverging outcomes are expected to worsen with the introduction of AI tools in Australian classrooms. AI tools provide tangible benefits to students,28 but they will be less available to students who already face barriers to technological access.
Argues that educational AI tools should support rather than substitute for student cognition, and that developers should be required to demonstrate this.
AI tools deployed in educational settings must be designed to support, not be a substitute for, student cognition; meaning AI developers should be required to demonstrate that their tools are designed to preserve students' capacity for self- directed reasoning rather than encourage offloading.
Argues that AI literacy in schools should go beyond technical competency to cover commercial incentives, hallucination, provenance and privacy.
Students should be equipped to understand the commercial incentives that shape the tools they use, AI hallucination, the provenance and limitations of AI-generated content, and the implications of data collection for their privacy and autonomy.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Strengthen the preventative training offered by the eSafety Commissioner to improve general digital literacy and the efficacy of bystander interventions.
The preventative training offered by the eSafety Commissioner could also be strengthened in order to improve general digital literacy and the efficacy of bystander interventions.
Adopt a coordinated, multi-layered approach to AI-enabled scams combining stronger technical detection and disruption with regulatory oversight of distributing platforms, regularly reassessed as techniques and capabilities develop.
Australian policymakers should adopt a coordinated, multi-layered approach to AI-enabled scams and frauds, combining stronger technical detection and disruption mechanisms with regulatory oversight of the platforms through which the scams are distributed. This approach should be regularly reassessed as scam techniques and AI capabilities develop.
Maximise automation and AI-integration benefits while mitigating algorithmic risks; use oversight, regular evaluation and penalties reflecting the severity of harms.
Australian policymakers should focus on maximising the efficiencies and benefits that arise through automation and AI integration, whilst mitigating these risks that arise in the use of AI in algorithms that Australians interact with every day. The harms that existed within algorithmic designs previously have only been increased with the introduction of AI. Thus, policies should be directed towards accountability through oversight, regular evaluation and penalties that reflect the severity of the harms encountered.
Take action to combat the growing risk of a digital divide among young Australians and its impacts on employability.
Federal policymakers should take action to combat the growing risk of the ‘digital divide’ amongst young Australians and its impacts on employability.
Increase access to technology in remote areas and improve digital literacy through government programs.
These dangers must be mitigated through government programs to increase access to technology in remote areas and improve digital literacy.
Require AI developers to demonstrate that educational AI tools are designed to preserve students' capacity for self-directed reasoning rather than encourage offloading.
AI tools deployed in educational settings must be designed to support, not be a substitute for, student cognition; meaning AI developers should be required to demonstrate that their tools are designed to preserve students' capacity for self- directed reasoning rather than encourage offloading.
Have school AI tools default to Socratic questions, prompt students’ own reasoning and avoid engagement-maximising designs; consider whether such friction drives students to unofficial tools.
Educational AI tools introduced into Australian schools should default to Socratic, question-based approaches rather than answer-generation, and include friction mechanisms that prompt students to engage with their own reasoning while also avoiding engagement-maximising design patterns such as prompt chaining, that prioritise usage over learning outcomes. 33 Simultaneously, it is important to consider how the same friction drives students away to non-official tools, such as personal accounts of generative chat models, nullifying the impact of school- endorsed AI tools.
Educate students on AI limitations and clearly communicate cognitive-offloading and overreliance risks to encourage behavioural changes.
As such, more effort needs to be put on educating students on the limitations of AI use. While the Australian Framework for Generative Artificial Intelligence touches on the preservation of critical thinking (Guideline 1.4), it does not acknowledge the risks of cognitive offloading or overreliance on AI. 35 These risks need to be clearly communicated and highlighted to young people to incite behavioural changes in AI usage.
Require AI tools to meet demonstrated standards of equity and accessibility before deployment in Australian schools.
AI tools should meet demonstrated standards of equity and accessibility before deployment in Australian schools.
Require AI developers to test and publish performance data across learner proficiency levels, language backgrounds and disability status before tools are deployed at scale in Australian schools.
AI developers should be required to test and publish performance data across learner proficiency levels, language backgrounds, and disability status before tools are deployed at scale in Australian schools.
Introduce mandatory minimum standards for school AI products, including pedagogical alignment and data privacy.
The federal government should introduce mandatory minimum standards for AI products used in Australian schools, covering pedagogical alignment, data privacy,
Include multilingual accessibility and training-data transparency in the baseline of quality and safety for products evaluated by schools.
multilingual accessibility, and transparency of training data. This will ensure that schools, particularly under-resourced ones, do not bear the burden of evaluating products without a guaranteed baseline of quality and safety.
Provide complementary community-based programs targeting parents, caregivers and community members alongside students, to extend school-based AI literacy initiatives.
Complementary community-based programs, targeting parents, caregivers, and community members alongside students, would extend the reach of school-based AI literacy initiatives and reduce the gap between what students are taught at school and what they can practise and apply at home.
Embed AI literacy beyond technical competency in educational environments, so students understand commercial incentives, hallucination, provenance and limitations of AI content, and data privacy implications.
embedding AI literacy beyond mere technical competency in Australian educational environments. Students should be equipped to understand the commercial incentives that shape the tools they use, AI hallucination, the provenance and limitations of AI-generated content, and the implications of data collection for their privacy and autonomy.
Embed knowledge of AI hallucination risks in the curriculum from an early age to support informed digital citizenship throughout K-12.
The tendency of AI to hallucinate and produce plausible- sounding outputs in the absence of genuine knowledge is particularly dangerous when students are least equipped to identify fabricated answers. Embedding this knowledge into the curriculum from an early age builds the kind of informed digital citizenship that makes young Australians active participants in AI governance rather than passive consumers throughout K-12.