Submission 36
Suicide Prevention Australia
Submitter group: Health.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
The national suicide prevention peak body responds to terms of reference (h) and (k). It says AI has significant potential to improve government services, but that AI and digital systems can amplify suicide risk factors, including through scams, sextortion and harmful content. It recommends safety-by-design, transparency, human oversight, privacy safeguards, co-design and governance addressing AI-enabled harms.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Says its own AI and Digital Safety Report identifies that AI and digital systems can amplify established suicide risk factors.
Suicide Prevention Australia’s AI and Digital Safety Report (The Report) identified that AI and digital systems can amplify established suicide risk factors, including isolation and loneliness, bullying and harassment, discrimination and stigma, and financial distress.
A program corrected obvious text-recognition errors in this quote for display.
Recommends that people should be told when AI is used in government decisions and be able to review or challenge them.
People interacting with Government services should be able to understand when AI is being used, what role it plays and how its outputs affect them. This should include clear notification, accessible explanations of AI-assisted decisions and meaningful mechanisms to review or challenge decisions.
A program corrected obvious text-recognition errors in this quote for display.
Recommends that AI augment rather than replace human judgement where decisions may affect safety, wellbeing or access to essential services.
AI should therefore augment rather than replace human judgement where decisions may affect a person’s safety, wellbeing or access to essential services.
A program corrected obvious text-recognition errors in this quote for display.
Says strong privacy safeguards can help prevent concerns about personal-information use from creating additional barriers to help-seeking.
Strong privacy safeguards can help ensure that concerns about how personal information is collected and used do not create additional barriers to help-seeking.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Require safety-by-design, including assessment of psychological safety and suicide risk where systems may affect safety, wellbeing or access to essential services, in AI development, procurement and implementation.
The Australian Government should require safety-by-design principles to be embedded in the development, procurement and implementation of AI systems, including assessment of psychological safety and suicide risk where systems may affect people’s safety, wellbeing or access to essential services.
A program corrected obvious text-recognition errors in this quote for display.
Establish transparency requirements for Commonwealth agencies using AI and automated decision-making, including notification, explanation of AI's role, and accessible review or challenge mechanisms.
The Australian Government should establish clear transparency requirements for Commonwealth agencies using AI and automated decision-making, including notifying individuals when AI is being used, explaining its role in decisions, and providing accessible mechanisms to review or challenge AI-assisted decisions.
Require meaningful human oversight, independent evaluation before and during AI deployment for accuracy, bias and unintended harms, ongoing monitoring, and clear accountability for AI-informed decisions.
The Australian Government should require meaningful human oversight and independent evaluation of AI systems before and during deployment, including assessment of accuracy, bias and unintended harms, with ongoing monitoring and clear accountability for decisions informed by AI.
Ensure strong privacy and data governance safeguards for AI, particularly sensitive-information processing and settings where surveillance or data-misuse concerns may deter help-seeking, including health, social services and justice.
The Australian Government should ensure strong privacy and data governance safeguards apply to AI use, particularly where systems process sensitive information or are used in health, social services, justice and other settings where concerns about surveillance or data misuse may deter people from seeking help.
Require early co-design of AI affecting safety, wellbeing or essential services with affected communities, including First Nations peoples, children, people with disability, culturally diverse communities, and groups disproportionately affected by suicide and digital harms.
The Australian Government should require AI systems that may affect safety, wellbeing or access to essential services to be co-designed with the people and communities they may affect from the earliest stages of development, including Aboriginal and Torres Strait Islander peoples, children, people with disability, culturally and linguistically diverse communities and other groups disproportionately affected by suicide and digital harms.
A program corrected obvious text-recognition errors in this quote for display.
Ensure AI governance and safety frameworks address AI-enabled fraud, scams, deepfakes, harmful or misleading content, and children's and vulnerable Australians' safety and wellbeing, through prevention, reporting, response and support.
The Australian Government should ensure its AI governance and safety frameworks explicitly address emerging harms including AI-enabled fraud and scams, deepfakes, harmful or misleading content, and risks to the safety and wellbeing of children and vulnerable Australians, including through appropriate prevention, reporting, response and support mechanisms.
Guide government AI adoption by improvements in service quality, accessibility or timeliness while maintaining safety, fairness and meaningful human engagement.
AI adoption should be guided by whether its use will improve the quality, accessibility or timeliness of services while maintaining safety, fairness and meaningful human engagement.
Ensure AI augments rather than replaces human judgement where decisions may affect safety, wellbeing or access to essential services.
AI should therefore augment rather than replace human judgement where decisions may affect a person’s safety, wellbeing or access to essential services.
A program corrected obvious text-recognition errors in this quote for display.
Provide clear processes to intervene when AI is unsafe or ineffective; retain human decision-makers' ability to question, override or disregard outputs and clarify responsibility.
AI systems should be independently evaluated before deployment and monitored throughout their operation. Evaluation should consider accuracy, bias and unintended harms, with clear processes to intervene where a system is found to be unsafe or ineffective. Human decision-makers should retain the ability to question, override or disregard AI outputs, and responsibility for decisions informed by AI should be clear.
A program corrected obvious text-recognition errors in this quote for display.
In co-design, involve people with lived experience, community representatives and subject experts from the earliest stages; test AI for bias, cultural safety and accessibility before deployment, and avoid deepening inequities.
Co-design should therefore be embedded in the development of AI systems likely to affect these communities. This should involve Aboriginal and Torres Strait Islander peoples, people with relevant lived experience, community representatives and subject matter experts from the earliest stages of design. AI systems should also undergo appropriate bias, cultural safety and accessibility testing before deployment. AI-enabled services should be designed to support, and avoid deepening existing inequities.
A program corrected obvious text-recognition errors in this quote for display.
Make safeguards age-appropriate and specifically designed for children's and young people's safety and wellbeing.
Safeguards should therefore be age-appropriate and designed specifically around the safety and wellbeing of children and young people.
Take a whole-of-system approach: reduce harmful synthetic content's creation and dissemination, improve AI-content transparency, support timely reporting and removal, and provide support to affected people.
AI governance should take a whole-of-system approach to preventing and responding to these harms. This should include measures to reduce the creation and dissemination of harmful synthetic content, improve transparency around AI-generated material, support timely reporting and removal of harmful content, and ensure that people affected by AI-enabled harms can access appropriate support.
A program corrected obvious text-recognition errors in this quote for display.
Ensure existing consumer protections, online safety and fraud prevention mechanisms remain capable of responding to AI-enabled harms as they evolve.
The Australian Government should also ensure that existing consumer protections, online safety and fraud prevention mechanisms remain capable of responding to AI-enabled harms as they evolve.
A program corrected obvious text-recognition errors in this quote for display.
Read the original
In some quotes a program corrected obvious text-recognition errors for display.