AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 549

Black Dog Institute

Submitter group: Health.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

Describes opportunities for AI in mental health research and care, and argues many general-purpose tools used for mental health support fall outside existing health regulation or anticipated uses. Recommends research investment, privacy reform, a Digital Duty of Care, an expert advisory group, literacy campaigns and an AI Safety Institute remit encompassing personal safety.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Argues that AI has potential to benefit mental health research and care, including early intervention and workforce capacity.

    AI is emerging as a powerful tool in mental health research and care, offering opportunities for early intervention, personalised treatment, ongoing monitoring, and enhanced capacity for the mental health workforce.

  2. Says most general-purpose AI systems were not designed for mental health care or independently evaluated, so many fall outside existing health regulatory frameworks.

    Most general-purpose AI systems were not designed for mental health care and have not been independently evaluated for their benefits and harms on mental health. As a result, many fall outside existing health regulatory frameworks or may be used in ways that were not anticipated by current intended use or regulation.

  3. States that there is no single legislative framework governing AI for mental health in Australia, with regulation spread across jurisdictions.

    There is no single legislative framework governing AI for mental health in Australia. Regulation is complex and relies on multiple factors that span different jurisdictions (e.g., health care, data privacy, regulation of medical technologies).

  4. States that AI chatbots are being used by people experiencing suicidal thoughts, creating a pressing need for safeguards for vulnerable users.

    The potential consequences of these risks are heightened by the fact that AI chatbots are being used by individuals experiencing suicidal thoughts, creating a pressing need for safeguards to protect vulnerable users.

  5. Argues that mental health information should be treated as sensitive health data with the highest level of protection.

    Mental health information should be treated as sensitive health data and afforded the highest level of protection.

  6. Argues that Australia lacks a national body monitoring AI-related harms to individuals and that the AI Safety Institute's remit should extend to personal safety.

    Australia currently lacks a national body responsible for monitoring and responding to AI- related harms experienced by individuals. This gap is particularly important in mental health settings. As Australians increasingly use AI for mental health support, the Australian AI Safety Institute's remit should extend beyond national security to encompass personal safety.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Acknowledge AI’s potential to transform health research, knowledge gaps about public, professional and system use for mental health, and innovation outpacing research and its funding.

    The Committee should acknowledge that: • AI has the potential to revolutionise health and medical research. • There are substantial knowledge gaps on the use of AI for mental health including how AI is being used for mental health by the public, mental health professionals and within the mental health system. • The use of AI technologies for mental health are innovating faster than research, and the funding systems that enable research, can keep pace.

  2. Fund a national AI and mental health research/innovation program uniting researchers, clinicians, educators, policy/technology experts and people with lived experience to ensure AI is developed and used safely, ethically and effectively.

    Recommendation 1: Increase funding for AI in mental health research Establish a national research and innovation program for AI and mental health that brings together world-leading researchers, clinicians, educators, policy advisors, technology experts and people with lived experience to ensure AI is developed and used safely, ethically and effectively for mental health:

  3. Advance knowledge of AI’s mental-health risks and benefits, including priority and marginalised groups; improve treatment access and outcomes, develop better treatments and accelerate research implementation.

    Enhance knowledge into the risks and benefits of AI for mental health: advance scientific knowledge about how AI can be used to improve treatment access and outcomes, develop better treatments, and speed up the time it takes for research discoveries to be implemented in practice. Advance scientific knowledge about the risks and benefits of AI, including for priority groups, and marginalised groups.

  4. Within the research program, train the mental health workforce in safe AI use and evaluate AI-enabled training’s use and impact in preparing the next generation of health professionals.

    Change practice: educate and train the health professional workforce in the safe use of AI in practice for mental health, and evaluate the use and impact of AI-enabled training to train the next generation of health professionals.

  5. Within the research program, generate and synthesise evidence to influence policy so AI is used safely, equitably and effectively for mental health.

    Drive policy reform: generate and synthesise evidence to influence policy to ensure AI is used safely, equitably and effectively for mental health.

  6. Build AI mental-health research capacity by supporting, growing, training and mentoring future researchers and health professionals, including students, early- and mid-career researchers and lived-experience researchers.

    Build research capacity: support, grow, train, and mentor the future generation of researchers and health professionals in AI in mental health including students, early-and mid-career, and lived experience researchers.

  7. Establish clear national complaints and reporting mechanisms to identify when AI-related mental health harms occur; potential mechanisms include mandatory incident reporting, public transparency reports, independent safety audits, whistleblower mechanisms and post-market surveillance.

    The need for clear national complaints and reporting mechanisms to identify when AI-related mental health harms occur. This is especially important when products fall outside established mechanisms for post-market surveillance, e.g., uses that are not software as a medical device. Potential mechanisms include mandatory incident reporting, public transparency reports, independent safety audits, whistleblower mechanisms and post-market surveillance and monitoring.

  8. Support proposed privacy reform restricting platform providers from using sensitive data for purposes such as advertising and placing stronger limits on selling or sharing data without consent.

    We support proposed reform to privacy laws in Australia which would restrict platform providers from using sensitive data for purposes such as advertising, and place stronger limits on selling or sharing data without consent.

  9. Treat mental health information as sensitive health data with the highest level of protection.

    Mental health information should be treated as sensitive health data and afforded the highest level of protection.

  10. Require meaningful, easy-to-understand informed consent, not relying on pre-ticked boxes or complex disclosures.

    Require meaningful and easy to understand informed consent: Consent should not rely on pre-ticked boxes or complex disclosures that may not be read or understood, particularly during periods of distress.

  11. Restrict secondary use of mental health data, such as targeted advertising or profiling.

    Platforms should not be able to use mental health data for secondary purposes such as targeted advertising or profiling.

  12. Limit the sharing or sale of mental health data, with any disclosure requiring explicit user consent.

    Limit the sharing or sale of mental health data, with any disclosure requiring explicit user consent.

  13. Legally require digital platforms to identify, prevent and mitigate foreseeable harms; disclose AI; avoid substituting AI advice for professional care; clinically regulate general-purpose chatbots’ diagnostic/therapeutic claims; mandate safety by design and vulnerable-user protection.

    Recommendation 3: Digital Duty of Care We support proposed Digital Duty of Care reforms. These reforms are needed to: • Ensure that it is a legal requirement for digital platforms to identify, prevent, and mitigate foreseeable harms arising from their services. • Mandate transparency that users are interacting with AI, ensuring AI-generated advice is not presented as a substitute for professional care. • Prevent unregulated (general purpose) chatbots from making diagnostic or therapeutic claims unless subject to appropriate clinical regulation. • Mandate safety by design and protect vulnerable users.

  14. Support a multidisciplinary expert group to advise government on AI for mental health.

    Recommendation 4: Support a multidisciplinary expert group to advise government on AI for mental health

  15. Enable the sector to speak to government with a coordinated voice, with strategic investment to support sector collaboration around priority issues.

    Enable the sector to speak to government with a coordinated voice: Strategic investment is required to support sector collaboration around priority issues.

  16. Fund and support a National AI in Mental Health Literacy campaign.

    Funding and support for a National AI in Mental Health Literacy campaign would position Australia as a global leader in developing the evidence, education, and safeguards needed to harness AI's potential while protecting mental wellbeing at population scale.

  17. Support the development of national digital and AI mental health literacy resources to equip people with the skills to use AI safely and effectively.

    Support the development of national digital and AI mental health literacy resources to equip people with the skills to use AI safely and effectively.

  18. Design and deliver accessible information about how AI works, its benefits and limitations for mental health, and the skills needed to identify safe tools and critically evaluate AI information.

    Design and deliver accessible information about how AI works; its benefits and limitations related to mental health, and the skills needed to identify safe tools, and critically evaluate information from AI models.

  19. Develop AI literacy resources for priority populations including young people and First Nations peoples.

    Develop AI literacy resources for priority populations including young people and First Nations peoples.

  20. Task the AI Safety Institute with monitoring harms and establishing reporting and accountability requirements as its remit expands to personal safety.

    As Australians increasingly use AI for mental health support, the Australian AI Safety Institute's remit should extend beyond national security to encompass personal safety. This expanded role should include developing safety standards, establishing reporting and accountability requirements, monitoring harms, and holding AI platforms accountable for risks to users.

  21. Fund and empower the AI Safety Institute to protect individuals, including mental-health users: set safety standards, investigate harms, require incident reporting and hold AI providers accountable for risks to vulnerable users.

    The Australian AI Safety Institute should have responsibility for protecting individuals from AI-related harms, including those arising from the use of AI for mental health support. This expanded remit should be supported by dedicated funding and powers to set safety standards, investigate harms, require incident reporting, and hold AI providers accountable for risks to vulnerable users.

Read the original

Open in the interactive guide