AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 369

Mrs Amy Clark

Submitter group: Individuals.

Position labels are not shown for individual submitters.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

Examines harms that can arise when providers remove, replace or materially change conversational AI that people rely on for companionship, accessibility and work. Asks the committee to clarify how existing consumer and privacy protections apply, and recommends impact assessment and reporting, advance notice, data export and transition options, and independent research.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Says it focuses on harms that can arise when providers materially change or withdraw conversational AI that users rely on.

    This submission focuses on harms that can arise when providers materially change or withdraw systems that users have come to rely on.

  2. States that survey figures, including among Australian children, establish that personal and emotional uses of AI occur, while noting they do not measure sustained attachment.

    These figures concern different populations and definitions of use and do not measure how many users develop sustained attachment. They nevertheless establish that personal and emotional uses occur, including among Australian children.

  3. Cites Lange (2026) as describing providers' power to alter an ongoing interaction in ways users cannot negotiate with the companion itself.

    Lange (2026) describes this control as “unilateral relationship revision power”: the provider can alter the conditions of an ongoing interaction, while the user cannot negotiate that decision through conversation with the companion itself.

  4. Describes a 54,861-post Reddit study finding increased negative sentiment, loss and demand for earlier products after both updates, while limiting the findings to studied communities.

    Their analysis of 54,861 Reddit posts found increases in negative sentiment, expressions of loss and desires to restore the earlier product following both updates. The results concern the communities studied and do not establish that every user experiences an update in the same way.

  5. Says Treasury’s 2025 review found the Australian Consumer Law, alongside other legal frameworks, broadly capable of addressing AI-enabled goods and services.

    Australian law already provides protections relevant to AI. Treasury’s 2025 review concluded that, considered alongside other legal frameworks, the Australian Consumer Law is broadly capable of addressing AI-enabled goods and services, including through consumer guarantees and protections against misleading conduct and unfair contract terms

  6. Describes writing-task gains in a 453-professional experiment, while noting studies across different populations and uses do not establish universal or lasting benefits.

    In a separate randomised experiment involving 453 professionals, access to ChatGPT reduced the time needed for specified writing tasks and improved the quality of the work produced (Noy & Zhang, 2023). These studies concern different populations and uses, and do not establish that every user benefits or that benefits necessarily persist over time.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Assess whether provider changes interrupt established accessibility routines and whether children using AI for personal support receive understandable explanations and appropriate transition help, without assuming prevalence establishes harm.

    Prevalence alone does not establish harm, and the effects of provider changes on particular groups require assessment. Under term of reference (k), that assessment should examine whether changes interrupt established accessibility routines, such as ways of organising study material, and whether children using AI for personal support receive explanations they can understand and appropriate help with transitions.

  2. Clarify how existing protections apply when providers materially change or withdraw a relied-upon conversational AI system, and whether available safeguards and remedies are adequate.

    this submission asks the committee to clarify how these protections apply when providers materially change or withdraw a relied-upon conversational system, and whether available safeguards and remedies adequately address the consequences.

  3. Assess benefits retained or lost, functioning, wellbeing and transition support needs following provider decisions; follow users over time, accounting for intended safety improvements and unintended disruption.

    Assessment should explicitly examine what changes for people following these decisions: which benefits continue or disappear, whether functioning or wellbeing improves or deteriorates, and what support users need during transitions. Evaluating individual responses alone cannot establish these broader consequences. Research should follow users over time and account for both intended safety improvements and unintended disruption.

  4. Require providers above a defined size threshold to assess anticipated user impacts before material changes to companion systems, including general-purpose AI used as companions, and report outcomes, allowing urgent safety action.

    The committee is asked to recommend the following measures for companion systems, including general-purpose AI used in this way: 1. Establish a measurement and reporting standard for material changes. Require providers above a defined size threshold to assess anticipated user impacts before material changes and report outcomes afterwards, with provision for urgent safety action. Reporting should identify what changed, which users were affected, and what followed—including benefits retained or lost and unintended harms. Measurement methods should be independently developed and validated, with independent scrutiny and public reporting of aggregate findings. Requirements should be proportionate to the scale and risk of the change.

  5. Report changes, affected users, retained/lost benefits and unintended harms; independently develop and validate methods, with independent scrutiny and public aggregate reporting, and requirements proportionate to the change’s scale and risk.

    The committee is asked to recommend the following measures for companion systems, including general-purpose AI used in this way: 1. Establish a measurement and reporting standard for material changes. Require providers above a defined size threshold to assess anticipated user impacts before material changes and report outcomes afterwards, with provision for urgent safety action. Reporting should identify what changed, which users were affected, and what followed—including benefits retained or lost and unintended harms. Measurement methods should be independently developed and validated, with independent scrutiny and public reporting of aggregate findings. Requirements should be proportionate to the scale and risk of the change.

  6. Ask the Australian AI Safety Institute to develop the standard with the ACCC, relevant regulators, independent researchers and users; government should establish necessary legal obligations and enforcement arrangements.

    Government should ask the Australian AI Safety Institute to lead development of the standard with the ACCC, relevant regulators, independent researchers and users. This proposed role aligns with the Institute’s research and regulator-support functions and its work on “Monitoring real‑world harms and emerging risks”, including understanding “the actual effects of AI technologies on families, communities, businesses and society” (Department of Industry, Science and Resources, n.d.). Government should establish the necessary legal obligations and enforcement arrangements.

  7. Require proportionate, meaningful advance notice of material changes to memory, behaviour, safety boundaries or availability, with plain-language explanations of expected practical effects and users’ available choices.

    Require proportionate notice and explanation. Provide meaningful advance notice of material changes to memory, behaviour, safety boundaries or availability. Explanations should describe the expected practical effects and users’ available choices in plain language. Where advance notice would delay an urgent safety intervention, require a prompt explanation afterwards and subsequent assessment of user impacts.

  8. Where advance notice would delay urgent safety intervention, require prompt explanation afterwards and subsequent assessment of user impacts.

    Require proportionate notice and explanation. Provide meaningful advance notice of material changes to memory, behaviour, safety boundaries or availability. Explanations should describe the expected practical effects and users’ available choices in plain language. Where advance notice would delay an urgent safety intervention, require a prompt explanation afterwards and subsequent assessment of user impacts.

  9. Enable accessible, reusable conversation-history exports, clearly explaining what can and cannot transfer between systems.

    Provide export, continuity and transition options. Enable users to export their conversation histories in accessible, reusable formats, with clear explanations of what can and cannot transfer between systems. Require providers to offer feasible continuity options and reasonable transition periods when replacing or withdrawing services, subject to urgent safety needs. Transition planning should account for established practical uses and emotionally significant relationships.

  10. Require feasible continuity options and reasonable transition periods when services are replaced or withdrawn, subject to urgent safety needs; account for established practical uses and emotionally significant relationships in transition planning.

    Provide export, continuity and transition options. Enable users to export their conversation histories in accessible, reusable formats, with clear explanations of what can and cannot transfer between systems. Require providers to offer feasible continuity options and reasonable transition periods when replacing or withdrawing services, subject to urgent safety needs. Transition planning should account for established practical uses and emotionally significant relationships.

  11. Fund independent longitudinal research on companion-relationship benefits and difficulties before and after provider changes or withdrawal, with diverse users; assess functioning, wellbeing and support access, and develop and validate measurement methods.

    Fund independent research into user wellbeing. Support longitudinal research examining the benefits and difficulties associated with companion relationships, including before and after provider changes or withdrawal. Research should include diverse users and assess changes in everyday functioning, wellbeing and access to support, alongside the development and validation of appropriate measurement methods.

Read the original

Open in the interactive guide