AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 229

Samantha Connor AM

Submitter group: Civil society.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

Argues dependent service users face unaccountable automated decisions and profiling. Reviews NDIS, aged-care and child-risk systems, data linkage, care-worker management, provider consolidation and outcomes markets. Seeks enforceable human decision/review rights, transparency, disability data sovereignty, AISI oversight and a conditional two-year pause in highest-risk uses. Supports accessibility AI and beneficial disability research.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Argues that consent offers little protection to people who depend on government services and cannot decline to provide information.

    Consent is the foundation of most Australian privacy law. For the people described in this submission, consent is a formality signed under duress.

  2. Says OAIC cautions that statutory authority does not establish actual automated decision-making use, and that agencies’ silence does not prove concealment.

    The OAIC is explicit that "[a]uthority to use ADM does not mean that ADM is used by the agency", and the review was deliberately limited: "We did not seek information from any of the reviewed agencies to confirm if they, or agencies that they administer, were using ADM to make decisions" (OAIC7). Silence is not proof of concealment.

  3. Says the UTS Human Technology Institute told a Senate committee that the NDIS Act amendment's authorisation of automated discretionary decisions is, to its knowledge, unprecedented in Australian law.

    The Human Technology Institute at the University of Technology Sydney put it bluntly to the Senate Community Affairs Legislation Committee on 29 May 2026: "Explicit legislative authorisation for automated systems to apply to discretionary decisions is, to our knowledge, unprecedented in Australian law."

  4. Argues that a nominal human in the loop who routinely accepts, or is forbidden from departing from, machine recommendations does not amount to oversight.

    this Committee should not accept the assurance that a nominal human in the loop is sufficient. A human who routinely accepts what the machine recommends, or who is forbidden from departing from it, is not oversight.

  5. Describes BOCSAR's warning that the aggregate performance of the model may be misleading as a justification for its use.

    BOCSAR's own warning is the most important sentence in the report: "These results are a form of model bias and demonstrate that aggregate performance of the model may be misleading as a justification for its use."

  6. Argues that sovereign AI capability is about legal control, comprehension, auditability and exit rather than where servers are located.

    Sovereign AI capability is therefore not a hosting question. Putting a server in Sydney changes nothing about who controls the model, who can compel disclosure of the data under foreign law, who understands the code, and whether Australia can walk away.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Guarantee meaningful human determination before automated/AI-assisted refusal, termination, suspension or material reduction of essential government payments, disability supports, aged-care entitlements, housing services or comparable benefits.

    Establish an enforceable right to meaningful human determination before an automated or AI-assisted system can refuse, terminate, suspend or materially reduce an essential government payment, disability support, aged care entitlement, housing service or comparable benefit.

  2. Legally define meaningful human determination, excluding data checks, routine acceptance and barred overrides; restore aged-care human override consistently with the Senate’s 2 July 2026 Bill.

    Define meaningful human determination in legislation. A person who checks data entry, who routinely accepts a computer recommendation, or who is barred from departing from a machine output is not exercising human oversight. Restore human override in aged care needs assessments, consistent with the Bill passed by the Senate on 2 July 2026.

  3. Prohibit fully automated adverse decisions involving NDIS access, reductions to essential supports, social security sanctions, aged care eligibility or funding, child removal and comparable high-risk decisions.

    Prohibit fully automated adverse decisions involving NDIS access, reductions to essential supports, social security sanctions, aged care eligibility or funding, child removal and comparable high-risk decisions.

  4. Prohibit secret algorithms in merits review, so affected people and decision-makers can examine the rules, inputs, outputs and reasoning; commercial confidentiality must not defeat procedural fairness.

    Prohibit secret algorithms in merits review. Where an automated system influenced a decision before the Administrative Review Tribunal or a court, the affected person and the decision-maker must be able to examine the relevant rules, inputs, outputs and reasoning. Commercial confidentiality must not defeat procedural fairness.

  5. Require prospective parliamentary scrutiny whenever legislation permits a Minister to add new categories of automated government decisions by legislative instrument, including under section 59C(2) of the National Disability Insurance Scheme Act 2013.

    Require prospective parliamentary scrutiny whenever legislation permits a Minister to add new categories of automated government decisions by legislative instrument, including under section 59C(2) of the National Disability Insurance Scheme Act 2013.

  6. Guarantee a human and non-digital pathway to every essential Commonwealth service.

    Guarantee a human and non-digital pathway to every essential Commonwealth service.

  7. Register all decision-making/materially influencing Commonwealth AI/automation publicly: purpose, data sources, vendor, contract value, affected decisions, machine-learning use, error rates, overrides, review rights and offshore processing.

    Create a public Commonwealth Algorithm Register listing every automated or AI system used to make or materially influence decisions about people, including purpose, data sources, vendor, contract value, decisions affected, whether machine learning is used, error rates, human override arrangements, review rights and offshore processing.

  8. Require a published human rights and algorithmic impact assessment before any high-risk government AI system is deployed, modelled on Article 27 of the EU AI Act.

    Require a published human rights and algorithmic impact assessment before any high-risk government AI system is deployed, modelled on Article 27 of the EU AI Act.

  9. Require specific disability, First Nations, children's rights, aged care and anti- discrimination assessments where a system affects those populations, prepared with those communities.

    Require specific disability, First Nations, children's rights, aged care and anti- discrimination assessments where a system affects those populations, prepared with those communities.

  10. Give people the right to know when AI or automated processing materially influenced a decision about them, what information was used, which factors were significant, and what was inferred about them.

    Give people the right to know when AI or automated processing materially influenced a decision about them, what information was used, which factors were significant, and what was inferred about them.

  11. Require high-risk models to publish false positive and false negative rates, including differences between population groups, rather than overall accuracy.

    Require high-risk models to publish false positive and false negative rates, including differences between relevant population groups, rather than relying on overall accuracy figures. BOCSAR's own finding that "aggregate performance of the model may be misleading as a justification for its use" should be the standard the Commonwealth adopts.

  12. Require expiry dates and independent revalidation for high-risk predictive models.

    Require expiry dates and independent revalidation for high-risk predictive models.

  13. Complete and legislate the whole-of-government automated decision-making framework recommended by the Robodebt Royal Commission at recommendation 17.1, including publication of business rules and algorithms to enable independent expert scrutiny.

    Complete and legislate the whole-of-government automated decision-making framework recommended by the Robodebt Royal Commission at recommendation 17.1, including publication of business rules and algorithms to enable independent expert scrutiny.

  14. Give people a statutory right to see, correct and challenge consequential derived information, including fraud scores, risk classifications, predicted behaviour and other algorithmic profiles.

    Give people a statutory right to see, correct and challenge consequential derived information, including fraud scores, risk classifications, predicted behaviour and other algorithmic profiles.

  15. Prohibit complaints, appeals, FOI requests, advocates, parliamentary contact, provider changes or challenges to government decisions becoming adverse fraud, compliance or behavioural-risk variables.

    Prohibit the use of self-advocacy as a risk variable. Making a complaint, lodging an appeal or an FOI request, engaging an advocate, contacting a parliamentarian, changing providers or challenging a government decision must never be used as an adverse fraud, compliance or behavioural-risk variable.

  16. Introduce strict purpose limitation. Information collected to provide disability, health, welfare, education or aged care services must not automatically become policing, fraud, intelligence, commercial pricing or predictive risk data.

    Introduce strict purpose limitation. Information collected to provide disability, health, welfare, education or aged care services must not automatically become policing, fraud, intelligence, commercial pricing or predictive risk data.

  17. Require data minimisation and enforceable retention periods, and require Services Australia to publish all data-matching protocols and to report data-matching activity to Parliament.

    Require data minimisation and enforceable retention periods, and require Services Australia to publish all data-matching protocols and to report data-matching activity to Parliament.

  18. Establish a disabled-governed Disability Data Sovereignty Framework across NDDA and government disability data, covering service/provider records, sensors, biometrics, profiles, AI training, inter-agency/law-enforcement sharing, overseas processing, research/commercial reuse and funding/risk/pricing/future-cost calculations.

    A national Disability Data Sovereignty Framework should be developed and governed by disabled people, and should apply to the National Disability Data Asset and to government administrative data about disability wherever else it sits. It should cover: • data collected through the NDIS, Medicare, Centrelink, hospitals, schools, housing, justice and child protection; • data produced by NDIS and aged care providers; • home sensor and behavioural monitoring information; • biometrics; • risk scores and other derived profiles; • use of disability data in AI training; • data sharing between agencies, including integrity and law enforcement taskforces; • overseas processing; • research reuse and commercial access; and • use of disability information to calculate funding, risk, prices or expected future cost.

  19. Legislate NDDA Charter prohibitions on compliance identification, individual funding adjustment and automated enforcement across NDIS, Services Australia, aged care and other linked administrative systems.

    Extend the National Disability Data Asset Charter protections across government. The prohibitions on identifying individuals for compliance purposes, on adjusting an individual's funding and on use in automated enforcement tools should apply to the NDIS, Services Australia, the aged care system and other linked administrative systems, and should be placed in statute rather than policy.

  20. Strengthen NDDA governance so that disabled people hold defined decision-making authority over high-risk additions, new uses and any change to the Charter. An advisory seat is not authority.

    Strengthen NDDA governance so that disabled people hold defined decision-making authority over high-risk additions, new uses and any change to the Charter. An advisory seat is not authority.

  21. Require disability-community approval for NDDA use in new AI, predictive analytics, training or commercial research, and before adding early-childhood, school, housing, justice or transport domains.

    Require a disability community approval process before National Disability Data Asset information may be used for new categories of AI, predictive analytics, model training or commercial research, and before any of the five planned domains, being early childhood development, school education, housing, justice and transport, is added.

  22. Prohibit the sale of disability administrative data and restrict commercialisation of models trained substantially on data generated through essential public disability services.

    Prohibit the sale of disability administrative data and restrict commercialisation of models trained substantially on data generated through essential public disability services.

  23. Develop separate First Nations disability data sovereignty arrangements with First Peoples Disability Network and First Nations communities, consistent with Indigenous Data Sovereignty and CARE Principles, rather than extending the disability framework.

    Develop separate First Nations disability data sovereignty arrangements in partnership with First Peoples Disability Network and First Nations communities, consistent with Indigenous Data Sovereignty and the CARE Principles, and not by extending a disability framework over them.

  24. Examine the Tāngata Whaikaha Māori Data Sovereignty Framework as an operational precedent, including community-trust IP, data kaitiaki, local storage, consent for linkage, overseas-transfer limits and no commercial sale.

    Examine the Tāngata Whaikaha Māori Data Sovereignty Framework developed through Te Ao Mārama as an operational precedent, including vesting of intellectual property in a community trust, the data kaitiaki role, local storage, consent for linkage, restrictions on overseas transfer and the prohibition on commercial sale.

  25. Adopt a national commitment equivalent to New Zealand's, making equity and data sovereignty an explicit priority of health and disability data policy, extending to groups with the poorest outcomes.

    Adopt a national commitment equivalent to New Zealand's, making equity and data sovereignty an explicit priority of Australian health and disability data policy, extending to disabled people and other groups with the poorest outcomes.

  26. Give disabled people meaningful access to information about what government disability datasets contain, how they are linked, and the categories and variables used to define and classify disability.

    Give disabled people meaningful access to information about what government disability datasets contain, how they are linked, and the categories and variables used to define and classify disability.

  27. Establish special rules for monitoring technology in people's homes. Refusal to accept continuous monitoring must not result in reduced funding, a higher risk score or any other adverse inference.

    Establish special rules for monitoring technology in people's homes. Refusal to accept continuous monitoring must not result in reduced funding, a higher risk score or any other adverse inference.

  28. Prohibit the use of behavioural sensor data to set or reduce an individual's funded supports without the person's free and specific consent, given without penalty for refusal.

    Prohibit the use of behavioural sensor data to set or reduce an individual's funded supports without the person's free and specific consent, given without penalty for refusal.

  29. Establish biometric privacy rules for faces, voices, fingerprints and comparable identifiers, presuming against mass surveillance absent express statutory authority, with necessity/proportionality tests consistent with OAIC’s July 2026 guidance.

    Create a strong biometric privacy regime covering facial recognition, voice recognition, fingerprints and comparable identifiers, with a presumption against mass biometric surveillance absent express statutory authority, and with necessity and proportionality tests consistent with the OAIC's July 2026 guidance.

  30. Treat data sovereignty as control rather than geography. Procurement must assess ownership, governing law, foreign access, model control, auditability, vendor dependence and Australia's practical ability to replace the supplier.

    Treat data sovereignty as control rather than geography. Procurement must assess ownership, governing law, foreign access, model control, auditability, vendor dependence and Australia's practical ability to replace the supplier.

  31. Require Commonwealth procurement contracts to guarantee government, parliamentary, regulator and judicial access to model documentation, audit logs, testing material, subcontracting arrangements and data location information.

    Require Commonwealth procurement contracts to guarantee government, parliamentary, regulator and judicial access to model documentation, audit logs, testing material, subcontracting arrangements and data location information.

  32. Independently review Fraud Fusion Taskforce data fusion, including NDIS flows into ACIC/Palantir environments; publish findings on legal authority, flows, access, retention and derived information.

    Conduct an independent review of cross-agency data fusion through the Fraud Fusion Taskforce, including the movement of NDIS information into Australian Criminal Intelligence Commission systems and any Palantir environment, with public findings on legal authority, data flows, access, retention and derived information.

  33. Require that IP in models trained on data from essential public services be held by or licensed to the Commonwealth on terms permitting publication, audit and termination, with no third-party licence without a published assessment.

    Require that intellectual property in models trained on data generated through essential public services be held by or licensed to the Commonwealth on terms that permit publication, audit and termination, and that no such model be licensed to a third party without a published assessment.

  34. Give the Australian AI Safety Institute statutory public sector oversight powers, including powers to audit systems, compel vendor documentation, inspect training and testing material, investigate incidents and publish findings.

    Give the Australian AI Safety Institute statutory public sector oversight powers, including powers to audit systems, compel vendor documentation, inspect training and testing material, investigate incidents and publish findings.

  35. Require Commonwealth agencies to notify the Privacy Commissioner and the Australian AI Safety Institute before deploying high-risk government AI, with a mandatory pre-deployment period.

    Require Commonwealth agencies to notify the Privacy Commissioner and the Australian AI Safety Institute before deploying high-risk government AI, with a mandatory pre-deployment period.

  36. Resource the OAIC and the Commonwealth Ombudsman to examine automated decision-making proactively rather than after harm, and require the OAIC's automated decision-making review to be repeated annually with agency responses compelled.

    Resource the OAIC and the Commonwealth Ombudsman to examine automated decision-making proactively rather than after harm, and require the OAIC's automated decision-making review to be repeated annually with agency responses compelled.

  37. Regulate algorithmic management of workers where automated systems determine or influence rosters, performance ratings, disciplinary action, workload or care task allocation, with specific provision for disability and aged care work.

    Regulate algorithmic management of workers where automated systems determine or influence rosters, performance ratings, disciplinary action, workload or care task allocation, with specific provision for disability and aged care work.

  38. Set enforceable trial standards for essential-service-dependent participants: public pre-registration, independent ethics review/evaluation, secondary-use rules, published IP arrangements and no penalties for declining.

    Establish enforceable standards for technology trials involving people dependent on essential government services, including public pre-registration, independent ethics review, clear secondary-use rules, published intellectual property arrangements, independent evaluation and a prohibition on penalising people who decline to take part.

  39. Investigate whether AI, reporting, cyber security and data integration requirements are increasing market concentration and driving private equity roll-ups or sales of small and family providers.

    Investigate whether AI, digital reporting, cyber security and data integration requirements are increasing market concentration and contributing to private equity roll-ups or the sale of small and family businesses in disability, aged care and other government-funded services.

  40. Require transparency in outcomes-based commissioning and social impact investment, including publication of the data and modelling used to set outcomes payments, calculate avoided government costs, select cohorts, define counterfactuals and determine investor returns.

    Require transparency in outcomes-based commissioning and social impact investment, including publication of the data and modelling used to set outcomes payments, calculate avoided government costs, select cohorts, define counterfactuals and determine investor returns.

  41. Invest in Australian accessibility AI as public infrastructure, including Easy English, speech and communication tools, screen reader and image description technologies, and narrow Australian models capable of explaining Australian law and services accurately.

    Invest in Australian accessibility AI as public infrastructure, including Easy English, speech and communication tools, screen reader and image description technologies, and narrow Australian models capable of explaining Australian law and services accurately.

  42. Require government Easy English and other AI-generated accessibility material to be tested with the people who use it, including people with intellectual disability, before publication.

    Require government Easy English and other AI-generated accessibility material to be tested with the people who use it, including people with intellectual disability, before publication.

  43. Require disabled people and other affected communities to hold decision-making positions in AI governance, procurement, ethics and evaluation, rather than being consulted after systems are purchased.

    Require disabled people and other affected communities to hold decision-making positions in AI governance, procurement, ethics and evaluation, rather than being consulted after systems are purchased.

  44. Assess Commonwealth AI/data policy against CRPD obligations on autonomy, equality, privacy, independent living, information access and participation, treating Article 31 as a floor.

    Require Commonwealth AI and data policy to be assessed against Australia's obligations under the Convention on the Rights of Persons with Disabilities, including autonomy, equality, privacy, independent living, access to information and participation, and treat Article 31 as a floor rather than a ceiling.

  45. Apply a two-year moratorium until legislation establishes meaningful human determination rights, a public algorithm register, published error rates and a Disability Data Sovereignty Framework.

    Until the framework recommended in this submission exists in legislation, with an enforceable right to a human determination, a public algorithm register, published error rates and a Disability Data Sovereignty Framework, the Commonwealth should apply a two-year moratorium on:

  46. Cover five moratorium areas: fully automated adverse NDIS/social-security/aged-care decisions; Commonwealth child-risk scoring and funding of state scoring; NDIS section 59C(2) designations; new service-delivery biometrics; and new linked disability-data training, cohort-selection or outcomes-pricing uses.

    the Commonwealth should apply a two-year moratorium on: • fully automated adverse decisions that refuse, cancel, suspend or reduce NDIS supports, social security payments, aged care eligibility or aged care funding; • deployment of predictive risk scoring of children by any Commonwealth agency, and Commonwealth funding of state predictive risk scoring of children; • new designations of automated decision provisions by legislative instrument under section 59C(2) of the National Disability Insurance Scheme Act 2013; • new government facial recognition or biometric identification deployments in service delivery settings; and • new uses of linked disability administrative data for AI training, cohort selection or outcomes pricing.

  47. Adopt a general principle for high-risk government technology: encourage systems individuals use for independence, and apply a substantially higher legal threshold to institutional systems that predict, ration, police, profile or monetise dependent people.

    Adopt a general principle for high-risk government technology. Systems used by individuals to increase their independence should be encouraged. Systems used by institutions to predict, ration, police, profile or monetise people who depend on government should face a substantially higher legal threshold.

  48. Adopt the principle: nothing derived from us, linked about us, inferred about us, or used to govern us without us.

    Nothing derived from us, linked about us, inferred about us, or used to govern us without us.

  49. Apply one test to every system: whether it gives the person more power over their own life or gives the institution more power over the person.

    We would ask the Committee to apply one test to every system in front of it: whether it gives the person more power over their own life or gives the institution more power over the person.

  50. Put the burden of proving automated-system safety on deploying governments rather than affected people.

    The burden of proving that an automated system is safe should sit with the government deploying it, not with the person it harms.

  51. Ask whether Commonwealth sovereign-wealth money is invested in the fund owning Aidacare and, if so, whether the Commonwealth received any part of its $540 million dividend.

    The directory does not state what each vehicle holds, so we ask the Committee: is Commonwealth money in the sovereign wealth fund invested in the fund that owns Australia's largest NDIS equipment retailer, and if so, was any part of that $540 million dividend paid to the Commonwealth.

  52. Pre-register high-risk essential-service technology trials with data collection, prospective reuse, IP ownership, possible entitlement/pricing effects, evaluator and penalty-free refusal disclosed.

    Any high-risk technology trial involving people dependent on essential government services should be publicly registered before it starts, stating what data is collected, what later uses are contemplated, who will own the resulting intellectual property, whether the system could later influence entitlement or pricing, who evaluates it, and whether a person can decline to participate without losing support.

  53. Have the committee test against evidence whether infrastructure measuring disadvantage is being connected to infrastructure monetising it.

    The infrastructure for measuring disadvantage is being connected to the infrastructure for monetising it. That is the sentence we would ask the Committee to test against the evidence rather than accept from us.

Read the original

Open in the interactive guide