AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Build independent Australian capability to test and evaluate AI systems

12 submissions. Each submission counts once for this shared ask. Counts do not measure importance or merit.

A shared ask groups requests for substantially the same action from three or more submissions when the group was created. Groups remain listed if withdrawals reduce their counts. AI models summarised recommendations, grouped them and checked the groups. The label summarises the requests; it is not a quote. A shared ask does not mean agreement on anything else.

Theme: AI safety testing and evaluation

Recommendations from submissions

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Extracts are chosen by an AI model. Submissions are listed in submission number order; recommendations follow their order on each submission’s page.

  1. Shop, Distributive and Allied Employees' Association (SDA) Submission 17

    1. Establish a sovereign, standing technical body for adversarial testing, continuous verification and training-data auditing of AI used in defence and critical infrastructure.

      A sovereign, standing technical body should be established to conduct adversarial testing, continuous verification of model behaviour, and training-data auditing for AI systems used in defence and critical infrastructure, so that Australia is not required to trust foreign-governed systems it cannot independently verify.

  2. Minderoo Foundation Submission 134

    1. Build national AI testing and evaluation capability.

      Build national AI testing and evaluation capability

  3. NCC Group Submission 185

    1. Retain sufficient domestic technical capability to independently validate international findings and assess AI systems deployed in Australian government, defence and critical infrastructure.

      Australia must retain sufficient domestic technical capability to independently validate international findings and assess systems deployed in Australian government, defence and critical infrastructure.

  4. AQ Intelligence (AQI), Guardrail Technologies Australia (GTA) and First Nations Strategic Alliance (FNSA) Submission 202

    1. Treat sovereign AI-assurance and AI-security capability as national infrastructure, extending sovereignty beyond compute and data centres to the guardrail, monitoring, assurance and security layer, onshore data handling and an AI-governance skills base.

      Treat sovereign AI-assurance and AI-security capability as national infrastructure. Extend the National AI Plan’s sovereignty logic beyond compute and data centres to the guardrail, monitoring, assurance and security layer, secure onshore data handling, and a home-grown AI-governance skills base.

  5. GovTech Australia Submission 258

    1. Support AI Safety Institute testing and evaluation with enduring technical capability and evaluation environments, including when foreign models or infrastructure are constrained.

      The Australian AI Safety Institute also has a role in strategic resilience. Its ability to test and evaluate emerging systems should be supported by enduring access to sufficient technical capability and evaluation environments, including during periods when access to foreign models or infrastructure is constrained.[8]

  6. Monash University Submission 277

    1. Treat national AI evaluation capability as strategic infrastructure, with the AI Safety Institute drawing systematically on multidisciplinary Australian research expertise for continuous testing, horizon scanning and emerging-risk assessment.

      treating national AI evaluation capability as strategic infrastructure, with the Australian AI Safety Institute drawing systematically on multidisciplinary Australian research expertise for continuous testing, horizon scanning and emerging-risk assessment

  7. Centre for AI, Trust and Governance at the University of Sydney Submission 286

    1. Develop sovereign evaluation capability to independently produce, obtain, interrogate and contest evidence about AI systems Australia depends on but did not build.

      This submission supports the call by independent researcher Dr. Rebecca Johnson for Australia to develop a "sovereign evaluation capability". This refers to the capacity to independently produce, obtain, interrogate and contest evidence about AI systems Australia depends on but did not build.

  8. Ms Tahsin Alamgir Kheya Submission 294

    1. Invest in independent government technical capability to measure and evaluate AI performance, fairness and risks; says the AI Safety Institute is well placed, subject to resourcing, expertise, independence and regulatory access.

      The Australian Government should invest in and maintain independent technical capability for measuring and evaluating AI system performance, fairness, and associated risks, rather than relying predominantly on developer-led self-assessment. The Australian AI Safety Institute is well placed to provide such capability, subject to appropriate resourcing, expertise, independence, and regulatory access.

  9. Commonwealth Bank of Australia Submission 316

    1. Evolve the Australian AI Safety Institute's capability to independently evaluate advanced models, identify emerging risks, learn from incidents and provide evidence to regulators and industry.

      The Australian AI Safety Institute should evolve its capability to independently evaluate advanced models, identify emerging risks, learn from incidents and provide evidence to regulators and industry.

      A program corrected obvious text-recognition errors in this quote for display.

  10. Name withheld Submission 393

    1. I support the establishment of the Australian AI Safety Institute and the development of independent Australian expertise capable of evaluating advanced AI systems, understanding emerging risks and working with international AI safety organisations.

  11. Mr Joel Leslie Submission 417

    1. Establish an AI evaluation and operations capability program, including a skills framework, training pathways, shared test environments and a regulator capability plan.

      6. National AI Establish an AI evaluation and Skills framework, training Government and regulators can evaluation and operations capability program. pathways, shared test independently assess, operate and institutional capability environments and regulator challenge AI systems. (g), (h), (k), (m) capability plan.

  12. Name withheld Submission 429

    1. Australia should protect good-faith AI safety research and evaluation from liability, and should fund independent held-out evaluation capacity.

Open in the interactive guide