AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 107

Dr Rebecca L. Johnson

Submitter group: Individuals.

Position labels are not shown for individual submitters.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

Argues Australia needs sovereign evaluation capability: the capacity to independently produce, obtain, interrogate and contest evidence about deployed and agentic AI systems. Recommends evidentiary requirements, risk-proportionate observability requirements, material-change triggers for re-evaluation, and independent expertise coordinated through the Office of AI, subject to privacy and confidentiality constraints.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Argues that governing AI requires valid evidence about deployed systems, a problem agentic systems make harder because the evaluated object changes across configurations.

    There is one cross-cutting problem underlying several Terms of Reference: you cannot govern what you cannot validly know. Agentic systems make that problem harder because the evaluated object changes across configurations, the relevant evidence may lie across trajectories, and that evidence can be distributed across institutions.

  2. Argues that evaluation should address the deployed configuration, not only the model, including prompts, tools, permissions, memory and institutional conditions.

    My research uses the term deployed configuration for this larger evaluative object: the model together with the prompts, tools, permissions, memory, accessible data, interfaces, surrounding software, human roles and institutional conditions that materially shape its behaviour.

  3. Says current Australian guidance recognises monitoring, record-keeping and reassessment after change, but leaves open what evidence an evaluator needs to make valid claims about deployed systems.

    Current Australian guidance recognises monitoring, record-keeping and reassessment after system change [8]. What it leaves open is evidentiary validity: what an evaluator must be able to observe and reconstruct before a governance claim about a deployed AI system is warranted, how far that claim travels across configurations, and when system change makes earlier evidence insufficient.

  4. Argues that a strategic vulnerability arises when dependence on foreign-developed AI is accompanied by dependence on their developers for the evidence used to assess those systems.

    The strategic vulnerability arises when dependence on foreign-developed systems is accompanied by dependence on their developers for the evidence used to assess those systems.

  5. Argues that whether a system change is material depends on whether it could plausibly alter the behaviour on which the earlier evaluation depended.

    A material change should be understood in relation to the claim being made. A new tool may be irrelevant to one evaluated property and decisive for another. The governance question is whether the change could plausibly alter the behaviour on which the earlier judgement depended.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Have the Office of AI coordinate whole-of-government evidentiary requirements for deployed AI governance claims, drawing on the AISI, sector regulators, security agencies and independent evaluation experts.

    The Office of AI should coordinate the development of whole-of-government requirements for evidence used to support governance-relevant claims about deployed AI systems, drawing on the AISI, sector regulators, security agencies and independent evaluation experts.

  2. Require evaluation claims to identify the tested configuration, testing conditions, material limitations and scope of applicability.

    Evaluation claims should identify the configuration tested, relevant conditions of testing, material limitations, and the scope within which the evidence remains applicable.

  3. For agentic systems, examine relevant actions and trajectories where a final output alone does not capture the behaviour assessed.

    For agentic systems, evaluation should examine relevant actions and trajectories where a final output alone does not capture the behaviour under assessment.

  4. Have the Office of AI coordinate risk-proportionate minimum evidence requirements enabling authorised investigators and evaluators to reconstruct consequential AI behaviour.

    The Office of AI should coordinate the development of risk-proportionate minimum evidentiary requirements sufficient for authorised investigators and evaluators to reconstruct consequential AI behaviour.

  5. Consider records of configuration history, instructions, tool actions, permissions, memory changes, external actions and human interventions; constrain access and retention for privacy, cybersecurity and legitimate commercial confidentiality without disabling regulatory investigation.

    The Office of AI should coordinate the development of risk-proportionate minimum evidentiary requirements sufficient for authorised investigators and evaluators to reconstruct consequential AI behaviour. These may include relevant configuration history, system instructions, tool actions, permission states, state or memory changes, consequential external actions and human interventions. Privacy, cybersecurity and legitimate commercial confidentiality should constrain access and retention appropriately. They should not leave regulators unable to establish what occurred.

  6. Have the Office of AI, regulators and technical evaluation bodies define review triggers for model, prompt, tool, permission, memory, data-access or workflow changes that could materially affect the previously evaluated property.

    The Office of AI should coordinate, with relevant regulators and technical evaluation bodies, clear material-change triggers for reviewing whether previous evaluation claims remain valid. Relevant changes may include a different underlying model, substantial prompt or system-instruction changes, new tools, expanded permissions, new memory functions, altered data access or significant changes to the institutional workflow in which the system operates. The trigger should turn on whether the change could materially affect the property previously evaluated.

  7. Have the Office of AI coordinate pathways for the AISI, sector regulators and relevant security agencies to obtain, share and independently interrogate evidence needed to investigate emerging AI risks.

    The Office of AI should coordinate clear pathways through which the AISI, sector regulators and relevant security agencies can obtain, share and independently interrogate evidence needed to investigate emerging AI risks.

  8. Support joint evaluation/investigation where evidence is distributed across developers, deployers, infrastructure providers or regulatory jurisdictions.

    These pathways should support joint evaluation and investigation where relevant evidence is distributed across developers, deployers, infrastructure providers or regulatory jurisdictions.

  9. Provide standing independent expertise in evaluation design, validity, agentic behaviour and sociotechnical context through whole-of-government coordination.

    Whole-of-government coordination should include standing access to independent expertise capable of assessing evaluation design, validity, agentic behaviour and sociotechnical context.

Read the original

Open in the interactive guide