Submission 475
Mrs Sue Cunningham
Submitter group: Individuals.
Position labels are not shown for individual submitters.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Proposes four connected actions to prepare Australia for serious AI incidents: a national process to identify priority AI risks, a national incident and near-miss reporting system including proportionate mandatory reporting of defined serious incidents, planning for the human and social consequences of incidents, and a partnership with community and human services organisations. Builds on existing crisis management arrangements.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Says Australia already has important parts of an AI safety and crisis-management system but that the connection between them is missing.
Australia already has important parts of an AI safety and crisis-management system: the Australian AI Safety Institute, regulators, the Australian Government Crisis Management Framework, the National Emergency Management Agency and trusted community-service networks. What is missing is the connection between them.
Recommends that one accountable authority lead a continuing cross-government assessment of serious AI risks and preparedness.
Assign one accountable coordinating authority to lead a continuing cross-government process for assessing the serious AI risks most relevant to Australia and coordinating preparedness against agreed scenarios.
Says existing reporting channels each identify problems within their remits but do not give a national, cross-sector view of serious AI incidents and near misses.
The channels can each identify problems within those remits, but together they do not provide a national, cross-sector view of serious AI incidents and near misses, reveal patterns between them or enable lessons to be learnt that could improve preparedness and response.
Says containing a technical failure will not by itself restore lost income, access to care, identity or trust.
Containing the technical failure will not restore lost income, re-establish access to care, correct a compromised identity or rebuild trust.
Recommends a funded national partnership, or equivalent coordination mechanism, with community and human services organisations to prepare for AI incidents.
Design, establish and fund a national AI social preparedness and response partnership - or equivalent standing coordination mechanism - with relevant not-for-profit, community-controlled and human services organisations.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Assign an accountable authority for continuing cross-government assessment of Australia’s serious AI risks and preparedness scenarios; publish a current, non-sensitive national reference point. Coordinate responsible agencies and sectors without necessarily owning every risk or action.
Assign one accountable coordinating authority to lead a continuing cross-government process for assessing the serious AI risks most relevant to Australia and coordinating preparedness against agreed scenarios. Maintain and publish a current, non-sensitive national reference point. The authority need not own every risk or action. It should bring together the agencies and sectors that do and establish:
Identify current and emerging serious or potentially catastrophic AI risks using common definitions; explain priorities using international evidence and Australian exposure, dependencies and vulnerabilities; develop scenarios covering affected systems/communities, compounding impacts and safeguards.
the serious and potentially catastrophic AI risks currently relevant to Australia, and those that could emerge next, using common working definitions; • the reasons those risks are currently prioritised, drawing on international evidence and Australian exposure, dependencies and vulnerabilities; • shared scenarios that show how a risk could arise in Australia, which systems and communities may be affected, how impacts may compound, and which plans, warning indicators and safeguards should be tested;
Record lead and supporting agencies, existing capability, remaining gaps and action underway, plus evidence, incidents or capability changes that would alter priorities, scenarios or readiness.
the lead and supporting agencies, the capability that already exists, the gaps that remain and the action underway; and • the evidence, incident or change in capability that would alter a priority, scenario or level of readiness.
Use the UK National Risk Register’s assessment function as a reference, while setting priorities through Australian conditions and institutions.
The United Kingdom's National Risk Register is the public version of its classified National Security Risk Assessment. It uses plausible scenarios to assess likelihood and impact and to identify the capabilities required for response and recovery. Its assessment process is increasingly dynamic, allowing risks to be reviewed as evidence changes.[5] Australia should use that function as a reference, while determining its priorities through Australian conditions and institutions.
Establish national AI incident, significant-hazard and near-miss reporting, investigation and learning; mandate proportionate reporting of clearly defined serious incidents, protect early hazard/near-miss disclosure, retain appropriate regulator routes, and aggregate common minimum information into cross-sector, non-sensitive findings.
Establish a national system for recognising, reporting, investigating and learning from serious AI incidents, significant hazards and near misses. Proportionate mandatory reporting should apply to clearly defined serious incidents. Protected reporting should allow hazards and near misses to be disclosed before substantial harm occurs. Existing regulators should remain the reporting route where appropriate, supported by common minimum information and a national function that aggregates signals, identifies recurring or cross-sector patterns and publishes non-sensitive findings and lessons.
Develop a proportionate domestic system able to exchange information with emerging international incident-reporting practice.
The OECD has developed a common reporting framework to help countries compare AI incidents across contexts, identify high-risk systems and evaluate effects on people and the planet. The European Union's AI Act requires providers of high-risk AI systems to report serious incidents, with reporting timeframes linked to severity.[10-11]Australia should develop a proportionate domestic system that can exchange information with this emerging international practice.
For each priority AI scenario, design and test support from first warning through long-term social recovery, potentially using the Commonwealth Crisis Management Framework and relevant recovery systems.
For each priority AI scenario identified under Recommendation 1, design and test how affected people will be identified and supported from the first warning through longer-term social recovery. The work could use the Australian Government Crisis Management Framework and relevant recovery systems as a foundation.
Identify affected people early, including those with greater exposure, fewer alternatives or compounding disadvantage; assess cumulative income, identity, health, care, safety, housing, service and information effects; allocate leadership across portfolios, jurisdictions and services.
early identification of affected people and communities, including those facing greater exposure, fewer alternatives or more difficulty recovering, and those for whom several forms of vulnerability or disadvantage may compound; • a shared view of cumulative effects on the same people across income, identity, health, care, safety, housing, essential services and trusted information; • clear leadership allocation when consequences cross portfolios, jurisdictions or service systems, so affected people are not left to navigate disconnected responses;
Provide accessible help, essential-support continuity, trusted communication and non-AI fallbacks when systems fail; plan transition to long-term recovery of service access, wellbeing, participation and trust.
accessible routes to help, continuity of essential support, trusted communication and workable non-AI alternatives where systems fail; and • a planned transition from immediate response and relief into longer-term recovery, including restoring access to services, wellbeing, participation and trust.
Test social-response arrangements through scenario exercises with the agencies and organisations expected to act, identifying unclear responsibilities, missing support and broken handovers before incidents.
The result should be tested arrangements that work from the first warning through immediate support and longer-term recovery. Testing means running scenario exercises with the agencies and organisations expected to act so that unclear responsibilities, missing support and broken handovers are found before a real incident.
Apply COVID-19 inquiry lessons before serious AI incidents: integrated social/human-rights/economic/health advice, priority populations, agreed responsibilities, escalation triggers, real-time review and post-emergency recovery.
The Commonwealth COVID-19 Response Inquiry found that a whole-of-government emergency requires integrated advice on social, human-rights, economic and broader health impacts, explicit attention to priority populations, clear pre-agreed responsibilities, escalation triggers, real-time review and post-emergency recovery.[13-14]Australia should apply those lessons before a serious AI incident rather than reconstruct the necessary arrangements during one.
Design, establish and fund a national AI social preparedness and response partnership, or equivalent standing coordination mechanism, with relevant not-for-profit, community-controlled and human services organisations.
Design, establish and fund a national AI social preparedness and response partnership - or equivalent standing coordination mechanism - with relevant not-for-profit, community-controlled and human services organisations.
Before incidents, build partners’ safety knowledge and scenario understanding; use population/dependency knowledge; strengthen continuity and non-AI fallbacks; agree escalation, practise response/recovery roles and plan mobilisation of services, people and resources.
Before a serious AI incident occurs, the partnership should build their AI safety knowledge, help them understand the priority scenarios and possible human consequences, draw on their knowledge of affected populations and service dependencies, strengthen service-continuity and non-AI fallback arrangements, agree how information and concerns will be escalated, practise defined response and recovery roles, and determine how additional services, people and resources would be mobilised.
Give the AI preparedness partnership defined membership, risk briefings, exercises, activation arrangements and sustained capability development, using the National Coordination Mechanism as a starting point.
It provides a starting point for cross-sector coordination, but an AI preparedness partnership would need defined membership, risk briefings, exercises, activation arrangements and sustained capability development.