Submission 240
Dr Alexander Saeri
Submitter group: Individuals.
Position labels are not shown for individual submitters.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Dr Alexander Saeri, in a personal capacity, describes expert and public survey evidence on AI risk. The author recommends that government measure how much AI risk Australians will accept and assess current risk against it, and apply mandatory standards to frontier AI developers, while noting developer standards alone are not sufficient.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Says surveyed experts judged 18 of 24 AI risk categories each at least 10% likely to cause catastrophe between 2025 and 2030 under business as usual.
Experts judged each of 18 of the 24 categories of AI risk as having at least a 10% chance of catastrophic outcomes between 2025 and 2030 under a ‘business as usual’ scenario.
Says experts judged that mitigations roughly halved catastrophe probabilities for the highest-rated risks but five categories stayed above 10%.
Pragmatic, cost-effective mitigations roughly halved the probability of catastrophe for the highest-rated risks (dangerous capabilities fell from 22% to 12%). Even so, five categories stayed above a 10% chance of catastrophic outcomes: dangerous capabilities, weapons and cyberattacks, environmental harm, inequality and unemployment, and power centralisation.
Argues that acceptable AI risk is a question of values that should be set through democratically accountable processes, not by the companies creating the risk.
This matters because how much risk a society should carry from a new technology is a question of values. That level should not be set by the companies creating the risk. It should be set through democratically accountable processes, informed by public values and by technical evidence.
Says its survey found 94% of Australian adults wanted advanced AI safety standards at least as strict as aviation, and 58% wanted stricter standards.
94% of Australian adults want advanced AI held to safety standards at least as strict as commercial aviation, and 58% want them stricter.
Says surveyed experts judged frontier developers and governance actors most responsible for addressing the most severe AI risks.
Experts judged general-purpose AI developers (the companies that build frontier models) and governance actors (governments, regulators and standards bodies) most responsible for addressing the most severe risks.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Find that catastrophic AI risk spans many categories; recognise dangerous capabilities, weapons and cyberattacks, power centralisation and competitive dynamics alongside the harms already named in the terms of reference.
The Committee should find that catastrophic risk from AI can emerge from many categories The Committee should find that catastrophic risk from AI can emerge from many categories, not just a few scenarios. Term of reference (k) names fraud, scams, deepfakes and child safety, each of which is causing real and growing harm today and needs urgent attention. At the same time, experts assessed 18 of 24 risk categories as carrying at least a 10% chance of catastrophic outcomes by 2030, and the categories they rate highest are different: dangerous capabilities, weapons and cyberattacks, power centralisation, and competitive dynamics. The Committee’s report should recognise these categories alongside the harms already named in its terms of reference.
Measure how much AI risk Australians will accept and for what benefits, and use the answer to review standards.
Alongside setting standards, the Government should measure how much AI risk Australians will accept, and for what benefits, and use the answer to review those standards.
Assess current AI risks against public tolerance and publish the result.
Current AI risks should then be assessed against that tolerance and the result published.
Apply mandatory AI standards to the companies that build powerful frontier AI systems provided to the Australian market.
Mandatory AI standards should apply to the companies that build powerful ‘frontier’ AI systems that are provided to the Australian market.
Consider adapting the EU General-Purpose AI Code of Practice for use in Australia.
The EU’s General-Purpose AI Code of Practice sets out how providers can comply and could be readily adapted in Australia.
Make standards and risk tolerance cover all catastrophic risks, using the MIT Repository’s 24 categories and EU Code systemic risks as starting points.
Both the standards and the risk tolerance in Recommendation 1 should cover the full range of catastrophic risk, not only the harms that are most visible today. Two useful starting points are the MIT AI Risk Repository’s 24 risk categories, which were the basis of the expert Delphi study, and the systemic risks that the EU General-Purpose AI Code of Practice requires providers to assess: chemical, biological, radiological and nuclear weapons, loss of control, cyber offence, and harmful manipulation.
Complement developer standards with competition, labour and sector-specific policy for risks such as power centralisation and unemployment.
Developer standards are necessary but not sufficient. Risks such as power centralisation and unemployment arise from market structure as much as from any single model, and will also need competition, labour and sector-specific policy.