AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 126

PauseAI Australia

Submitter group: Civil society.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

PauseAI Australia argues that frontier AI development poses a plausible extinction risk, citing reported misalignment incidents, while acknowledging uncertainty about whether superintelligence will be built. It recommends an international pause treaty, a legislated ban on superintelligence, AI standards that define safety to include extinction risk, shutdown powers, mandatory reporting and more AI Safety Institute funding.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Argues that extinction from advanced AI is the risk deserving most attention and is plausible within our lifetimes.

    The risk arising from AI which deserves the most attention is that AI will plausibly kill every Australian within our lifetimes.

  2. Argues that frontier AI development should pause until systems can be demonstrated safe, and that this requires a global treaty.

    The only responsible course is to pause frontier AI development until such systems can be demonstrated safe. The race is global, so a global treaty is required.

  3. Argues that AI companies cannot credibly self-regulate, citing their not monitoring systems, not volunteering information and not answering direct governmental questions.

    With this combination of not monitoring their systems, not volunteering information, and not answering direct governmental questions, AI companies cannot credibly self-regulate.

  4. States that announced Australian AI standards address data centre infrastructure rather than the safety of the AI systems trained in them.

    The standards are to cover power, grid connections, energy efficiency and water. They only address safety in the context of the buildings rather than the AI systems trained within them.

  5. States that regulators and the AI Safety Institute currently depend almost entirely on voluntary cooperation from AI companies to identify emerging risks.

    The ability of regulators, the Australian AI Safety Institute and the intelligence and security community to identify and respond to emerging risks currently depends almost entirely on the willingness of AI companies to cooperate.

  6. Says it supports Good Ancestors and others in seeking significantly more funding for the Australian AI Safety Institute.

    We strongly support the position of Good Ancestors and others that the Australian AISI needs significantly more funding.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Advocate for a treaty enforcing a temporary pause on training the most powerful general AI systems until we know how to build them safely and keep them under democratic control.

    That the Australian Government advocates for a treaty enforcing a temporary pause on the training of the most powerful general AI systems, until we know how to build them safely and keep them under democratic control.

  2. Legislate a prohibition on developing superintelligence, not lifted before broad scientific consensus that it will be done safely and controllably and strong public buy-in.

    That the Australian Government legislate a prohibition on the development of superintelligence, not lifted before there is 1.) broad scientific consensus that it will be done safely and controllably, and 2.) strong public buy-in.

  3. Define safety in the Australian AI Standards legislation to include catastrophic and extinction risk from frontier model training and deployment.

    That the Australian AI Standards legislation defines safety to include catastrophic and extinction risk from frontier model training and deployment.

  4. Empower a designated Commonwealth authority to direct large-scale Australian AI-compute operators to suspend specified training, deployments or workloads on defined grounds including credible loss-of-control evidence, reporting to Parliament; potentially allow emergency data-centre shutdowns.

    That the AI standards legislation provides a power for a designated Commonwealth authority to direct an operator of large-scale AI compute in Australia to suspend a specified training run, deployment or workload, on defined grounds including credible evidence of loss of control, reporting the decision to Parliament. This could include government power to, in an emergency, shut down one or more data centres.

  5. Require AI developers and large-scale compute operators to report installed compute, frontier training and safety incidents to a Commonwealth regulator, with the AI Safety Institute giving technical assessment.

    That AI developers and operators of large-scale AI compute in Australia be required to report to a Commonwealth regulator their installed compute, frontier training, and safety incidents, with Australia’s AI Safety Institute giving technical assessment.

  6. Increase funding for the Australian AI Safety Institute to at least $50 million per year.

    That the government increases the funding for the Australian AI Safety Institute to at least $50 million per year.

  7. Have the Committee and its members discuss AI extinction risk with Parliament, Government and the public.

    That this committee and its members start discussing the extinction risk from artificial intelligence with the Parliament, the Government and the public.

  8. Include a cap on frontier training until consensus that larger models are safe, verification of training inputs, and enforcement in a verifiable treaty.

    A verifiable treaty requires: 1. a cap on the scale of frontier training runs until there is consensus larger models are safe; 2. verification of training run inputs; and 3. enforcement.

    A program corrected obvious text-recognition errors in this quote for display.

  9. Do not let AI companies dictate international agreements or the pace of a slowdown.

    Note that while it is good that the AI companies themselves - as noted in the previous section - want to at least slow down and get some international agreements put in place, they should not be the ones to dictate the shape of those agreements or the pace of a slowdown.

  10. Offer to convene the US and China to negotiate a treaty pausing frontier AI development.

    While some people may downplay the role that Australia could play in this process, as a US ally and significant trade partner of China, currently the main players in artificial intelligence, Australia is perfectly placed to offer to convene them to negotiate this treaty.

  11. Have the Australian Government immediately treat AI extinction threats as a national priority.

    and the Australian government must immediately begin to treat the extinction threat from artificial intelligence systems as a national priority.

  12. Ensure legislative and regulatory arrangements and their enforcing institutions can respond as rapidly as model capabilities increase.

    One important point to note is that model capabilities continue to increase extremely rapidly. 46 47 In order to stay ahead of them, the legislative and regulatory arrangements put in place, alongside the institutions tasked to enforce them, must be able to respond just as rapidly, which is challenging.

  13. Require real oversight of AI companies, including mandatory disclosure, regular inspections, unfettered investigations of incidents and enforceable shutdown mechanisms for breaches.

    The Australian government must insist on real oversight, which should include mandatory disclosure, regular inspections, unfettered investigations of incidents and enforceable shutdown mechanisms for breaches.

  14. Put in place risk-management regulation and legislation visibly effective to the public; openly discuss extinction threats to build trust.

    The Australian government needs to put in place regulation and legislation that effectively manages the risks but, importantly, is also seen to do so by the Australian public. To build trust the government needs to be openly discussing the extinction threat, banning the creation of superintelligence and pushing for a global treaty.

  15. Work with PauseAI Australia to raise AI extinction threats with Parliament, Government and the public.

    We ask that you work with PauseAI Australia to raise the issue of the extinction threat from artificial intelligence with the Parliament, the Government and the public.

  16. Begin laying the groundwork for a treaty with China and the US immediately.

    Bringing China and the US into a treaty will take time, and time may be extremely short. We therefore urge that the groundwork begin to be laid immediately.

Read the original

In some quotes a program corrected obvious text-recognition errors for display.

Open in the interactive guide