Submission 422
Mr Andrew Cossen
Submitter group: Individuals.
Position labels are not shown for individual submitters.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
The author, a PauseAI volunteer writing personally, argues that losing control of advanced AI could be catastrophic. The submission asks for committee scrutiny, an independent AI Safety Institute, an international pacing treaty and compute controls. It acknowledges possible short-term benefits and uncertainty about whether a benign but unrecognisable AI future would be worse.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Says nobody knows how to guarantee that an advanced system’s goals remain compatible with human goals, because these systems are grown rather than directly programmed.
Nobody knows how to guarantee such a system’s goals stay compatible with ours. This is particularly the case given the systems are “grown, not built” – you can’t simply program in goals.
Says estimates of the level of risk vary, but argues that even the lowest demand action.
I admit, the estimates as to level of risk vary. But even the lowest demand that we take action. If an engineer told you there was a 10% chance a bridge collapses, you wouldn’t open the bridge.
Says they do not know whether a future with human experience decoupled from the world would be worse, and asks what values should be preserved.
That gets into deep philosophical questions: what values are we actually trying to preserve, what’s left of a person whose experience is decoupled from the world, what any of it is for. I am not saying that world, or the Brave New World of Aldous Huxley, is necessarily worse. I don’t know.
Argues that rules to pause or pace AI development should be negotiated, since the race dynamic among developers can be overcome with the right framework.
Negotiate the rules necessary to pause or pace AI development. At the moment everyone is racing hard: the dilemma is they all know it could end in disaster, but if it doesn’t each wants the prize. With the right framework this can be overcome.
Argues that Australia can set precedents as a middle power, citing the under-16 social media limit, tobacco plain packaging and flight recorders.
Australia legislated an under-16 social media age limit in November 2024, the first country in the world to do it, and numerous others are following suit. Before that: plain packaging on tobacco, and mandatory flight recorders, which Australia required before anyone else. Middle powers set precedents that larger powers then find politically possible.
Describes a published survey of 2,778 AI researchers in which 38% to 51% put at least a 10% chance on advanced AI leading to outcomes as bad as human extinction.
In a survey run in late 2023 and published in 2024, 2,778 published AI researchers were asked for predictions. Between 38% and 51% of them put at least a 10% chance on advanced AI leading to outcomes as bad as human extinction
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Consider seriously, before proceeding, whether people would want an unrecognisable future even if advanced AI succeeds.
Whatever the number, we arrive somewhere unrecognisable. We are currently arguing about how fast to get there without asking whether we would want to live there. That deserves serious consideration before we press on.
Negotiate international treaty rules to pause or pace AI development, overcoming the development race.
Develop an international treaty. Not unlike nuclear weapons. Negotiate the rules necessary to pause or pace AI development. At the moment everyone is racing hard: the dilemma is they all know it could end in disaster, but if it doesn’t each wants the prize. With the right framework this can be overcome.
Develop oversight mechanisms for the treaty, considering reporting and verification through advanced AI chips.
Develop the oversight mechanisms. There are already smart people working on this. Reporting and verification through advanced AI chips is looking like a solution.
Allow narrow AI to pursue targeted benefits during a pause, including drug discovery, drought-resistant crops and climate mitigations.
Pausing doesn’t mean we don’t get the benefits. We can still allow narrow AI to pursue targeted goals: drug discovery, drought-resistant crops, climate change mitigations.
State in the report that loss of human control over advanced AI could be catastrophic and a safety and national security risk.
State in your report that loss of human control over advanced AI systems could have catastrophic consequences, and is a safety and national security risk to Australia.
Hold a risk hearing with the Australian AI Safety Institute, PauseAI Australia and international experts such as Bengio and Hinton, who could appear by video.
Hold a hearing on it. Take evidence specifically on the risk — from the Australian AI Safety Institute, from PauseAI Australia, and from international experts such as Bengio and Hinton, who could be invited to appear by video.
Fund the AI Safety Institute properly, give it stronger powers, and make it an independent body.
Back the AI Safety Institute. On limited funding it punches above its weight, testing unreleased frontier models with the Australian Signals Directorate and sharing methods with the UK and Canadian institutes. Fund it properly, give it teeth, and stand it up as an independent body instead of a unit inside a department.
Pursue and help broker an international agreement on verification and pacing of frontier AI development.
Recommend that Australia actively pursue an international agreement on the verification and pacing of frontier AI development, and help broker it.
Make access to Australian compute for frontier training conditional on transparency about training and safety standards.
Recommend that access to Australian compute infrastructure for frontier-scale AI training be conditional on transparency about what is being trained and to what safety standard.
Add to the Australian Standards for AI a power to require advanced chips in large data centres to support features needed by an international agreement.
The Australian Standards for AI, announced July 2026 and due in Parliament in early 2027, is the natural vehicle: alongside its announced scope of energy, water and copyright, it should carry a power to require that advanced AI chips in large data centres support the features an international agreement would rely on.