Submission 108
Dr Rob Nicholls
Submitter group: Individuals.
Position labels are not shown for individual submitters.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Addresses term of reference (i) only and expresses no view on AI training on creative content. Argues existing law leaves two large gaps and proposes a single statutory duty of care for AI developers, deployers and compute operators, plus mandatory registration of large-scale AI compute operators, in the planned 2027 legislation. Recommends no new regulator and no SOCI Act change.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Argues that two large gaps in existing AI law require new statutory architecture rather than incremental clarification.
The two large gaps cannot be closed by the incremental clarification of existing law that the National AI Plan contemplates. They require new statutory architecture.
Argues the online-safety duty is narrower than AI requires because it covers online services rather than the systems, processes and corporate decisions of AI businesses.
It is a duty of care for online safety, not a duty of care for wellbeing, and it attaches to online services, not to the systems, processes and corporate decisions of the businesses that build and deploy AI.
Argues that existing law addresses AI harms in fragments, reaching only some systems and harms and generally relying on enforcement after harm.
The conclusion is that existing law addresses AI harms in fragments, each fragment being a law designed for a different purpose, applied to AI by analogy, reaching some AI systems and some harms and not others, and enforced in most cases by a person who has already suffered harm.
Argues that a compute register with nominated Australian entities provides an enforcement route against otherwise unreachable offshore developers.
A register of compute operators, with a nominated Australian entity for each, provides the point of enforcement. A developer that is in breach of its duty and beyond the reach of Australian process can be reached through the compute on which its model is served to Australians.
Says a new AI regulator would defer effective regulation for years, given existing expertise and the cost and delay of establishing an institution.
I do not recommend a new AI regulator. Australia has regulators with relevant expertise, and the cost and delay of establishing a new institution would defer effective regulation by years.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Find that existing laws apply to AI in fragments but impose no general duty on developers, deployers or compute operators to take reasonable steps against serious AI harm, warranting reform.
Recommendation 1. That the Committee find that Australia's existing laws and regulatory frameworks, while they apply to AI in fragments, do not impose on any developer, deployer or compute operator a duty to take reasonable steps to prevent serious harm from AI, and that the absence of such a duty is a gap that warrants reform.
Enact the Social Media Committee’s overarching wellbeing duty, extended to the AI supply chain, in the planned 2027 Australian Standards for AI legislation.
Recommendation 2. That the Australian Standards for AI legislation to be introduced in 2027 enact the duty of care recommended by the Joint Select Committee on Social Media and Australian Society, extended from digital platforms to the AI supply chain.
Require reasonable steps, so far as reasonably practicable, to make systems, processes and corporate decisions within each entity’s control safe for Australians against serious individual or collective wellbeing harm.
The content of the duty should follow the Joint Select Committee's formulation, with the primary duty in work health and safety law as the drafting model. Each entity that owes the duty must take reasonable steps, so far as is reasonably practicable, to ensure that the systems and processes within its control, and the corporate decisions that shape them, are safe for Australians and do not cause serious harm to their wellbeing, individually or collectively.
Apply the duty to model developers, deployers affecting people in Australia through business or public functions, and operators of large-scale Australian AI compute.
The duty should be owed by three classes of entity. They are developers, being entities that train, substantially fine-tune or otherwise create AI models. They are deployers, being entities that use an AI system in the course of a business or undertaking, or in the exercise of a public function, in a way that affects persons in Australia. And they are compute operators, being entities that operate large-scale AI compute in Australia, which I define in Part B.
Limit duties to matters within each actor’s control: developers’ model properties, deployers’ use context and compute operators’ provision and security.
Each class owes the duty in respect of the matters within its control. A developer owes the duty in respect of the properties of the model. A deployer owes it in respect of the context of use. A compute operator owes it in respect of the provision of compute and the security of what runs on it.
Apply the duty regardless of domicile where models, systems or compute affect people in Australia or are located in Australia.
The duty should apply regardless of the domicile of the entity, provided that the model, system or compute affects persons in Australia or is located in Australia.
Specify reasonable steps in principle in the Act and in detail in standards, including documented pre-deployment risk assessments and reassessments at intervals no longer than twelve months.
The reasonable steps should be specified, at the level of principle in the Act and in detail in standards made under it, to include: (a) conducting and documenting, before deployment and at intervals of not more than twelve months, a diligent assessment of the risks to wellbeing that the model, system or compute, and the business model within which it is offered, may create or contribute to;
Require mitigation plans with reasonable measures for each identified risk and review of their effectiveness.
implementing a risk mitigation plan containing measures that are reasonable in the circumstances to mitigate each identified risk, and reviewing the effectiveness of those measures;
Require safety by design when designing or modifying models, systems or compute, addressing risks before release.
applying safety by design principles to the design and modification of the model, system or compute, so that risks are addressed before release rather than after harm;
Require systems to identify, respond to and report serious-harm incidents and near misses.
maintaining systems for the identification of, response to and reporting of incidents in which serious harm has occurred or has nearly occurred;
Require developers and compute operators to secure model weights, training data and compute against unauthorised access.
maintaining the security of model weights, training data and compute against unauthorised access, in the case of developers and compute operators;
Require developers to provide deployers information reasonably necessary to discharge their own duties.
providing to deployers, in the case of developers, the information reasonably necessary for the deployer to discharge its own duty;
For deployers deciding about people or mediating what they see, require transparent complaints, human review and appeal, understandable AI mediation and, where practicable, the ability to alter or decline it.
providing, in the case of deployers whose systems make or inform decisions about persons or mediate what persons see, a transparent complaints mechanism with human review and a right of appeal, and a meaningful ability for the person to understand and, where practicable, to alter or decline the AI mediation;
Require entities above documentary thresholds to give independent researchers and public-interest organisations effective mandatory data access and submit to regulator audit.
providing effective, mandatory data access to independent researchers and public interest organisations, for entities above the documentary thresholds, and submitting to audit by the regulator;
Require regulatory cooperation, including reasonably required access to systems, records and personnel.
cooperating with the regulator in the exercise of its functions, including by providing access to systems, records and personnel as reasonably required.
Avoid limiting AI researcher access to the Online Safety Act model requiring Australian-university employment, university ethics approval and research confined to that Act.
An approved researcher must be employed by an Australian university, and the research must be approved by the university's own human research ethics committee and confined to matters dealt with by that Act. The AI duty's researcher access obligation should not be read down to that footing, but the scheme is worth citing as evidence that Parliament is prepared to legislate the mechanism, and the university employment gate is a precedent to avoid rather than to follow.
Define wellbeing to cover physical and mental health, financial position, autonomy, freedom from manipulation and unlawful discrimination, children’s interests and collective infrastructure, health, social, democratic and national-security wellbeing.
Wellbeing should be defined to include the physical and mental health of persons, their financial position, their autonomy and freedom from manipulation, their freedom from unlawful discrimination, the particular interests of children, and the collective wellbeing of Australians in the security and resilience of critical infrastructure, public health, social cohesion, the integrity of democratic processes and national security.
Expressly include meaningful AI assistance for chemical, biological, radiological or nuclear weapons, computer-system attacks, and developer or deployer loss of control.
The last category should expressly include the provision by an AI system of meaningful assistance to a person in the development of a chemical, biological, radiological or nuclear weapon or in an attack on a computer system, and the loss of the ability of a developer or deployer to control a system.
Limit enforcement to failures causing or creating a material risk of serious harm, while requiring risk assessment across the full wellbeing definition.
The enforcement of the duty should be directed to serious harm. The definition of wellbeing sets what the risk assessment must address. A threshold of seriousness sets what the regulator may act upon, so that the duty is not a general-purpose complaints mechanism. The two operate together. A developer must assess and mitigate risks to wellbeing across the definition. A regulator may take enforcement action where the failure to do so has caused, or creates a material risk of, serious harm.
Coordinate the duties: deem digital-duty compliance for an online service and online-safety harm to satisfy the general duty; reach wider wellbeing harms and leave no AI system or serious wellbeing harm uncovered.
Recommendation 3. That the duty be designed alongside the digital duty of care being developed under the Online Safety Act 2021 (Cth), so that compliance with the digital duty of care in respect of an online service and an online safety harm is deemed to satisfy the general duty, so that the wider wellbeing harms that the Joint Select Committee identified are reached by the general duty, and so that no AI system and no serious harm to wellbeing falls outside any duty.
Apply documentary risk-assessment, publication and reporting duties only above thresholds set in the Act and adjustable by rules; all entities retain the underlying duty.
The duty should apply in tiers. All developers, deployers and compute operators should owe the duty. The documentary obligations of risk assessment, publication and reporting should apply only to entities above thresholds set in the Act and adjustable by rules.
Set deployment documentary thresholds by scale and contexts affecting access to employment, credit, insurance, housing, health care, education, essential services or public-authority decisions.
threshold is a combination of scale and context, with the documentary obligations applying to deployment in the contexts that the 2024 guardrails consultation identified as high-risk, being those that affect a person's access to employment, credit, insurance, housing, health care, education, essential services or a decision by a public authority.
Use installed capacity for compute-operator thresholds; below thresholds, retain the duty, omit documentary obligations and guide enforcement through published guidance.
For compute operators, the threshold is installed capacity, which I address in Part B. Below the thresholds, the duty applies but the documentary obligations do not, and the regulator's enforcement should be guided by published guidance. This is the model of the digital duty of care, and it is proportionate.
Use a training-compute measure as the developer threshold for documentary obligations.
For developers, the appropriate threshold is a measure of training compute
Enforce through guidance, monitoring, directions, undertakings and civil penalties, with registration suspension or cancellation for compute operators.
The duty should be enforced through the full pyramid. That pyramid runs from guidance and compliance monitoring, through formal directions and enforceable undertakings, to civil penalties and, for compute operators, suspension or cancellation of registration.
Align civil penalties with the Competition and Consumer Act: the greater of $100 million, three times the benefit obtained, or 30% of adjusted turnover during the breach period.
Civil penalties should be aligned with those in the Competition and Consumer Act 2010 (Cth), being the greater of $100 million, three times the benefit obtained, or 30 per cent of adjusted turnover during the breach period, so that they are meaningful for the largest entities.
Make principal-entity senior managers personally accountable for systemic failures attributable to decisions under their authority, using the financial-accountability model.
At the top tier, senior managers of the principal entity should be personally accountable for systemic failures attributable to decisions made under their authority, on the model of the financial accountability regime.
Use a regulator-enforced civil duty without a private action, while allowing courts to treat statutory breaches as evidence of breach of an existing common-law duty.
The duty should be a civil duty enforced by the regulator, not a private right of action, though nothing should prevent a court from treating a breach of the statutory duty as evidence of breach of a common law duty where one exists.
Provide a safe harbour for harm not reasonably identifiable in a standards-compliant risk assessment, where identified mitigations were implemented and required incidents reported.
There should be a safe harbour. An entity that has conducted its risk assessment in accordance with the standards, has implemented the mitigations that the assessment identified, and has reported incidents as required, should be taken to have complied with the duty in respect of a harm that its assessment could not reasonably have identified.
Use existing lead regulators, a statutory Digital Platform Regulators Forum coordination mandate and an AI Safety Institute advisory function; establish no new regulator.
Recommendation 4. That the duty be administered under a lead regulator model, with the Digital Platform Regulators Forum given a statutory coordination mandate and the Australian AI Safety Institute given a statutory advisory function, and that no new regulator be established.
Do not administer the duty through the Office of AI or AI Safety Institute, given their coordinating and evaluation roles and lack of statutory regulatory independence.
Nor do I recommend that the duty be administered by the Office of AI, which is a coordinating body within the Department of the Prime Minister and Cabinet, or by the Australian AI Safety Institute, which is an evaluation body within the Department of Industry, Science and Resources. Neither is an independent statutory regulator, and independence from government matters when the government is itself a large deployer of AI.
Assign eSafety online services, ACCC consumer-facing deployers, and ACMA compute registration and operator duties.
The eSafety Commissioner would administer the duty in respect of online services. The ACCC would administer it in respect of deployers in consumer-facing contexts, consistent with its consumer protection mandate and with Education Act, as amended and effective 1 January 2027, adopts the same threshold and imposes additional obligations on frontier developers with annual revenue above US$500 million. the breadth of the definition of goods in the Australian Consumer Law. The Australian Communications and Media Authority (the ACMA) would administer the register of compute operators and the duty as it applies to them, for reasons that I give in Part B.
Assign sectoral regulators, including APRA, ASIC and TGA, duties for deployers within their sectors.
Sectoral regulators, including the Australian Prudential Regulation Authority, the Australian Securities and Investments Commission and the Therapeutic Goods Administration, would administer the duty as it applies to deployers within their sectors.
Give the AI Safety Institute a statutory technical advisory role on models, thresholds and standards, and allow regulators to commission evaluations.
The Australian AI Safety Institute should have a statutory function to advise regulators on the technical evaluation of models, on thresholds and on standards, and regulators should be able to commission evaluations from it.
Find that the SOCI Act adequately regulates data centre facility security where its thresholds are met, should not regulate the safety of what is computed in them, and needs no amendment.
Recommendation 5. That the Committee find that the SOCI Act adequately regulates the security and resilience of data centre facilities where its thresholds are met, that it does not and should not regulate the safety of what is computed in those facilities, and that it does not require amendment.
Retain SOCI escalation only for ASIO-assessed security prejudice where Ministerial-power preconditions, including inability of another regulatory system to address the risk, are met.
The SOCI Act remains the top of the escalation pathway. Where the operation of compute is assessed by ASIO to be prejudicial to security in the defined sense, and the preconditions of the Ministerial direction powers are met, including the requirement that no other regulatory system could address the risk, those powers are available.
Establish an ACMA-maintained Register of AI Compute Operators, the instrument for any requirement to make compute available to Australian users, with above-threshold operation by unregistered operators prohibited.
Recommendation 6. That the Australian Standards for AI legislation establish a Register of AI Compute Operators, maintained by the ACMA, that the register be the instrument through which any requirement on providers of large-scale compute to make compute available to Australian users is imposed, that operation of AI compute above a threshold of installed capacity in Australia be prohibited otherwise than by a registered operator,
Provide civil penalties for unregistered operation above threshold and regulator access to urgent Federal Court injunctions.
Register as a precondition of lawful operation, with unregistered operation above the threshold prohibited and subject to civil penalties, and with the regulator empowered to seek urgent injunctive relief in the Federal Court.
Require a nominated Australian entity with domestic service address, capacity to sue and be sued, and authority to receive directions and process; permit service on the regulator if none is maintained.
Nominate an Australian entity with a domestic address for service, capable of suing and being sued in Australian courts in relation to the operator's Australian compute, with authority to receive regulatory directions and legal process on the operator's behalf. Where no nominated entity is maintained, process should be capable of service on the regulator, on the model of section 601CX of the Corporations Act 2001 (Cth).
Designate the group entity actually controlling compute allocation as principal entity, jointly and severally responsible with the nominated Australian entity.
Designate a principal entity within the operator's corporate group, being the entity with actual authority over the allocation of the compute, which bears responsibility jointly and severally with the nominated Australian entity for compliance.
Require descriptions of compute location, installed capacity and customer classes, updated annually and on material change.
Provide a description of the registered compute, including its location, its installed capacity, and the classes of customer to which it is provided, updated annually and on material change.
Require customer identification for each customer receiving compute above a threshold.
Maintain a customer identification program, so that the operator knows the identity of each customer to which it provides compute above a threshold,
Require training-run commencement and completion reports above a rules-set threshold initially aligned with 10^25 or 10^26 operations, including developer identity.
Report to the regulator the commencement and completion of any training run on the registered compute that exceeds a compute threshold set by rules, initially aligned with the international thresholds of 10^25 or 10^26 operations, together with the identity of the developer.
Require reasonable offence-prevention steps for Commonwealth, state and territory laws and reasonably necessary assistance to their authorities for criminal enforcement and national security.
Take reasonable steps to prevent the registered compute from being used in the commission of an offence against a law of the Commonwealth, a state or a territory, and give officers and authorities of the Commonwealth, the states and the territories such assistance as is reasonably necessary for enforcing the criminal law and safeguarding national security.
Require rules-based security of model weights and training data against unauthorised access, exfiltration and tampering, with incident reporting.
Maintain the security of model weights and training data held on the registered compute against unauthorised access, exfiltration and tampering, to a standard set by rules, and report incidents.
Require suspension or cessation of compute for customers in serious duty breach whom regulators cannot otherwise reach, subject to written reasons, a hearing except in urgent cases, and merits review.
Comply with a direction from the regulator to suspend or cease the provision of compute to a specified customer where the regulator is satisfied that the customer is in serious breach of its own duty of care and cannot otherwise be reached. This is the intermediary enforcement mechanism, and it should be subject to the safeguards of written reasons, a right to be heard except in urgent cases, and merits review.
Use registration to impose and monitor any chosen favourable-compute-access condition for Australian start-ups, innovative small businesses, researchers and not-for-profits; the submission takes no view on that condition’s content.
Comply with any condition of registration made under the rules that gives effect to Expectation 5, being the provision of access to compute for Australian start- ups, innovative small businesses, researchers and not-for-profits on favourable terms. I express no view in this submission on the content of that condition, which is a matter for terms of reference (e) and (f). I note only that the register is the instrument through which it can be imposed and monitored,
Do not require compute operators to assess the safety of customer models.
Registration would not require the operator to assess the safety of its customers' models.
Limit operators’ duty to compute recipients, terms and security, leaving developers their own duties.
The operator's duty of care under Part A is a duty in respect of the matters within its control, being who it provides compute to, on what terms, and how securely. The duty of the developer is the developer's own.
Use registration on objective criteria to remove grant discretion and political intervention in permission to operate compute.
Second, licensing in Australian practice tends to carry discretion in the grant, with fitness and propriety tests and ministerial involvement. Discretion in the grant of a licence to operate compute would be an invitation to political intervention in an industry in which Australia is competing for investment. Registration on satisfaction of objective criteria removes that discretion.
If Parliament prefers licensing, adopt the same design without material change; do not adopt neither registration nor licensing.
If Parliament prefers licensing, the design in this Part can be adopted without material change. The register becomes a licence register and the conditions become licence conditions. What Parliament should not do is adopt neither.
Set registration and training-reporting thresholds by rules on AI Safety Institute advice.
Recommendation 7. That the thresholds for registration and for training run reporting be set in rules on the advice of the Australian AI Safety Institute,
Indicatively measure registration thresholds by aggregate Australian accelerator capacity, using FLOPS and an initial megawatt proxy; consider around ten megawatts of accelerator load, with smaller enterprise and edge facilities excluded.
My indicative view is that the threshold should be set by reference to the aggregate accelerator capacity under an operator's control in Australia, expressed in floating-point operations per second, with a proxy in megawatts of information technology load for the purposes of initial identification. A threshold in the order of ten megawatts of accelerator load, which is well below the scale of any hyperscale facility, would capture every hyperscale and specialist AI cloud facility in Australia and would exclude enterprise data centres, edge facilities and the research computing facilities operated by universities and by the National Computational Infrastructure and the Pawsey Supercomputing Research Centre.
Register research and public compute on a light-touch tier: nominated Australian entity, capacity description, customer identification and section-313-style duties, excluding financial-sufficiency and principal-entity requirements.
Research and public compute should be treated differently from commercial compute, but should not be excluded. The National Computational Infrastructure and Pawsey operate compute of a scale that is capable of training models of concern, and they provide it to a large and diverse research community. They should be registered, on a light-touch tier that requires the nominated Australian entity, the description of capacity, the customer identification program and the section 313-style obligation, but not the financial sufficiency and principal entity requirements.
Exclude Australian Government compute, including Defence and intelligence agencies, given security obligations under other arrangements.
The Australian Government's own compute, including that of the Department of Defence and the intelligence agencies, should be excluded, on the same basis that the SOCI Act excludes assets that are already subject to security obligations under other arrangements.
Prevent refusal of compute to identified customers on identification grounds, protect regulator information against disclosure and non-administrative uses, and require program guidance within transition.
The Act should provide that the customer identification program is not a ground for refusing compute to an identified customer, that information provided to the regulator is protected from disclosure and from use for any purpose other than the administration of the Act, and that the regulator must publish guidance on the operation of the program within the transition period.
Require ACMA and the Cyber and Infrastructure Security Centre to share information about registered operators also responsible under SOCI.
The ACMA and the Cyber and Infrastructure Security Centre should be required to share information about registered operators that are also responsible entities under the SOCI Act.
Give the Office of AI a coordinating role in designing rules.
The Office of AI should have a coordinating role in the design of the rules.
Allow duty-of-care regulators to request ACMA compute-suspension directions.
The regulators that administer the duty of care under Part A should be able to request the ACMA to issue a direction to a compute operator under paragraph (i) above.