Submission 505
Mr Gareth Davies
Submitter group: Individuals.
Position labels are not shown for individual submitters.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Argues that autonomous AI needs licensed operators and named Supervisors responsible for correctibility, rather than individual outputs. The formal recommendations exempt ordinary low-risk uses; the developed scheme adds low-risk disclosure/logging and medium-risk registration, supervision and insurance. Proposes stronger high-risk duties, conditional public insurance and AI education, while stressing that institutional design cannot itself supply practitioners’ judgment.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Says eSafety imposes class-wide design obligations but does not constitute the supervisory architecture of a particular deployment.
Some reach further. eSafety's enforceable codes and standards impose design obligations across a class of service and extend to services that distribute AI models. Even there the regulator sets conditions for a class of service; it does not constitute the supervisory architecture of a particular deployment.
Argues that existing legal categories such as product liability and negligence do not fit autonomous AI, because responsibility is diffused across actors.
Product liability requires a defect; there is none. Negligence requires duty-bearers identifiable in advance, and the chain of relationships in autonomous AI deployment, from developer through deployer, integrator, supervisor and user, diffuses the duty across actors none of whom plausibly carries responsibility alone.
Proposes a named human AI Supervisor inside the entity, with personal obligations for the integrity of the supervisory architecture and not liable for individual AI outputs.
The third is the AI Supervisor, the named human officer inside the entity, carrying personal obligations for the integrity of the supervisory architecture. The Supervisor is not liable for individual AI outputs.
Proposes a duty not to guarantee that the system stays within permitted parameters, which it says cannot presently be assured, but to keep its conduct open to interrogation and intervention.
The distinction is deliberate: the duty is not to guarantee that the system stays within permitted parameters, which cannot presently be assured, but to keep its conduct open to interrogation and intervention.
Argues that obligations should be scaled by risk class and operational context rather than by technology type.
A local AI model assisting a hobbyist sits in a different regulatory frame from a model supporting clinical triage or one executing autonomous trades. Risk class, not technology type, is the right basis on which to scale.
Argues that insurance acts as a de facto regulator, pricing risk, demanding audits, requiring governance standards and refusing cover where the supervisory architecture is inadequate.
Insurers price risk, demand audits, require governance standards, and refuse to cover deployments where the supervisory architecture is inadequate. The insurance market becomes a de facto regulator, as it already functions in professional services, construction and aviation, and it absorbs new risk classes faster than doctrine or legislation does.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Find that high-risk autonomous AI deployment requires a person answerable in advance for arrangements keeping conduct monitorable, interrogable and stoppable, and that Australian law currently imposes no such requirement.
That the Committee find that the deployment of autonomous AI in high-risk settings should require a person answerable in advance for the arrangements that keep the system's conduct monitorable, interrogable and capable of being stopped, and that no Australian law imposes such a requirement at present.
Examine licensing deployments where supervisory failure could cause serious harm, through licensed entities with named personally responsible Supervisors, modelled on aviation, therapeutic-goods sponsorship and financial-services licensing; exempt ordinary and low-risk uses.
That the Committee recommend the Government examine a tiered regime in which the obligations fall only on deployments where a supervisory failure could cause serious harm, such as clinical decision-making, financial transactions with execution authority, and autonomous physical systems. In that class, the system would be deployed only through a licensed entity with a named Supervisor personally responsible for the arrangements described in Recommendation 1, on the model of the Air Operator's Certificate, the therapeutic goods sponsor and the Australian Financial Services Licence. Ordinary and low-risk uses would carry no new obligation.
Require highest-risk licensees to carry insurance and appear on a public register; examine a public insurer of last resort where commercial cover for correlated or catastrophic exposure is unavailable.
That the Committee recommend that licensees in the highest risk class carry mandatory insurance and appear on a public register, and that the Government examine whether a public insurer of last resort is required where cover for correlated or catastrophic exposure is unavailable on commercial terms.
Specify in the operating licence the permitted operational context, output standards, audit/insurance obligations, suspension/revocation conditions and the Supervisor’s operational and structural role.
The second is the Operating Licence. It specifies the operational context within which the entity is permitted to act, the standards its outputs must meet, the audit and insurance obligations it carries, the conditions of suspension and revocation, and the role of the Supervisor in operational and structural terms.
Place primary AI-conduct liability on the entity; make the Supervisor personally responsible for supervisory-architecture and governance failures, insulated from ordinary commercial direction, rather than liable for individual outputs.
The third is the AI Supervisor, the named human officer inside the entity, carrying personal obligations for the integrity of the supervisory architecture. The Supervisor is not liable for individual AI outputs. Liability attaches instead to failures in the supervisory architecture itself, including design, monitoring and escalation systems, and to governance failure such as permitting the entity to operate outside the conditions the licence requires. The role is structurally insulated from ordinary commercial direction within the entity, in the manner of an AFSL responsible manager, a TGA responsible person, or an aviation chief pilot. 16. In legal terms, the entity bears primary liability for the AI’s conduct, while the Supervisor carries responsibility for maintaining the integrity of the governance architecture.
Require the Supervisor to preserve correctibility through auditability, traceability, human override and defined escalation triggers, keeping outputs open to interrogation and intervention.
The Supervisor’s duty is to maintain the conditions under which the AI’s conduct remains correctible. The distinction is deliberate: the duty is not to guarantee that the system stays within permitted parameters, which cannot presently be assured, but to keep its conduct open to interrogation and intervention. In operational terms, this includes auditability, traceability, 4 human override mechanisms and defined escalation triggers. These conditions exist to preserve the capacity to interrogate the AI’s outputs and test them against the judgment they are meant to serve. An output that cannot be interrogated cannot be corrected before it causes harm.
Assess compliance against what a reasonably competent supervisor of comparable deployments would maintain; output failure alone does not establish breach, and unforeseeable failure despite adequate arrangements falls to insurance.
The standard is one of process, not outcome. The duty is discharged by maintaining the conditions of correctibility, judged against what a reasonably competent supervisor of comparable deployments would maintain. An output failure is not of itself evidence of breach. The question is not whether the particular failure was foreseen but whether the arrangements for detecting and correcting it were in place when it arrived. If they were, and the system failed in a way no one could have anticipated, the operator has met the standard and the loss falls to insurance.
For Tier 1 personal-hardware models and simple assistive tools, apply disclosure and logging without entity registration or a Supervisor; retain ordinary user negligence liability.
Tier 1. Local language models on personal hardware and simple assistive tools. General disclosure and logging requirements. No entity registration, no Supervisor appointment. Liability sits with the user under ordinary negligence principles, as it does for any ordinary tool.
For Tier 2 non-critical enterprise deployments, require registration, a designated Supervisor, specified human-review triggers, basic audits and professional indemnity insurance; permit external/shared or conditionally vendor-affiliated supervision.
Scope. Most enterprise AI deployments in non-critical functions. For example, an enterprise HR screening tool or an internal legal summarisation system. Safety mitigations. Mandatory entity registration, designated Supervisor (who may be external, shared or, subject to conditions of independence, supplied through a vendor-affiliated service), human review triggers for specified outputs, basic audit obligations, and professional indemnity insurance.
For Tier 3 serious-harm deployments, require full AI Operator constitution, personal Supervisor duties, benchmarking at registration/material changes of use and entity insurance with separate Supervisor cover.
Tier 3. AI in medical decision-making, financial transactions with execution authority, autonomous physical systems, and any deployment where loss of supervisory integrity could cause serious harm. Full AI Operator constitution with personal obligations on the Supervisor, mandatory capability benchmarking at registration and at material change 6 of use, and mandatory insurance carried by the entity with separate cover for the Supervisor.
Consider phased implementation: voluntary registration and disclosure, then mandatory Tier 3 licensing, with gradual extension as the supervisory profession and insurance market mature.
Implementation need not be immediate or total. A phased path is plausible: voluntary registration and disclosure first, mandatory licensing for Tier 3 deployments next, and gradual extension as the supervisory profession and the insurance market mature.
Require risk-proportionate insurance for every Tier 2 and Tier 3 AI Operator.
Every Tier 2 and Tier 3 AI Operator must carry insurance proportionate to its risk class.
If a public insurer of last resort is used, require architecture testing, conditions, pricing, withdrawal powers and recovery rights against upstream developers whose gross negligence causes loss.
A public insurer of last resort must still test the supervisory architecture, attach conditions, price what it accepts, retain the power to withdraw, and hold rights of recovery against an upstream developer whose gross negligence produced the loss.
Condition Tier 3 licensing on Supervisor coverage and insurer audit access to the supervisory architecture.
A Tier 3 licence that requires the Supervisor to be covered, and gives the insurer audit access to the supervisory architecture, is the condition on which that market forms.
Consider school-curriculum AI literacy from early childhood, including risk assessment and verifying outputs.
AI literacy from early childhood, including the assessment of risks and the practice of verifying outputs, belongs in the school curriculum the way other forms of civic literacy do.
Suspend the licence if a departing Supervisor is not replaced.
The fourth is continuity. Supervisors leave. An entity-and-licence regime ties operation to current valid supervision. Lose the Supervisor, and the entity must appoint a successor or have its licence suspended.
Attribute AI outputs to the corporation as its own acts and knowledge without treating AI as a mind or requiring a human directing mind for that attribution.
On Meridian's reasoning, a substantive rule may attribute the AI's outputs to the corporation as the corporation's own acts and knowledge, without treating the AI as a mind and without requiring any human to be the corporation's directing mind for that purpose.37 This is the attribution the regime intends: the entity answers for outputs as its own conduct,
Make the public register identify licensed operators, their Supervisors and their insurance.
A licensed AI Operator is on a register. Its counterparties, regulators, harmed parties and the public can know which entities are operating under licence, who their Supervisors are, and what insurance is in place.
Make the Supervisor’s duty prevail over commercial interests; protect against removal or reduced powers for constraining or pausing deployment.
The insulation carries a priority rule. Where the entity’s commercial interest in keeping the system running conflicts with the conditions of correctibility, the Supervisor’s duty to the architecture prevails. A direction from the entity to continue operating is no answer to a breach, and the licence must protect the Supervisor against removal, or against any narrowing of the role’s powers, for exercising the judgment to constrain or pause the system.
Require Tier 3 Supervisors to be supplier-independent: no relationship reasonably compromising judgment to constrain/pause; deem supplier employment, payment or material interest compromising.
7.3. Tier 3: High risk Scope. AI in medical decision-making, financial transactions, autonomous physical systems, and any deployment where loss of supervisory integrity could cause serious harm. Clinical triage systems and autonomous trading systems with execution authority are the paradigm cases. Safety mitigations. A full AI Operator constitution with personal statutory duty on the Supervisor. The Supervisor must be independent of the suppliers of the deployed system, in the sense auditor independence law uses where no relationship may exist that a reasonable person would conclude compromises the judgment to constrain or pause the system, and employment by, payment from, or a material interest in a supplier is deemed to do so.
Approve initial Tier 3 Supervisors individually for named deployments, drawing on comparable judgment in safety-critical engineering, clinical safety and banking model risk, with transitional recognition of experience.
The first Tier 3 Supervisors cannot be drawn from a profession that does not yet exist. They can be approved person by person for a named deployment, as CASA approves an operator’s key personnel for that operator,48 drawn from the fields in which comparable judgment is already formed, among them safety-critical engineering, clinical safety and model risk in banking, and given transitional recognition for experience in the way the financial adviser reforms gave it to experienced advisers.
Tie licensing to operational context; require periodic capability benchmarking, repeated whenever system changes could affect the basis of the previous assessment.
The licensing obligation must follow operational context, not source code. The second is behavioural drift. Models are updated, retrained or replaced, and the system running a year after registration may not be the system that was assessed. Periodic capability benchmarking, repeated whenever a change to the system could affect the basis of the previous assessment, is how that is caught.
Allow assured public, pooled or statutory coverage where correlated/catastrophic risk is assessable but private capital will not carry it and significant public benefit exists; do not back unpriceable risk.
Where instead the risk can be assessed but no private insurer will hold it, because the exposure is too correlated or too catastrophic for private capital to carry rather than because it cannot be priced at all, and the deployment carries significant public benefit, the precedents from other fields point not to prohibition but to substitution, as with terrorism reinsurance pools, statutory nuclear liability schemes, and government war-risk cover for aviation. The requirement, properly stated, is assured coverage rather than private coverage. A government acting as insurer of last resort, or an equivalent pooled or statutory assurance, can stand in, provided it keeps the discipline that gives the mechanism its force.
Use conditions for model diversity, independent fallback and capability benchmarking; expose concentration through the public register and apply heaviest conditions to highest-correlation deployments.
conditions can require model diversity, independent fallback, and capability benchmarking precisely to prevent the monoculture that makes failures correlate; the public register makes concentration visible to the regulator before it is realised as loss; and tiering keeps the highest- correlation deployments under the heaviest conditions.
Keep breach penalties uninsurable by making insurance against monetary penalties an offence, following model work health and safety law.
and the penalty for breach must be kept uninsurable. Section 199A of the Corporations Act 2001 (Cth) bars a company from indemnifying its officers against a pecuniary penalty order but does not stop the officer buying cover for it; the model Work Health and Safety Act goes further and makes it an offence to insure, or to be insured, against a 66 Ben-Shahar and Logue (n 50) 197. 39 monetary penalty, which is the rule this regime should adopt.
Limit each Supervisor’s operator caseload so the duty can be discharged for every operator.
The condition that makes this workable is capacity. The licence should limit the number of operators one Supervisor may hold, because the duty cannot be discharged for an operator the Supervisor has no time to examine.
Require supervision services to be legally separate from vendors, with fees independent of continued deployment, operator appointment and replaceability without tool replacement; retain the same overriding duty.
The service must be a legal entity separate from the vendor. Its fees must not depend on continued deployment. The Supervisor must be appointed by the operator, not the vendor, and holds the same duty, including the priority rule in section 6.2, whoever pays for the service. And the operator must be able to replace the Supervisor without replacing the tool.
Adopt co-regulation in which industry develops standards and government approves and enforces them.
A co-regulatory model is the realistic path. Industry develops the standards, government approves and enforces them,
Include independent technical expertise on the body setting co-regulatory standards.
independent technical expertise sits on the body that sets the 69 Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation) [2016] OJ L119/1, art 37(2) (a group of undertakings may designate a single data protection officer) and art 37(6) (the officer may fulfil the tasks on the basis of a service contract). 41 standards.
Require foreign Tier 3 operators serving Australia to hold a local licence, permitting their own foreign-company registration with a service agent rather than an Australian subsidiary.
A foreign operator deploying a Tier 3 system into Australia requires a licence, but may hold it in its own name as a foreign company registered to carry on business here, with a local agent for service, as foreign companies already hold AFSLs.
Give claimants direct action against the insurer when the operator cannot be reached or cannot pay.
Claimants recover from the insurance the licence requires, which means the licence must also give them a direct right of action against the insurer where the operator cannot be reached or cannot pay,
Require a fit-and-proper Supervisor, tier-proportionate insurance and correctibility architecture; grant licences to applicants meeting those conditions.
The applicant must appoint a Supervisor who meets a fit and proper person test, carry insurance proportionate to its tier, and show that the architecture of correctibility is in place including the audit trails, escalation triggers and override mechanisms that keep the AI's conduct answerable to professional judgment. An applicant who satisfies these conditions must be granted the licence, as an AFSL applicant must.
Confine licensing discretion to Supervisor fitness, exercised by the regulator with reasons and merits review; exclude refusal of qualified applicants on national-interest or industry-policy grounds.
The fit and proper person test does bring an evaluative judgment into the grant.77 The claim is not that discretion disappears but that it is confined. It goes only to the fitness of the named Supervisor, it is exercised by the regulator rather than a minister, and it is subject to reasons and merits review, which is how AFSL practice has contained it. What the regime excludes is discretion to refuse a qualified applicant on grounds of national interest or industry policy, which is the concern.
Allow the regulator to enforce the Supervisor’s duty without waiting for an identifiable beneficiary to sue.
The licence states what the Supervisor's duty requires, the regulator checks compliance 62 Breen v Williams (1996) 186 CLR 71, 113; Pilmer v Duke Group Ltd (in liq) (2001) 207 CLR 165, 197–98 [74]. 63 Bristol and West Building Society v Mothew [1998] Ch 1, 16–18. 36 through audit, directors-and-officers cover responds to governance failure, and the regulator can enforce the duty without waiting for an identifiable beneficiary to sue.