AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 117

Royal Australian and New Zealand College of Radiologists

Submitter group: Health.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

The radiology and radiation oncology college describes AI opportunities in imaging and cancer care, saying many applications are yet to be widely clinically validated and the benefit is conditional on tools performing as intended. It sets conditions: risk-proportionate, nationally consistent regulation, stronger pre- and post-market obligations, local validation of offshore tools, Indigenous data sovereignty, workforce investment.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. States RANZCR's position, held since 2022, that current regulatory mechanisms have not kept pace with AI and may no longer be fit for purpose.

    RANZCR's position, held since 2022 and unchanged, is that current regulatory mechanisms have not evolved alongside recent breakthroughs in AI technology and may no longer be fit for purpose.

  2. Argues that AI reliably absorbing routine and repetitive work is the health system’s most substantial AI productivity opportunity.

    AI that reliably absorbs routine and repetitive work is one of the few available means of addressing that gap, and RANZCR considers this the most substantial productivity opportunity that AI offers the health system.

  3. Argues that requiring offshore-trained tools to be validated on Australian data is first a patient safety measure and also contributes to sovereign capability.

    A requirement to validate offshore-trained tools on Australian populations is first a patient safety measure. It also creates a standing domestic requirement that only Australian data can satisfy, and so contributes directly to the sovereign capability the Committee is examining.

  4. Argues that clinicians must interpret clinical appropriateness, weigh bias and exercise judgement for human oversight to be effective.

    the health care professional be capable of interpreting the clinical appropriateness of a result, weighing the potential for bias and exercising clinical judgement regarding the findings. Where a clinician does not have that capability, human oversight of an AI tool becomes a formality, and the safeguard on which much of the regulatory framework depends ceases to operate.

  5. States that the level of autonomy granted to an AI device should relate to the complexity and risk of the task.

    the level of autonomy granted to a device should relate to the degree of complexity and risk involved in the task being undertaken.

  6. Says multiple layers of health regulation and governance create a risk of gaps, duplication and inconsistent requirements as AI use expands.

    Australia's health system carries multiple layers of regulation, governance and professional responsibility, including therapeutic goods regulation, jurisdictional health service policy, professional regulation of practitioners, accreditation standards and local clinical governance. As AI use expands, this creates a risk of gaps, duplication and inconsistent requirements.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Make AI regulation in healthcare risk-proportionate and nationally consistent, with the level of evidence for safety and efficacy commensurate with risk, designed to complement rather than duplicate existing regulatory systems.

    That regulation of AI in healthcare be risk-proportionate and nationally consistent, with the level of evidence required to demonstrate safety and efficacy commensurate with the level of risk, and designed to complement existing regulatory systems rather than duplicate them.

  2. Require AI labelling of training/testing populations, intended clinical context, limitations, contraindicated contexts and supervision; require software version numbers and regulator clarity on updates and relabelling after significant changes.

    That AI systems are labelled with a description of their training and testing populations, the clinical context in which they are intended to be used, their limitations, the contexts in which they should not be used, and the level of supervision required. All software should carry a version number, and regulatory authorities should provide clarity on acceptable mechanisms for updating AI systems and the labelling requirements that follow significant changes.

  3. Require manufacturers’ ongoing monitoring advice, defined responsibilities for clinical users and manufacturers, and an assistance contact.

    That manufacturers provide advice about ongoing monitoring, with clearly defined monitoring responsibilities for both clinical users and manufacturers and a contact point for assistance.

  4. Require manufacturer failure advice covering taking systems offline, continuity arrangements and clinical investigation; prevent failures affecting broader health IT systems.

    That manufacturers provide advice on how to address system failures, including whether a system should be taken offline, what continuity arrangements apply while it is offline, and how a clinical investigation should proceed. AI tools must also operate in such a way that a failure does not affect the broader IT systems of the health environment in which they are deployed, including electronic medical records and radiology information and picture archiving systems.

  5. Validate offshore-trained and tested AI tools on Australian datasets before clinical deployment, with explicit Aboriginal and Torres Strait Islander data sovereignty and governance.

    That AI tools trained and tested offshore be validated on Australian datasets before clinical deployment, with explicit provision for Aboriginal and Torres Strait Islander data sovereignty and governance.

  6. Involve clinicians who will use an AI tool and retain professional responsibility for care in procurement and deployment decisions in publicly funded health services.

    That clinicians who will use an AI tool, and who retain professional responsibility for patient care, be involved in decisions to procure and deploy it in publicly funded health services.

  7. Treat investment in clinical workforce capability, including AI literacy in medical training and post-fellowship upskilling, as a necessary component of AI adoption in health rather than assumed.

    That investment in clinical workforce capability, including AI literacy in medical training and post-fellowship upskilling, be treated as a necessary component of AI adoption in health rather than assumed.

  8. Consider health imaging for domestic domain-specific AI models.

    Where the Commonwealth is seeking areas of sectoral expertise in which domestic data could support domain-specific models of genuine value, health imaging warrants close consideration.

  9. Prevent transfer of patient data outside the care environment without consent, ethics approval or legal authority; observe revocation and dataset removal rights.

    Patient data must not be transferred out of the clinical environment in which care is provided without patient consent, approval from an ethics committee, or where required or permitted by law. The rights of the patient, including the right to revoke consent and be removed from a dataset, must be observed.

  10. Require pre-procurement disclosure of training dataset characteristics and testing environment, including under-represented populations.

    The characteristics of the training dataset and the environment in which it was gathered and tested must be clearly stated prior to procurement, including information on which populations are under-represented.

  11. Understand who controls data entered in public generative AI and whether permission is needed even for de-identified data before clinical use.

    Data entered into publicly available generative AI tools may be used to train those tools. The question of who subsequently controls that data, and whether permission is required to provide it even where it is considered de-identified, needs to be understood before such tools are used in clinical settings.

  12. Restrict AI to clinically justified tasks enhancing care; exclude inequitable bias, non-healthcare commercial record mining, and systems bypassing healthcare professionals, including direct-to-consumer AI that may contradict treating clinicians.

    That AI systems are applied only to clinically justified tasks that enhance patient care. Inappropriate uses include systems that demonstrate undesirable biases leading to inequity, systems not required for the provision of healthcare services such as the mining of patient records for commercial purposes, and systems that bypass healthcare services and circumvent the role of medical professionals, including direct to consumer AI that may contradict the recommendations of the clinicians providing care.

  13. Require appropriate safety evidence for the intended population and clinical context; explicitly show any system altering clinical outcomes is safe in use, with outcomes equal or superior to current practice.

    That AI systems are proven to an appropriate standard of evidence and deemed safe for the population and clinical context in which they are intended to be applied. Any system that can alter the clinical outcomes of patients should be explicitly shown to be safe in its clinical context of use, with patient outcomes equal or superior to current clinical practice.

  14. Governments, regulators, clinicians, consumers and healthcare organisations should work together to develop practical, evidence-based requirements for clear, nationally consistent and risk-based governance of AI in healthcare.

    RANZCR supports clear, nationally consistent and risk-based governance of AI in healthcare. Governments, regulators, clinicians, consumers and healthcare organisations should work together to develop requirements that are practical, evidence-based and responsive to clinical realities.

  15. Keep patient-care responsibility principally with clinicians while sharing it with deploying management and manufacturers; reflect that distribution in procurement.

    Responsibility for decisions about patient care rests principally with the health care professional, but the College's position is that it should be shared with the management which took the decision to deploy the tool and with the manufacturer which developed it. Procurement practice should reflect that distribution of responsibility.

  16. Relate AI device autonomy to task complexity and risk.

    the level of autonomy granted to a device should relate to the degree of complexity and risk involved in the task being undertaken.

  17. Provide human oversight or review from referral through diagnosis to treatment; require interoperability, robust ongoing monitoring and rapid disablement on failure.

    Along the path from referral to diagnosis and to treatment, human oversight or review should take place. Implementation also requires interoperability with existing systems, ongoing robust monitoring, and processes to ensure an AI tool can be rapidly disabled in the event of failure.

Read the original

Open in the interactive guide