Submission 137
MDR Security
Submitter group: Companies.
An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.
Summary
Read the original submission for the full text.
Argues that AI policy should reflect technical realities, anticipate errors and outages, and retain human accountability. Prefers existing laws while seeking documented resilience and strict standards for decisions about people. Recommends comparative-advantage investment, scrutiny of foreign dependencies, critical-application consideration of narrow models, protected public-service experimentation, and integration of AI safety into cyber security, including deployment risks.
Key points
Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.
Argues that Australia should pursue comparative advantage to retain agency and autonomy while collaborating internationally, rather than seek technological self-sufficiency.
Australia does not have the scale to be self-sufficient across the whole technology stack; instead it needs to leverage the economic theory of comparative advantage to carve out a position that retains agency and autonomy while collaborating with others.
Argues that physical data location mitigates only a small subset of risks and does not deter hackers.
the physical location of electronic data storage only mitigates a small subsection of risks. Hackers do not care where a system is located.
Says narrow, domain-specific models may require more setup but can be more predictable within their domains and more efficient for critical applications.
we believe that more effort should be put into considering the trade-off against narrower, domain specific models, especially for key critical applications. They may require more effort to train and set up, but they can be more predictable within their domain and far more efficient
Argues that AI builders and operators must remain accountable under existing laws and regulations and cannot claim ignorance of the risk of mistakes.
The people and organisations that build and operate such systems must be accountable for their actions under the existing laws and regulations. Because every AI system makes mistakes (see (m) below), no operator should be able to claim ignorance of the risk.
A program corrected obvious text-recognition errors in this quote for display.
Argues that every process relying on AI must consider what happens when it makes a mistake.
Therefore, every time a process relies on AI, the process must include consideration of what happens when the AI makes a mistake.
A program corrected obvious text-recognition errors in this quote for display.
Argues that agency experimentation should be tolerated, while production systems that make or inform decisions about people must meet strict standards.
The committee should draw a clear line between experiments, where some failure is expected and should be tolerated, and production systems that make or inform decisions about people, which must meet strict standards of lawfulness, accuracy, transparency and human accountability.
Recommendations
Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.
Design AI for errors and outages; require Commonwealth agencies and critical infrastructure operators to document error detection, correction and service continuity without AI.
Every implementation of AI should be designed on the assumption that the AI will sometimes be wrong and sometimes be unavailable. Commonwealth agencies and critical infrastructure operators should be required to document how errors are detected and corrected, and how the service continues without the AI system.
A program corrected obvious text-recognition errors in this quote for display.
If used, define sovereign AI capability as assured access, trust and leverage; guide investment by comparative advantage, focusing on capabilities others depend on.
“Sovereign AI capability” has different meanings to different people. If the term cannot be avoided, it should be defined in terms of assured access, trust and leverage, not some misguided idea of complete self-sufficiency. Investment to develop sovereignty should be guided by the theory of comparative advantage, focusing on areas where Australia can deliver capability that others depend on.
A program corrected obvious text-recognition errors in this quote for display.
Convene economists, technical experts and end-users to identify the most promising areas.
The Government should convene economists, technical experts and end-users to work together to identify the most promising areas.
Assess foreign-interference risk beyond data location: system operators, foreign governments’ powers to compel data access or service withdrawal, and continuity if a foreign supplier withdraws.
Assessments of foreign interference risk should not simply focus on where data is stored, but instead consider who operates an AI system, which foreign governments can compel access to the data or withdrawal of the service, and whether the function could keep running if a foreign supplier withdrew.
A program corrected obvious text-recognition errors in this quote for display.
For critical applications, give narrow, domain-specific and locally deployable models equal consideration with large general-purpose models, weighing reliability, auditability, energy and water use and resilience, not just benchmark performance.
For critical applications, narrow, domain-specific and locally deployable models should be given equal consideration with large general-purpose models, weighing reliability, auditability, energy and water use and resilience, not just benchmark performance.
Keep accountability for AI systems with the people and organisations that build, deploy and operate them, so that 'the AI did it' is never a defence.
Accountability for AI systems should remain with the people and organisations that build, deploy and operate them. ‘The AI did it’ should never be a defence.
A program corrected obvious text-recognition errors in this quote for display.
The APS should protect well-designed experimentation, while holding AI systems that make or inform decisions about people to strict standards of lawfulness, accuracy, transparency and human accountability.
The Australian Public Service should protect genuine, well-designed experimentation to be able to realise the benefits of AI, while holding AI systems that make or inform decisions about people to strict standards of lawfulness, accuracy, transparency and human accountability.
A program corrected obvious text-recognition errors in this quote for display.
Integrate AI safety and security into cyber security frameworks; include deployment risks in the AI Safety Institute’s remit, recognising frontier models may not suit every application.
Alsafety and security should be integrated into existing cyber security frameworks and methods rather than treated as a separate discipline, and the AI Safety Institute’s remit should cover the risks of deploying AI, not only the capabilities of frontier models — especially noting that frontier models may not be the solution for every potential AI application.
A program corrected obvious text-recognition errors in this quote for display.
Design AI pilots as experiments with a clear hypothesis, advance success and stop criteria, accountable business ownership and a realistic production pathway if successful.
However, such pilots do need to be designed as experiments, with a clear hypothesis, success and stop criteria set in advance, an accountable business owner, and a realistic path to production if they work.
Consider whether AI systems suit Australia’s valuable applications, appropriate data context and reliability requirements.
Ensuring that we have AI systems that are suited to what Australia needs, ie systems suited to the applications that provide value to our economy, trained on data that is appropriate to our context, and which are reliable for the purposes we use them for.
A program corrected obvious text-recognition errors in this quote for display.
Consider assured AI access for critical processes, minimising dependence on revocable software services and offsetting foreign dependencies with Australian capabilities providing leverage.
Ensuring security of supply, ie that when we need access to AI systems we are confident we can get it, particularly where we integrate them into critical processes. This could include minimising dependence on using “software as a service” where the provider could cut off access without notice’, but also where we do have dependencies on other countries (eg supply of high end GPUs) offsetting this with our own comparative advantage in other capabilities that provide opportunities for leverage.
A program corrected obvious text-recognition errors in this quote for display.
Consider supplier trustworthiness supported by testing evidence, transparency about changes and incidents, and advance notice before models used in critical processes change or retire.
Ensuring that we have confidence in the suppliers of the systems, that they are providing products that are safe, reliable and trustworthy, backed by evidence of testing, transparency about changes and incidents, and advance notice before a model that a critical process depends on is changed or retired.
Use Australian training data where economically or practically justified, avoiding national preference that sacrifices access to the most effective systems.
However, we need to be clear that models should be trained on Australian data where this makes economic or practical sense — eg making training cheaper, or making the results more accurate in the context where they will be used. The temptation should be avoided to stray into misplaced “nationalistic pride” that a model is somehow intrinsically better because it only uses Australian data, at the cost of access to the most effective models and systems.
Identify and nurture Australian sectoral expertise for domestic use and export to gain leverage over foreign capabilities Australia depends on.
Australia should identify and nurture areas of sectoral expertise not only for domestic use but for export, establishing a critical role in AI systems that other countries rely on, which in turn gives us leverage over capabilities we rely on them for.
A program corrected obvious text-recognition errors in this quote for display.
Ensure agencies have access to independent technical expertise so they can be intelligent customers of AI suppliers rather than relying on suppliers to assess their own products.
agencies need access to independent technical expertise to be intelligent customers of AI suppliers, rather than relying on suppliers to assess their own products.
A program corrected obvious text-recognition errors in this quote for display.
Do more work on digital identity to promote its adoption and ensure its end-to-end security.
in particular more work is needed on digital identity to promote its adoption and ensure its end-to-end security
Prioritise robust digital identity and out-of-band verification of payment and access requests to reduce fraud and scam risks.
Reducing these risks depends instead on fundamental security architecture: robust digital identity’, and out-of-band verification of payment and access requests — and in particular more work is needed on digital identity to promote its adoption and ensure its end-to-end security”.
Prioritise cyber security basics: correct configuration, prompt patching, avoiding unnecessary internet connections and segmenting networks.
The answer is for organisations first to get better at the basics: configuring systems correctly, patching promptly, not connecting systems to the internet unnecessarily and segmenting networks.
Direct cyber defence investment to doing the cyber security basics faster, including finding and fixing vulnerabilities, rather than the latest agentic AI security product.
Where AI might help defenders, it will be by doing those basics faster, particularly finding and fixing vulnerabilities before attackers do. That is where investment in cyber defence should go, rather than into the latest agentic AI security product.
A program corrected obvious text-recognition errors in this quote for display.
Understand where AI input and output data goes, who can access it and how it may be used, especially with remote or software-as-a-service systems.
The data that goes into and out of AI systems is often highly sensitive and valuable. Therefore, there needs to be a clear understanding of where this data goes, who has access to it and how it might be used; this is especially true of AI systems that run remotely (eg in data centres) or are operated by software-as-a-service providers on behalf of end-users.
A program corrected obvious text-recognition errors in this quote for display.
Include cyber security teams in understanding and managing AI risks.
In the early days of AI, cyber security teams were often seen as the “department of no” and people went around them to try out the shiny new AI toys. They should now be an integral part of understanding these risks and working out how best to manage them.
A program corrected obvious text-recognition errors in this quote for display.
Take evidence from technical experts who understand and can explain the underlying technology, alongside those who study its implications.
We encourage the committee to take evidence from those who understand, and can explain, the underlying technology, alongside those who study its implications.
Read the original
In some quotes a program corrected obvious text-recognition errors for display.