AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 128

Nuvento

Submitter group: Companies.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

An Australian AI company that states a commercial interest in Australian AI procurement, drawing on its RAVEN analysis platform (which it describes as its own account, not an independent assurance finding), argues for Australian-controlled capability. It recommends paid government procurement trials, whole-of-system sovereignty assessments, independent domain-specific evaluation, and verifiable accountability for AI used in consequential decisions.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Argues that Australia should build the capability to develop, operate, evaluate and maintain AI systems under Australian control.

    Australia should build the capability to develop, operate, evaluate and maintain AI systems under Australian control.

  2. Recommends assessing sovereignty across the whole AI system, including data processing, administrative control and dependencies.

    Assess sovereignty across the whole AI system. Require evidence about data processing, administrative control, model and software dependencies, update arrangements and operational continuity, proportionate to the use case and its sensitivity.

  3. Argues that government adoption should start from a defined operational problem and a baseline.

    Government adoption should start with a defined operational problem and a baseline. In an analytical workflow, relevant questions include how much time staff spend locating source material, checking proposed links and correcting errors.

  4. Recommends funding shared evaluation resources and secure testing environments measuring task quality, harmful errors, human effort and operating cost.

    Fund shared evaluation resources and secure testing environments that measure task quality, harmful errors, human effort and total operating cost.

  5. Argues that AI used for consequential decisions needs a clear chain of responsibility, noting that incorrect associations or missed connections carry consequences in intelligence analysis.

    AI used to inform consequential decisions needs a clear chain of responsibility. In intelligence analysis, an incorrect association can direct attention towards the wrong person, while a missed connection can prevent relevant evidence being examined.

  6. Says agencies need to fund workforce capability as part of adoption, including analyst and technical staff training.

    Agencies also need to fund workforce capability as part of adoption. Analysts require training in source verification, uncertainty and automation bias; technical staff require skills in integration, security, evaluation and model maintenance.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Make trials time-limited and accessible to Australian suppliers, with published criteria, proportionate assurance and an agency transition budget.

    Establish paid, time-limited procurement trials with a defined decision on production adoption. Give Australian suppliers accessible opportunities to demonstrate value against published criteria, with proportionate assurance requirements and an identified agency owner and transition budget.

  2. Run paid competitive agency trials specifying owner, data, security, baseline, evaluation, decision date, lawful procurement route and implementation/support funding; participation must not guarantee production contracts, and proportionate requirements must preserve privacy, security and probity.

    Establish paid, competitive trials around bounded agency problems. Each should identify the business owner, permitted data, security conditions, baseline, evaluation method and date for the adoption decision. Tender documentation should explain the lawful route to any subsequent procurement and the funding needed for implementation and support. Trial participation should not guarantee a production contract. Requirements should be proportionate to the trial’s risk and stage, while preserving essential privacy, security and probity controls.

  3. Make background intellectual property, data ownership, interoperability and exit arrangements explicit in procurement.

    Procurement should make background intellectual property, data ownership, interoperability and exit arrangements explicit.

  4. Require whole-system sovereignty evidence about processing, administrative control, dependencies, updates and continuity, proportionate to use-case sensitivity.

    Assess sovereignty across the whole AI system. Require evidence about data processing, administrative control, model and software dependencies, update arrangements and operational continuity, proportionate to the use case and its sensitivity.

  5. Establish a sovereignty evidence schedule covering locations, processing/telemetry, access/keys, licences, updates/vulnerabilities, recovery and exit; sensitive deployments must demonstrate functions without external connectivity and identify unavailable functions.

    The government should establish a common evidence schedule for sovereignty assessments in AI procurement. It should cover data and backup locations; external processing and telemetry; privileged access and key control; software and model licences; update and vulnerability management; recovery; and export or transition arrangements. Sensitive deployments should demonstrate required functions under loss of external connectivity and identify any functions that are unavailable.

  6. Distinguish application development, model development, hosting and operational control in sovereignty assessments.

    The assessment should distinguish application development, model development, hosting and operational control.

  7. Fund independent, domain-specific evaluation resources and secure testing environments measuring task quality, harmful errors, human effort and total operating cost, with governed access to Australian data.

    Support independent, domain-specific evaluation and governed access to Australian data. Fund shared evaluation resources and secure testing environments that measure task quality, harmful errors, human effort and total operating cost.

  8. Set evaluation collaboration and data-access arrangements with agencies, research organisations and industry: synthetic then authorised representative data, independent tests separate from training/tuning, and purpose, privacy, consent, copyright, retention and relevant Indigenous cultural governance.

    Support shared evaluation datasets and secure testing facilities through collaboration between agencies, research organisations and industry. Use synthetic data for early development, then appropriately authorised representative data to test operational relevance. Keep independent test material separate from training and tuning data. Access arrangements should address purpose, privacy, consent where required, copyright, retention and cultural governance, including Indigenous data where relevant.

  9. Require effective privacy and security controls against internal misuse for locally hosted systems.

    Locally hosted systems also require effective privacy and security controls against misuse within the organisation.

  10. Support domestic integration and evaluation expertise alongside research into models where a demonstrated Australian need justifies it.

    Investment should support domestic integration and evaluation expertise alongside research into models where a demonstrated Australian need justifies it.

  11. Agencies procuring AI for consequential use should specify and verify provenance retention: sources, processing, model versions and material configurations, with controlled access and retention periods sufficient for investigating outputs.

    Agencies procuring AI for consequential use should specify and verify: • Evidence and provenance: retain sufficient information about sources, processing steps, model versions and material configurations to investigate an output, with controlled access and retention periods.

  12. Reassess material changes to models, data sources and workflows, and assign a responsible owner for deciding whether continued use is justified.

    Change control: reassess material changes to models, data sources and workflows. Assign a responsible owner for deciding whether continued use is justified.

  13. Test access restrictions, disclosure risks and hostile source material, and maintain a process to report incidents, restrict affected functions and retest after fixes.

    Security and misuse testing: test access restrictions, disclosure risks and hostile source material. Maintain a process to report incidents, restrict affected functions and retest after fixes.

  14. Provide routes to correct source records and erroneous associations, assess affected downstream outputs, and support review or challenge consistent with applicable law and operational constraints.

    Correction and challenge: provide routes to correct source records and erroneous associations, assess affected downstream outputs, and support review or challenge consistent with applicable law and operational constraints.

  15. Give authorised staff the evidence, time, training and authority to reject an AI output, and do not make a model score or generated allegation the sole basis for adverse action.

    Meaningful human review: give authorised staff the evidence, time, training and authority to reject an output. A model score or generated allegation should not be the sole basis for adverse action.

  16. Describe purpose, limitations and oversight publicly without personal or operationally sensitive details; provide detailed records to authorised reviewers and supplier documentation distinguishing implemented, tested, independently assessed and planned functions.

    Public transparency should describe purpose, limitations and oversight without disclosing personal information or operationally sensitive details. Detailed records should be available to appropriately authorised reviewers. Supplier documentation should identify functions that are implemented, tested, independently assessed or still planned.

  17. Evaluate intelligence-analysis task quality: correct/incorrect associations, missed connections and whether cited evidence supports answers.

    For intelligence analysis, a useful evaluation would measure: • Task quality: correct and incorrect record associations, missed connections, and whether cited evidence actually supports an answer.

  18. Measure human effort, operational performance and safety in intelligence-analysis evaluation, including verification/correction, unresolved specialist cases, reliability/recovery, full costs, unauthorised retrieval, disclosure, hostile source instructions and group error differences.

    Human effort: time to complete a defined task, verification and correction time, and unresolved cases requiring specialist review. • Operational performance: processing time, reliability, recovery and cost per completed task, including infrastructure, integration and support. • Safety: unauthorised retrieval, sensitive information disclosure, malicious instructions in source documents and differences in error rates across relevant groups.

  19. Publish evaluation methods and aggregate findings where lawful, including from unsuccessful trials, and test domain-specific models, retrieval-based systems and simpler methods against the same task.

    Publish methods and aggregate findings where lawful, including unsuccessful trials. Test domain-specific models, retrieval-based systems and simpler methods against the same task.

  20. Do not treat synthetic demonstrations alone as establishing operational accuracy or readiness.

    Synthetic demonstrations alone should not establish operational accuracy or readiness.

  21. Make explicit in funding and evaluation the distinction between retrieving Australian documents for an answer and training a model on Australian data.

    Retrieving Australian documents for an answer is different from training a model on Australian data; funding and evaluation should make that distinction explicit.

  22. Fund workforce capability during adoption: analyst training in source verification, uncertainty and automation bias, and technical training in integration, security, evaluation and model maintenance.

    Agencies also need to fund workforce capability as part of adoption. Analysts require training in source verification, uncertainty and automation bias; technical staff require skills in integration, security, evaluation and model maintenance.

  23. Involve staff in designing/testing changed workflows and evaluate workload, skill retention and service quality alongside time saved.

    Staff should help design and test the changed workflow. Evaluation should examine workload, skill retention and service quality as well as time saved.

  24. Make knowledge transfer and locally maintainable documentation explicit deliverables in funded projects.

    Knowledge transfer and locally maintainable documentation should be explicit deliverables in funded projects.

  25. Apply measures consistently to all suppliers, including Nuvento; consider Australian ownership alongside demonstrated capability, value for money and controls appropriate to the task.

    These measures should support competition and apply consistently to suppliers, including Nuvento. Australian ownership should be considered alongside demonstrated capability, value for money and the controls appropriate to the task.

  26. Nominate the Digital Transformation Agency to coordinate this work with procurement, industry, research and security bodies and participating agencies.

    Nuvento proposes that the government nominate the Digital Transformation Agency to coordinate this work with relevant procurement, industry, research and security bodies, and participating agencies.

  27. Within six months of accepting the recommendations, publish a common trial/evidence template and select limited agency problems, with funded owners, evaluation plans and decision points; tailor sovereignty/assurance requirements to information and consequences.

    Within six months of accepting these recommendations, the coordinating bodies should publish a common trial and evidence template and select a limited portfolio of agency problems. Each trial should have a funded agency owner, an evaluation plan and a defined decision point. Sovereignty and assurance requirements should be tailored to the information and consequences involved.

  28. Within twelve months, evaluate the portfolio publicly with sensitive-information protection: participation, decision times, task outcomes, correction effort, operating costs and reasons to proceed/stop; base further adoption funding on evidence.

    Within twelve months, government should publish an initial evaluation of the portfolio, with appropriate protection of sensitive information. It should report supplier participation, time to decision, measured task outcomes, correction effort, operating costs and reasons for proceeding or stopping. Funding for further adoption should depend on the evidence.

Read the original

Open in the interactive guide