AI Inquiry Atlas

An unofficial, independent guide to the public records of the Joint Select Committee on Artificial Intelligence. Not connected with the Parliament of Australia or the committee. Its makers also made a submission (no. 481; see About). Summaries and scores are written by AI and can be wrong: check the original before relying on them.

Submission 570

Mr Steve Davies

Submitter group: Individuals.

Position labels are not shown for individual submitters.

An AI model identified the submitter group and wrote the summary, key points and recommendation summaries. Counts do not measure importance or merit.

Open in the interactive guide

Summary

Read the original submission for the full text.

Asks whether AI’s lack of conscience precludes supporting public-sector moral analysis. The author proposes scrutiny and an independent, bounded trial of their Deep Truth framework, based on Bandura’s moral disengagement mechanisms, with humans retaining judgement. States it lacks independent evaluation. Attachments propose a draft testing rubric and trial design, including platform comparison and independent raters.

Key points

Quotes are the submitter’s own words. Text taken from documents, including scanned pages read by text-recognition software, can differ from the PDF. Check the original. Listed in the same order as on the submission’s page in the interactive guide.

  1. Asks the Committee to test the reasoning that AI cannot support moral analysis because it has no conscience.

    I ask the Committee to examine one reasoning step in how AI is being kept out of parts of the public sector: whether the absence of conscience necessarily prevents a tool from supporting moral analysis.

  2. Asks the Committee to clarify why the Australian Public Service Commission declined further consideration of the framework, including future versions.

    I also ask the Committee to seek clarification from the Australian Public Service Commission, and from other agencies with responsibility for AI in government as it sees fit, of the basis on which the Commission decided not to consider the framework, including any future versions.

  3. Argues that a moral lens need not hold a conscience but needs a stated method, checkable evidence and a boundary between what the tool identifies and what a person decides.

    A lens does not need to hold a conscience. It needs a stated analytical method, evidence that can be checked, and a clear boundary between what the instrument identifies and what a person decides.

  4. Describes the framework as a human-in-the-loop design in which the human decides, which it says bears on accountability measures for AI use in agencies.

    For the Committee, this is a human-in-the-loop design where the human is the decision-maker, which bears directly on accountability measures for AI use in agencies (term of reference (h)).

  5. States that Deep Truth is a single-author framework that has not been independently evaluated and provides input to human judgement.

    Deep Truth is a single-author framework. It has not been independently evaluated. Its findings are an input to human judgement, not a substitute for it.

  6. Describes the framework as applying Bandura's eight mechanisms of moral disengagement as a lens to text, policy and practice.

    Professor Albert Bandura identified eight mechanisms by which people and institutions disengage from the moral weight of what they do. Deep Truth applies these mechanisms as a lens to text, policy and practice.

Recommendations

Recommendations are shown beside the submission’s own words. Listed in the same order as on its page in the interactive guide.

  1. Seek the APSC’s reasons for declining consideration, including future versions, consulting other agencies responsible for government AI as the Committee sees fit.

    I also ask the Committee to seek clarification from the Australian Public Service Commission, and from other agencies with responsibility for AI in government as it sees fit, of the basis on which the Commission decided not to consider the framework, including any future versions.

  2. Consider testing the proposition that AI cannot support moral analysis against how such tools are designed and governed.

    Whether the proposition that AI cannot hold a conscience, and so cannot support moral analysis, should be tested against how such tools are actually designed and governed.

  3. Consider a bounded, transparent trial of such tools, designed and conducted independently of the agencies concerned.

    Whether a bounded, transparent trial of tools of this kind should be designed and conducted independently of the agencies concerned, as sound practice.

  4. Consider whether design-stage moral review belongs alongside existing assessment tools for AI in government.

    Whether design-stage moral review belongs alongside existing assessment tools for AI in government, relevant to term of reference (h).

  5. For the proposed trial, use a fixed test set covering documented harm, institutional texts, citizen testimony and controls with little disengagement.

    1. Fixed test set. Choose documents of known character: some with documented harm (for example, the Robodebt Royal Commission record), some institutional texts, some citizen testimony, and some control texts with little disengagement.

  6. Run identical documents through at least three AI platforms using the same Persona version.

    2. Same inputs, several platforms. Run each document through at least three AI platforms using the same version of the Persona.

  7. Use at least two independent raters, blinded to the platform, to score each output against the ten invariants.

    3. Two or more independent raters. Raters score each output against the ten invariants without knowing which platform produced it.

  8. Measure rater agreement with an ordinal statistic such as weighted kappa; discuss and record reasons for differences greater than one point.

    4. Measure agreement. Use a standard agreement statistic for ordinal scales, such as weighted kappa. Where raters differ by more than one point, they discuss and record the reason.

  9. Publish agreement and variance across platforms and raters, and implications for refining the Persona.

    5. Report variance openly. Publish where platforms and raters agree, where they differ, and what the differences suggest for refining the Persona.

  10. Have the researcher assist independent testing within agencies.

    Testing within agencies is the logical next step. As Australian Public Service agencies have an interest in the outcome, that testing should be designed and run independently of them with assistance from the researcher.

  11. Ask which agencies were consulted in reaching the decision.

    The Committee may wish to ask substance of the material” why, and which agencies were “I am also not able to facilitate the consulted in reaching that uploading, promotion or inclusion of Deep decision.

Read the original

Open in the interactive guide