Check the Document Before the Error Travels

TL;DR

Public boards, municipalities, hospitals, and health systems should make verification a routine step before consequential documents circulate or become public. Institution-operated AI can support that work through independent, adversarial review. The standard should be traceable evidence, corrected defects, and accountable human approval.

---

A board packet contains two different figures for the same expenditure. Meeting minutes record a motion as approved but omit an amendment. An internal memo turns a tentative recommendation into an apparent commitment.

These are ordinary document defects. Their consequences grow as other people rely on them.

At the author’s desk, correcting a number may require checking a spreadsheet and changing a sentence. After circulation, the same correction may require notifying colleagues, revising a presentation, reopening a discussion, and establishing which version everyone should use. Publication adds another audience and another correction process.

I want public institutions to catch these problems closer to their origin. Review should happen before minutes are published and before consequential internal documents become someone else’s input.

Apply engineering discipline to institutional documents

Software engineering offers a useful precedent. We review specifications, inspect code, run automated checks, and resolve defects before releasing changes. NIST’s Secure Software Development Framework explicitly recommends human review and automated analysis, with findings recorded and addressed in the development workflow. Public institutions should apply comparable discipline to the documents through which they govern and operate.¹

Some municipalities already use AI to prepare meeting minutes. The Sun-Gazette reports that Farmersville, California, voted in September 2026 to use Grok for initial drafts, followed by city-clerk review.² The Illinois Municipal League’s guidance likewise calls for thorough checking of AI-generated minutes, particularly vote counts and names.³ These examples establish an emerging drafting practice. I am proposing that institutions extend the discipline to reviewing documents, regardless of who wrote them.

A useful review asks whether the document faithfully represents its evidence. Are the figures consistent? Does the cited source support the claim? Has a qualification disappeared? Does the record distinguish a discussion, a recommendation, and an authorized decision?

For minutes, that requires comparison with recordings, motions, amendments, and other authoritative records. For an internal proposal, it may require the underlying calculations, applicable policy, and earlier decisions. A model reading only the finished prose cannot establish that an omitted event occurred.

Review independently, then challenge the findings

In my software work, I use repeated adversarial review of specifications before implementation. The Agent Flywheel methodology describes a related practice: review an artifact, correct its defects, and repeat with fresh eyes until the revisions converge. The same discipline belongs in institutional document work.⁴

For consequential documents, I favor two or more capable language models from different manufacturers. Each should first review the document independently, without seeing the other’s conclusions. Then they should challenge one another’s findings against the source material.

There is research behind this approach. Du and colleagues found that multiagent debate improved factuality and reasoning on the tasks they studied. Their experiments included a small test involving models from different providers. That supports using debate as a review technique; it does not establish a guaranteed result for municipal or hospital documents.⁵

The reviewers need a specific assignment. One can examine factual support, numbers, dates, and attribution. Another can look for contradictions, missing conditions, unsupported conclusions, and plausible alternative interpretations. Both should also examine the whole document so their assigned roles do not create gaps.

Every material finding should identify the passage, explain the defect, and point to the evidence. A challenge to a finding needs the same discipline. Unsupported criticism should be rejected, just as unsupported assertions in the document should be corrected.

The document owner then resolves the findings. After revision, the reviewers check the corrections and read the complete document again. Fixing one paragraph can create an inconsistency elsewhere.

This is a repeatable process:

1. Review independently against the source material.
2. Challenge findings and proposed corrections.
3. Resolve material issues and record the reasons.
4. Review the revised document again, including a fresh independent pass.
5. Obtain human approval of the final version.

The stopping rule should be explicit: identified material defects have been resolved, consequential claims have been checked, and fresh review finds no new material issue. If rounds keep reopening the same disagreement, a responsible person must settle the underlying question.

Agreement between models is insufficient evidence of correctness. A large study of model errors found substantial correlation, including between models from different providers. Manufacturer diversity is a useful design choice, but the source record remains the authority.⁶

Convergence means the review has stopped finding material defects under the checks performed. It cannot prove that every possible error has disappeared. This is familiar engineering practice: define the acceptance criteria, verify them, and retain responsibility for the result.

Protect the organization’s constraint

The Theory of Constraints lens makes the purpose clearer. A constraint is the factor that most limits the organization’s ability to deliver its mission. Depending on the institution, that may be clinical capacity, engineering judgment, legal review, or another scarce resource.

Document defects matter when they consume that capacity. An ambiguous memo can trigger another meeting. An unsupported claim can occupy legal staff. An incorrect assumption in a board paper can send an operational team down the wrong path. Early review earns its place by preventing this downstream work.

That gives us a practical dilemma. The organization needs reliable information to make sound decisions. It also needs timely decisions to carry out its mission. Staff therefore feel pressure both to add review and to move documents along.

The assumption worth challenging is that dependable review requires another centralized approval queue. Much of the checking can happen at the author’s desk, before handoff. Routine documents receive a brief check. Board papers, policy changes, and documents with significant financial or operational consequences receive deeper adversarial review. Specialists receive the questions that require their judgment.

This is how review can protect a constraint rather than become one. The test is whether it reduces the total work required to reach a dependable decision.

Keep the information under institutional control

A second dilemma concerns confidentiality. Institutions need analytical assistance, and they need control of their information. The apparent conflict often rests on an assumption that capable AI requires uploading documents to a public consumer service.

I favor locally operated review systems under institutional control. Where multiple models are used, every model must operate within the institution’s approved information boundary. Source access, permissions, logs, and retention need deliberate controls. Changing manufacturers must never become an excuse to distribute sensitive documents more widely.

The institution should also preserve the integrity of the record. Review must faithfully capture disagreement, qualifications, and uncomfortable facts. It must never become a way to rewrite what happened into a more convenient account.

NIST’s Generative AI Profile recommends reviewing and verifying sources and citations in model outputs. Institutions should apply that requirement to the reviewers’ findings as rigorously as to the documents being reviewed.⁷

Test the review and improve the work upstream

Before adopting the workflow broadly, test it against documents with known defects: an incorrect total, an omitted amendment, a citation that fails to support its claim. Track what the reviewers catch, what they miss, and how much unnecessary work their false alarms create. Then measure downstream corrections and decision delays.

Recurring findings should improve templates, source capture, and author guidance. Otherwise the institution merely becomes better at repairing the same defects.

Public institutions should make early verification part of normal work. Meeting minutes deserve it before publication. Consequential internal documents deserve it before colleagues act on them. Repeated adversarial AI review offers a practical way to strengthen that discipline while keeping evidence and responsibility with the institution.

If your organization is looking at a particular document workflow, I would welcome a discussion about where review would prevent the most rework. You can reach me at john@common-sense.com.

This article was developed with significant AI assistance, including source research and synthesis.

Sources

Footnotes

1. National Institute of Standards and Technology, Secure Software Development Framework (SSDF), Version 1.1, February 2022. Practice PW.7, printed pages 14–15, covers code review, automated analysis, and recording and resolving findings. ↩

2. Karis Caddell, “Farmersville to use transcription tool for verbatim minutes”, The Sun-Gazette, October 3, 2026. Reports the September 29 council vote and city-clerk review of initial drafts. ↩

3. Illinois Municipal League, Preparing Meeting Minutes With Artificial Intelligence (AI) Tools, August 27, 2026. The Quality Control section calls for thorough review, with particular attention to vote counts and names; the guidance advises against AI drafting of closed-session minutes. ↩

4. Jeffrey Emanuel, The Complete Flywheel Guide. See the convergence and fresh-eyes review guidance. This is a practitioner methodology, not an empirical study of public-sector document review. ↩

5. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch, Improving Factuality and Reasoning in Language Models through Multiagent Debate, 2023 preprint; published at ICML 2024. Section 3.3 includes an exploratory cross-provider experiment on 20 mathematics problems. ↩

6. Elliot Kim, Avi Garg, Kenny Peng, and Nikhil Garg, Correlated Errors in Large Language Models, June 2025 preprint. Reports correlated errors across evaluated models, including models with different providers and architectures. ↩

7. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 2024. Action MS-2.5-003, printed page 31, recommends reviewing and verifying sources and citations in model outputs. ↩