Why a Government AI Must Never Decide
By Anushka Appala and Dr. Janio Rosales
There is a seductive pitch making its way through ministries across Latin America: let artificial intelligence approve the benefits, issue the permits, flag the fraud, and clear the backlog. Point the model at the queue and let it decide.
It is the wrong idea, and getting it wrong is not a product bug — it is a constitutional crisis. An AI that wrongly denies a subsidy to an indigenous farmer, or fabricates an authorization for a procurement, has not "made an error." It has violated a right. The failure mode of a stochastic system making binding public decisions is not measured in accuracy percentages; it is measured in people harmed and trust destroyed.
So we want to make an argument that sounds like a limitation but is actually the whole point: a government AI must never decide.
Probabilistic tools, deterministic decisions
The confusion comes from treating "AI" as one thing. It is not. A large language model is a probabilistic system — extraordinary at understanding a request in Q'eqchi', summarizing a case file, drafting an acuerdo, or spotting an anomaly, but fundamentally a machine that predicts a likely output. It can be wrong in ways it cannot explain.
The execution of a public decision must be the opposite: deterministic. Given the same inputs, it must return the same result, every time, for a reason a citizen can read and a court can review. "Approve if the DPI is valid, the person is living, there is no duplicate benefit, and the socioeconomic threshold is met" is not machine learning — it is published law, rendered as code.
The mistake is letting the first thing do the second thing's job. The fix is to keep them strictly separate.
Governed autonomy: four roles, one guarantee
In NaciluzIA we call this supervised algorithmic governance, and it assigns four roles that never blur:
- AI recommends — it analyzes, translates, predicts, and drafts.
- Policies decide — a deterministic engine executes the binding rule.
- Officials oversee — a named human signs every consequential action.
- Audits verify — a cryptographic, tamper-evident ledger records everything.
Between any AI suggestion and any real-world effect sits a gate that is physically incapable of acting without both an authorized rule result and a human signature. The model proposes; the substrate disposes. This is not a policy we promise to follow — it is a property of the architecture.
Why the limit is also the unlock
Here is the part that surprises people: refusing to let AI decide is exactly what makes AI in government deployable. It is what lets the system stand up in court under an e-signature law. It is what satisfies an auditor who needs to know which rule justified which payment. It is what a development bank's responsible-AI framework is looking for — the Inter-American Development Bank notes that of its 26 borrowing countries, only five even have laws covering automated decisions. A design that is legally explainable by construction is not a compromise; it is the product.
And on corruption — the most tempting place to "let the AI catch it" — the same discipline holds. Our platform surfaces patterns and evidence for a human investigator; it does not determine misconduct. It identifies what requires legally authorized human review. "AI eliminates corruption" is a claim that loses credibility the moment it is spoken. "AI makes corruption automatically discoverable, and leaves judgment to people" is a claim you can defend.
The real promise
We are not building technology that takes power from public institutions. We are building the infrastructure that lets a government's decisions become faster, traceable, multilingual, and accountable — without ever handing a binding choice to a machine. That is a smaller claim than the hype, and a far more valuable one.
A state that can show where every quetzal went, in the citizen's own language, on their phone — and prove a human stood behind each decision — earns the right to outlast a single political cycle.
That is worth building carefully. It is why, in our system, the algorithm never gets the last word.
Cada decisión, a la luz.
— Anushka Appala and Dr. Janio Rosales