The AI can't do your taxes

In March 2023, four months after ChatGPT launched, we shipped a button I half expected to regret. It sat beside a household's results on PolicyEngine and said "Explain with AI" .^[1] Consider the kind of case it existed for — hypothetical here, but of a shape the engine computes constantly. A family of four in Connecticut earning $47,000 a year qualifies for about $475 a year in WIC benefits. The number is right, and the why is buried: an income threshold expressed as a percentage of a poverty guideline that itself depends on household size, a categorical-eligibility rule that can qualify the family through a different program entirely, participation windows, documentation requirements — dozens of intermediate steps the model tracks and no ordinary person will ever read. Open source made that logic visible. It did not make it legible. Anyone could inspect the eligibility code on GitHub; a determined user could in principle trace the calculation tree from inputs to answer. In practice, almost no one ever would, and a right answer nobody can follow fails the family almost as thoroughly as a wrong one — they cannot tell which fact about their lives is doing the work, so they cannot tell what would change it.

The button closed the gap by handing the whole tree — every intermediate value, every parameter — to a language model, which restated the result in sentences a person could use: you qualify because your youngest child is under five and your income falls below 185 percent of the poverty line for a household your size.

The model did not compute the answer. It read the answer back.

We chose that division of labor and meant it to last. A deterministic engine produces the numbers — encoding the rules exactly, returning identical outputs for identical inputs, auditable to the line of statute — and a language model explains them, never touching the arithmetic, calling the engine instead of consulting its own weights. The arrangement had a second dividend we did not fully appreciate at the time: because every explanation started from the engine's calculation tree, every explanation was checkable against it. Later in this chapter we would learn, expensively, what happens when an explanation floats free of the tree.

In the spring of 2023, though, the architecture looked to some smart people like a failure of nerve. The models were improving weekly; surely they would soon just do taxes. What settled the argument was a series of measurements.

Three benchmarks

In 2023, researchers at Johns Hopkins and the University of Maryland gave GPT-4 nine curated, self-contained sections of the Internal Revenue Code — the SARA benchmark, built so that every answer is derivable from the supplied text — and asked it 276 true-or-false questions about them. It got 186 right: 67 percent .^[2] Better than a coin, useless for a filing. The detail worth keeping is the shape of the ninety errors. Not one was arithmetic. Every miss was a misreading of the law the model had been handed.

Pause on what a misreading looks like from the receiving end, because it is the failure mode that makes this domain dangerous rather than merely hard. A model that misreads a statute does not stammer. It produces a fluent, structured, confident derivation — the same register in which it produces correct ones — and simulated comprehension looks exactly like comprehension. A bare wrong number invites a second look. A wrong number wrapped in a plausible explanation actively discourages one, which is why the failures in this section matter more, not less, as the models' prose improves.

Two years later, the tax-software company Column Tax raised the stakes from reading to filing. TaxCalcBench, published in July 2025, put frontier models to work on hand-built federal returns for tax year 2024 — synthetic cases with the taxpayer's complete information supplied, each filing scored line against line .^[3] Under strict scoring, the best performer, Google's Gemini 2.5 Pro, produced a fully correct return fewer than a third of the time; Anthropic's Claude Opus 4 managed 27 percent. Loosen the standard to within five dollars per line and the best model still cleared only about half. The failures had a signature: instead of looking up the IRS tax tables the law requires, models computed tax from the bracket percentages, landing three to five dollars off on return after return — a mechanism behind 15 to 20 percent of the failures by itself. And note what the test was not. These were bounded, synthetic, federal-only cases, simpler than what a storefront preparer sees in March. The models failed the easy version.

By 2026 we had built the harder version ourselves. A federal return is one slice of a household's situation; most families meet taxes and benefits at once, an EITC phasing out while SNAP phases down while Medicaid eligibility flips near a threshold. PolicyBench, our public benchmark, draws realistic household scenarios from calibrated microdata — so the cases cluster where actual households are complicated, with multiple earners, mixed-age children, stacked benefits — and asks two dozen frontier and open-weight models for the complete tax-and-benefit result, scored against the engine's answer to within a dollar .^[4] As of mid-2026, the best model still got roughly one in nine household calculations wrong by more than that dollar, and the typical model missed about one in four. The dollar threshold earns its keep: the misses are not rounding. They are a household ruled ineligible for Medicaid that qualifies, a food-assistance amount off by real money, a credit at the wrong value. A benefit determination is a number that decides whether a family gets help this month.

Three benchmarks, three years, one verdict. The obvious rejoinder at every step was that the next model generation would grow out of it, and the dates are the answer: several generations of frontier models came and went between the first test and the last, and the failure survived them all, migrating from reading comprehension to table lookups to program interactions as the tests grew more realistic. Each benchmark measured the same collision from a different angle — a machine trained on the patterns in text meeting a body of rules that is not a pattern. The rules change every January. They differ across fifty states. They turn on interactions among dozens of provisions, and next year's thresholds appear in no training set, because no one has written them yet. The next chapter takes up why this is permanent; what mattered in 2023 was the practical inference. If frontier models cannot reliably compute a return, they need a tool. AI systems do not memorize multiplication tables; they call calculators. An AI answering a tax question should call a tax engine.

The bottleneck moves

Our first attempt at the tool was crude enough to be funny now. In 2023 PolicyEngine generated a structured block of text — a reform's parameters against baselines, its budgetary and distributional results, a note on style — for the user to copy into ChatGPT, which spun it into readable analysis .^[5] The numbers were right because they came from the engine. The integration layer was a human being with a clipboard, ferrying data between two systems by hand.

Tool calling ended the ferrying. GPT-4, Claude, and their successors gained the ability to invoke a function mid-response: hit an API, take the result, keep going. Ask what happens if the Child Tax Credit becomes fully refundable, and the assistant translates the question into policy parameters, calls the engine, receives the quantitative results, and explains what came back. Language on the outside, computation on the inside. Then the wiring standardized. In November 2024 Anthropic open-sourced the Model Context Protocol, a common way for AI systems to discover and invoke external tools; OpenAI adopted it in March 2025, and Google followed within months. A PolicyEngine endpoint became findable and callable by any compatible assistant, no bespoke per-provider integration required. The plumbing stopped being the project.

What that buys in practice is a change in who can ask a question. A researcher writes, in plain English, that she wants to know how extending premium tax credits would affect insurance coverage among middle-income families in several states. The assistant translates that into computational steps — define the reform, select the population, run the simulations, aggregate the results — the engine does the analysis, and the assistant renders the findings back in her language. She never sees the parameters file. The engine never sees her prose. Each side does the work the other cannot.

Which moved the problem. In 2023 the binding constraint was the model: could it pick the right tool, format the call, map a person's question onto the tool's inputs without garbling them? By 2025 the models had cleared that bar so thoroughly that nobody discussed it anymore, and the constraint migrated to the tools — whether they were accurate, comprehensive, current, and legible to whoever stood downstream. An AI that flawlessly calls an inaccurate calculator is worse than useless, because it launders a wrong number through a confident, well-formatted sentence. And a calculator can be arithmetically correct and still fail its callers: slow to update, undocumented, full of integration traps.

Tool quality turns out to be several properties, each its own work. Accuracy. Comprehensiveness — a tool covering federal tax and three benefit programs returns the wrong household total the moment a fourth program is in play. Currency — when Congress rewrote the Child Tax Credit in July 2025, a tool answering family tax questions was wrong until someone updated it, and currency is that race, run each time a legislature meets. Legibility to the developer, the auditor, the caseworker downstream. None of these yields to more training. Each has to be built, and kept built, inside the tool.

Where authority lives

The architecture draws four lines, and the AI crosses none of them. It does not set policy parameters: the Child Tax Credit maximum in our engine is $2,200 because Congress set it at $2,200 in the One Big Beautiful Bill Act ,^[6] and every parameter carries its legislative or regulatory provenance. It does not estimate behavioral responses: elasticities come from the economic literature and from explicit, recorded methodological choices, never from a model's in-the-moment hunch. It does not judge whether a policy is good: the engine can report that a reform cuts poverty 3 percent and costs $50 billion, and whether that trade is worth making belongs to humans. And it does not overrule the engine: if the computation says a household owes $10,000, the explanation explains $10,000. Four boundaries, one rule seen from four sides — authority for a number belongs with whoever can be held to account for it. A legislature for a parameter. A documented literature for an elasticity. A voter for a value judgment. The engine for a computation. Never the system that is merely good at describing the number.

One place upstream deserves a careful exception, because it looks like a violation and is the opposite. A production model should never let an AI silently pick an elasticity and bury it in the code. But an AI can be a fast, auditable way to survey what the plausible values even are before a human commits. One recent experiment put twenty-six economic quantities — canonical labor-supply and tax elasticities among them — to eleven frontier models, repeating each elicitation fifteen times to recover a distribution of answers rather than a single number .^[7] A related project with the economist Jason DeBacker probes what language models imply about the elasticity of taxable income, down to replicating lab experiments on simulated taxpayers .^[8] These elicitations are diagnostics. They show what distribution a model carries, where models disagree, and where a change of wording moves the answer — and the useful finding is usually the scatter, not the consensus, because a spread wide enough to flip a reform from progressive to regressive names exactly which assumption the result is hostage to. The parameter a model actually uses still gets written down, sourced, and left open to review.

Day to day, what the interface adds is reach, with a seam. When we tested letting several specialized agents coordinate on a research task, they handled the standard distributional runs and stumbled where programs interacted — implementing each correctly in isolation, then missing how a benefit taper in one collides with a phase-out in another .^[9] The mechanical translation transfers; the coordination logic, so far, does not, and the seam where programs interact is where a human analyst still stands. The line between augmentation and automation runs through that seam, measured rather than asserted.

The reach cuts both ways. A congressional staffer who knows policy but not programming, an advocacy group with no data team, can now run analyses that used to require an economist — and so can anyone hunting for a cherry-picked scenario or a confident misreading. Those risks existed under the old system too, concentrated among the few who could run the models. Broader access invites broader checking, and the explanation is the part that scales best: show a worker that she faces a 67 percent marginal tax rate, and the engine hands over the pieces — federal tax, payroll tax, an EITC phasing out, a partial credit clawback — for the model to assemble into the one plain sentence explaining why an extra dollar of her earnings mostly vanishes. The old system could compute that rate too. It could not put the explanation in front of the person living it.

Three builders, one wall

We were not the only ones who ended up here. In November 2025 the IRS deployed Salesforce's Agentforce agents across three of its offices — the Office of Chief Counsel, the Taxpayer Advocate Service, and Appeals — for case summarization, document search, and policy navigation .^[10] The deployment followed a workforce reduction, and its boundaries were explicit: every final decision stayed with a human, and the AI could neither disburse funds nor make an eligibility determination.

California had started earlier. Poppy, the state's digital assistant for its own workforce, began its pilot on September 29, 2025, across 67 departments — built in-house, run on state servers rather than an external cloud, drawing on roughly ten models (the state's technology department lists Claude, Gemini, GPT, and Nova among them), and grounded on public state data to hold down hallucination .^[11] More than 2,800 employees used it during the pilot; the statewide rollout came in July 2026. Poppy helps a state worker find and understand a rule. It computes no one's entitlement.

Look at what both deployments protect. In each, the rules, the statutes, and the calculations stay human-maintained and human-auditable underneath; the AI sits on top, summarizing, retrieving, navigating — the access layer to an authority it does not hold. That is the same wall we built into a benefits calculator, drawn independently by a federal agency spending on commercial software and a state government writing its own.

An open-source project, a commercial vendor inside a federal agency, and a state building for itself, none copying the others, landed on the same division of labor: deterministic systems for the answers, AI for the access. Three builders under three sets of incentives hit the same wall in the same place. The problem put it there.

When the explanation lied

The last lesson of the benchmark years came from our own board, and it stung. PolicyBench originally published an AI-written narrative beside each scored scenario — an explanation layer, meant to make the results legible to auditors. The narratives read well. Some of them lied. One described a household of retirees as enrollees in a "non-elderly expansion" category that could not apply to them: the mechanism was invented, the prose was fluent, and the combination sailed past human review precisely because it sounded like an explanation. An auditor who read it inherited the fabrication whole.

The fix went deeper than a prompt. We stopped letting the narrative come from the model's imagination and made it come from the engine's internals — every explanation regenerated from the actual variables and intermediate values the computation produced, with validators that reject any mechanism the underlying trace does not support. An ungrounded claim now fails the build before it can reach a reader. The general lesson runs through the next two chapters: even the audit layer needs ground truth. If you want an AI to check work — its own or anyone else's — you give it traces, the numbers the system actually computed. You do not give it prose.

So the models became the interface, and by 2026 the interface was good. That only sharpens the question of what it is an interface to. Part II ended with a criterion — an encoding of the law earns trust at the unit of one household, the level where its answer can be proved against something real — and this chapter has been the criterion's negative image: three benchmarks showing what happens when the answer comes from a system with no unit anyone can check. A tool worth routing the public's tax and benefit questions through has to clear a bar most software never faces: not merely accurate but accountable — every number traceable to a line of statute, checkable against an independent calculator, reconcilable to totals a government actually counted. At the scale of the whole law, no such tool existed.

Building one turned out to be possible. That is the next chapter.