The Long View on AI / 02

When Token Costs Approach Zero, What Remains Expensive?

Falling model prices do not make the full cost of AI disappear. The scarce and expensive parts are moving into energy, usable data, integration, verification, trust and organizational change.

M.K.September 4, 202612 min read

A light stream of data rises from a small compute node above a much larger system of grids, data centres, archives and human review

M.K. / FIELD NOTESAI Infrastructure / Compute Economics / 2026

When token costs approach zero, expensive things do not disappear. They move.

The posted price of model input and output still matters, but it measures only how much material a model processed. It does not count what an organization spends to obtain a result that can be accepted, traced, acted on and defended: electricity and capacity, usable data, system integration, human review, recovery from errors, and changes to work itself.

For a low-risk rewrite of an internal email, the model charge may already be negligible. For a system that touches credit, procurement, health, public services or a large software estate, the scarce resource is usually not the first answer. It is the evidence that the answer deserves to enter the real workflow.

This essay is therefore not a price comparison and does not forecast a particular provider. It asks a more durable question: as model output becomes cheaper, where do the real scarcity and cost move?

My answer is six places: energy and capacity, usable data, integration, verification and security, trust and accountability, and organizational learning. To see whether AI is truly getting cheaper, the unit of account must also change—from cost per million tokens to total cost per accepted outcome.

Start with the boundary: a token price is not a total cost

A token is a useful unit for metering what a model reads and writes. It is a poor unit for measuring economic value. One million tokens might represent low-risk classification that nobody reads, or thousands of insurance recommendations that require careful review. The posted model charges could be similar while the cost of error, attention and evidence is entirely different.

As of September 1, 2026, official provider pages already divide model service into several prices. Input, cached input, output, processing speed, delayed work, tool use and data location may all be charged differently. At that date, for example, OpenAI listed its cost-oriented GPT-5.6 Luna at US$0.20 per million input tokens, US$0.02 for cached input and US$1.20 for output. Google listed Gemini 3.5 Flash-Lite at US$0.30, US$0.03 and US$2.50 for the same broad categories. Anthropic listed Claude Opus 5 at US$5 for input, US$0.50 for a cache hit and US$25 for output.

These are models with different capabilities and intended uses. The numbers are not an apples-to-apples ranking. Their shared lesson is more important: even the model bill is shaped by the workload, not by one token price. Reused context, output length, latency, tool calls and service location can matter as much as the headline input rate.

The longer-run decline is real. The Stanford 2025 AI Index compared systems at a fixed capability threshold and estimated that the cost of reaching GPT-3.5-level performance on MMLU fell from US$20 per million tokens in November 2022 to US$0.07 by October 2024, a reduction of more than 280 times. That is a historical measurement, not evidence that every AI task became 280 times cheaper. Capability thresholds, output length, reliability and deployment conditions also changed.

A more useful equation is:

Total cost per accepted outcome = (model + data + tools + infrastructure + human review + error + change costs) ÷ outcomes accepted and put to work

The denominator matters. Producing ten thousand cheap recommendations that nobody trusts may be less efficient than producing one hundred expensive recommendations that reliably close a case.

What can be confirmed now: a six-layer cost structure

1. Energy and capacity: a cheaper unit can coexist with higher total demand

Model and chip efficiency can improve while aggregate electricity use rises. Lower prices invite more frequent use, longer contexts, more candidate answers, more tool calls and more continuous background work. Savings at the unit level can be absorbed by new demand.

The International Energy Agency’s base case projects global data-centre electricity use rising from roughly 415 TWh in 2024 to about 945 TWh in 2030, still just under 3% of global electricity consumption. The IEA is explicit about the uncertainty and publishes cases shaped by efficiency gains, adoption and energy bottlenecks. The figure should not be read as a certain future. It identifies a physical mismatch in speed: a data centre may be built in a few years, while grids, substations and new supply often require longer planning and large capital commitments.

The local effect may be more important than the global share. Data centres concentrate around suitable grids, fibre, water, land and permitting. The U.S. Department of Energy, citing Lawrence Berkeley National Laboratory, estimated that data centres used about 4.4% of U.S. electricity in 2023 and could use 6.7% to 12% by 2028. A wide range is not a weakness to hide; it is evidence that location, buildout, utilization and efficiency remain decisive variables.

Energy is therefore not a fixed surcharge on a token. It is a capacity constraint with a place, a timetable and an opportunity cost for other users of the grid.

2. Usable data: having documents is not the same as having permissioned knowledge

Most organizations do not lack files. They lack information with clear provenance, permissions, versions, quality controls and retention rules. Before contracts, product records, customer conversations or internal knowledge can support a model, somebody must answer: Who owns this? Who may see it? Which version is authoritative? How is an error corrected? Do the original access rules survive after the material leaves its source system?

The cost here includes deduplication, access control, source tracing, privacy review, vocabulary alignment and rechecking after the underlying records change. A model may read the material cheaply. Deciding what is worth reading, what is allowed to be read and how a mistake will be detected still requires domain knowledge and continuing maintenance.

This is especially relevant in Canada, where many organizations operate across provincial, federal and international requirements, and where English and French information may not be equally complete. Local context is not simply more text. It is a maintained relationship among language, authority, date and purpose.

3. Integration: the answer is usually the middle of the workflow

Real work rarely ends when text appears. A customer-service suggestion must connect to identity, account history and a ticket. A procurement recommendation must check inventory, price, supplier terms and approval limits. Code must pass tests, review, security controls and release rules.

Every added system introduces permissions, timeouts, versions, retries, partial failures and record-keeping. A model can produce the right sentence and still fail the task because a tool returned stale data, a write did not complete, or an exception had no owner.

This is why reducing human effort does not automatically reduce total cost. Decisions that people once handled implicitly may need to become explicit rules, tests and interfaces. Integration spending can rise before it falls. The long-term result depends on whether the workflow becomes repeatable and observable, not whether a demonstration succeeds once.

4. Verification and security: important outcomes cannot rely on averages

AI output is not a fixed software response. A system may perform well on common cases and fail on a minority language, a rare event, a new policy or a hostile input. The NIST AI Risk Management Framework and its AI profile treat governance, context mapping, measurement and management as lifecycle activities rather than a one-time gate before launch.

Verification therefore includes representative evaluations, thresholds for acceptable error, adversarial testing, change monitoring, decision records, incident handling and reassessment after a model update. High-impact uses also need human review and a route for appeal.

These costs can rise as model calls get cheaper. If an organization produces ten times more material without improving how it selects and checks cases, it may simply create a larger review queue. The binding resource becomes qualified attention.

5. Trust and accountability: the cost of being wrong is off the model invoice

An incorrect internal summary may be cheap to fix. An incorrect refund, denial of service, disclosure of private information or discriminatory ranking can become a legal liability, a damaged relationship and a loss of institutional legitimacy. Those losses appear after the call and cannot be averaged away by a low rate per token.

A trustworthy system needs an explicit chain of responsibility. Who can approve an action? Who receives an alert? Who can override it? Who explains the decision to an affected person? Who rechecks the service when the provider changes the model?

This does not mean keeping a person in every loop forever. It means matching human responsibility to the consequences of the action. Low-impact cases may be checked by rules and sampling; consequential cases may require named approval, preserved evidence and a meaningful route to correction.

6. Organizational learning: the tool is cheap; changing work is not

The strongest field evidence shows that gains depend heavily on the setting. An NBER study of 5,179 customer-support agents found that access to an AI assistant increased issues resolved per hour by about 14% on average, with larger gains among newer and lower-skilled workers. Microsoft Research combined three randomized field experiments at Microsoft, Accenture and an unnamed Fortune 100 company. Across 4,867 developers, access to a coding assistant was associated with a 26.08% increase in completed tasks, although the researchers note that each individual experiment was noisy.

In a sharply different result, METR studied 16 experienced developers doing 246 real tasks in mature open-source projects. With early-2025 AI tools allowed, completion time increased by 19%. The authors carefully limit the result to that tool era, those participants and work in codebases the developers knew deeply.

These findings do not cancel one another. Together they support a more cautious conclusion: AI does not carry a fixed productivity percentage from one workplace to another. The outcome depends on task structure, worker experience, quality standards, feedback speed and the surrounding workflow. Training, work allocation, performance measures and management responsibility are part of the cost.

My analysis: scarcity moves from producing to accepting

When text, code, images and recommendations become abundant, producing a candidate is no longer the primary scarcity. Attention comes next: which result matters? Then verification: how do we know it is sound? Then accountability: who bears the consequence if it is wrong? Finally, organizational capability: can a good result be turned into repeatable action?

There is a historical parallel. As computing and storage became cheaper, organizations did not stop spending on technology. More of the bill moved into software, cybersecurity, data management and people. The analogy has a limit. AI participates in judgement, and its quality is harder to measure than storage capacity. Verification and accountability can therefore represent a larger share of the total.

I expect three conditions to coexist over the next five to ten years.

Standardized work becomes dramatically cheaper. Tasks with clear inputs, low error costs and machine-checkable outcomes can approach a utility-like price through small models, reused context and flexible timing.
High-impact work remains expensive. Model charges fall, but data rights, domain review, continuous monitoring, insurance and accountability dominate. The durable advantage is a trusted process, not a larger call count.
Total spending rises with demand. Each use is cheaper, but more devices, agents and background processes run continuously. Falling unit cost and rising aggregate spending on power, capacity and review can both be true.

My position is that organizations should not decide whether to scale from a price per million tokens. They should define a measurable completed outcome, establish a baseline without AI, and compare full cost and error distribution. If the denominator is output volume, the system is rewarded for creating more work. If the denominator is accepted outcomes, model choice, workflow design and human responsibility return to the same ledger.

Signals to watch over the next 12–24 months

Six observable signals would show where scarcity is moving:

Do organizations report cost per resolved case, completed task or accepted decision, rather than only users and token volume?
As small-model and reused-context prices fall, do charges for tools, search, data location and high-speed service become a larger share of the bill?
Can large deployments maintain quality after a model update without extensive manual reevaluation?
Do incident, appeal, rework and human-recovery rates fall as usage grows, rather than merely average benchmark scores rising?
Do grid interconnection, power agreements and local construction become slower constraints than chip supply?
Does performance variation among organizations using similar tools narrow? If it remains wide, organizational capability is likely the scarce asset.

What would change my mind?

I would reduce my confidence in the “expensive complements” view if mature organizations demonstrate, across several high-impact fields, that end-to-end cost per accepted outcome falls roughly in step with posted model prices. That evidence would also need to show that review, incidents, appeals and integration maintenance have not simply moved to another department; model replacement does not require major reevaluation; and energy and capacity demand remain nearly flat during rapid growth in use.

The opposite pattern would support the cost-shift view: the model share of the technology budget falls while spending on data governance, security, integration, people and energy rises.

There is substantial uncertainty. Provider prices change quickly, negotiated discounts are rarely public, energy projections span wide ranges, and productivity studies are specific to tasks and time. This essay offers a testable way to account for costs. It is not investment advice and cannot replace legal, security or financial assessment for a particular sector.

Frequently asked questions

Will token costs literally reach zero?

“Approach zero” is an economic claim, not necessarily free service. The model charge can become tiny relative to labour, integration and error costs. Frontier capability, low latency, long contexts and specialized tools may continue to command a meaningful price.

Why can’t a company save in proportion to the drop in model prices?

The model invoice is only one link in the chain. Higher use, data preparation, workflow changes and review can absorb the savings. The relevant comparison is total cost per successfully completed outcome.

Do small models or local deployment solve the cost problem?

They can reduce transmission, latency and external service charges, and they may improve privacy and control. Hardware, updates, monitoring, security and operations then move inside the organization. Whether that is cheaper depends on stable demand and in-house capability.

Does human review make AI uneconomic?

Not necessarily. Low-risk, machine-checkable work can route only exceptions to people, while high-impact work can focus human attention on a small number of consequential decisions. The issue is not whether a person appears anywhere, but whether scarce attention is placed where it most reduces harm and rework.

What is the first metric an organization should establish?

Track total cost per accepted and completed outcome, including human time, failures, rework, appeals and incidents. Changing the denominator from output volume to useful outcomes reveals whether cheaper models create real savings or merely create more material to process.