The real cost of enterprise AI: why price per token falls short

When a company starts experimenting with artificial intelligence, one question appears almost immediately:

How much does one million tokens cost?

It is a useful question. It lets us compare models, providers, and configurations. But when AI stops being an experiment and becomes part of real business processes, that question starts to look too small.

A recent TechRadar Pro article makes exactly that shift in perspective: behind every token there is physical infrastructure —compute, memory, networking, electricity, and cooling— and at enterprise scale those decisions can shape the economics of the system as much as the provider’s published price.

The idea can be summarized like this:

price per token ≠ total cost of operating AI

And with the arrival of AI agents, there is a second complication: a single human task can trigger many inferences, tool calls, verifications, and retries.

So perhaps the more interesting metric is not how much it costs to generate tokens, but:

How much does it cost to correctly complete a business task?

From price per token to cost per business outcome.

A token is a unit of consumption, not a unit of value

Language models need some way to measure how much text they process and generate. Tokens work well for that purpose.

An API bill can be approximated with something like:

model cost =
    input tokens × input rate
  + output tokens × output rate

That tells us how much we consume.

It does not tell us how much value we obtain.

Two agents can spend exactly the same number of tokens and produce completely different outcomes:

Agent A
100,000 tokens
→ correctly solves 95 tasks

Agent B
100,000 tokens
→ correctly solves 45 tasks

Looking only at tokens, they seem equivalent.

Looking at results, they clearly are not.

This difference matters because companies do not buy tokens for the sake of tokens. They consume them to accomplish something: answer customers, analyze documents, write software, investigate incidents, process claims, detect fraud, or automate operations.

The economic unit should move closer to the outcome that actually matters.

The visible bill is only the first layer

Even when a company uses an API and never buys a GPU, every interaction still depends on physical infrastructure somewhere.

There is compute, memory, networking, storage, power, and cooling behind the response.

If the company also builds its own AI platform, additional operational layers appear:

model
+
infrastructure
+
platform
+
observability
+
security
+
governance
+
engineering
+
retries and failures

The total-cost stack for operating an AI capability.

We can think of total AI cost as a conceptual equation:

Total AI cost =
    inference
  + infrastructure
  + power and network
  + storage
  + platform
  + observability
  + security and governance
  + engineering
  + failures, retries, and corrections

Not every company pays those line items directly. With an API, many of them are bundled into the provider’s price.

But economically they still exist.

That distinction matters as consumption grows to millions of requests and AI becomes a critical business dependency.

Consumption-based APIs: excellent when we need elasticity

The API model has enormous advantages.

A company can start without buying hardware, operating an accelerator cluster, or sizing capacity months in advance.

It is especially attractive for experimental or variable workloads:

low demand       → pay little
high demand      → pay more
zero demand      → almost zero inference cost

Infrastructure becomes a service.

That lowers initial investment and makes it possible to switch models quickly.

So it would be a mistake to frame the debate as “API bad, own infrastructure good.”

The right question is which economic model fits the actual workload best.

When does dedicated capacity become interesting?

When a workload becomes very large, steady, and predictable, the equation can change.

Instead of paying an incremental rate for every interaction, an organization can examine whether it makes sense to buy or reserve inference capacity and keep it sufficiently utilized.

Conceptually, two different curves appear:

Conceptual comparison between consumption APIs and dedicated capacity.

APIs tend to favor:

experimentation;
irregular demand;
uncertain growth;
teams that want to minimize infrastructure operations;
rapid access to frontier models.

Dedicated capacity can become attractive when there are:

sustained and predictable workloads;
high hardware utilization;
strong data-residency requirements;
specific latency needs;
a need to control versions and behavior;
enough scale to amortize infrastructure and operations.

The break-even point is not universal.

It depends on model price, hardware, real utilization, electricity, cooling, operations salaries, storage, networking, required availability, and many other factors.

That is why every “API vs. self-hosted” chart should be interpreted as a model, not a law.

Agents change the economics even more

A simple chatbot can look like this:

question
   ↓
model
   ↓
answer

An agent can look more like this:

objective
   ↓
reasoning
   ↓
search
   ↓
result
   ↓
reasoning
   ↓
tool call
   ↓
result
   ↓
verification
   ↓
finished?
  ↙   ↘
no     yes
↓       ↓
repeat  answer

One agent task can involve multiple inferences and tool calls.

Every loop can add more context and generate new tokens.

There can also be calls to external services, browsers, databases, sandboxes, search engines, storage, or execution tools.

Suddenly the economics of a task no longer depend on one inference.

We can model it in simplified form:

Cost of an agent task =
    Σ inferences
  + Σ tool calls
  + auxiliary compute
  + storage
  + execution time
  + retries
  + required human supervision

This explains why a model with cheaper tokens does not necessarily produce the cheapest agent.

A slightly more expensive model that needs fewer attempts can end up costing less per completed task.

Example: cheap per token, expensive per result

Imagine two fictional configurations for processing 10,000 tasks.

Configuration A

Average cost per attempt:    $0.012
Average attempts per task:   2.8
Final success rate:          90%

Approximate cost:

10,000 × 2.8 × $0.012 = $336

Successful tasks:

10,000 × 90% = 9,000

Cost per successful task:

$336 / 9,000 ≈ $0.037

Configuration B

Average cost per attempt:    $0.020
Average attempts per task:   1.3
Final success rate:          97%

Approximate cost:

10,000 × 1.3 × $0.020 = $260

Successful tasks:

10,000 × 97% = 9,700

Cost per successful task:

$260 / 9,700 ≈ $0.027

Configuration B uses a more expensive attempt, but ends up cheaper per useful outcome.

This example is deliberately hypothetical. Its purpose is not to recommend a specific model, but to show why optimizing a single rate can lead us to the wrong decision.

A more useful metric: cost per successful task

For an agent system we can define:

Cost per successful task =
    total system cost
    ─────────────────
    correctly completed tasks

The numerator should include everything material to that operation.

For example:

model
+ tools
+ infrastructure
+ execution
+ retries
+ storage
+ observability
+ human correction work

The denominator should not simply be “responses produced,” either.

It has to measure valid outcomes.

In programming it could be:

accepted PRs with green CI

In support:

cases resolved without reopening

In document processing:

documents processed that pass validation

In a financial workflow:

transactions processed correctly without manual intervention

That connects economics with quality.

And we still have not included the value of the outcome

Cost per successful task improves measurement substantially, but it still looks only at the cost side.

The next layer is to compare that cost with the economic value produced.

A simplified version would be:

AI ROI =
    economic value generated
    ────────────────────────
    total cost of the capability

An automation that costs $5 per task could be extraordinarily profitable if it replaces a $100 operation.

Another that costs five cents could be useless if it produces results nobody needs.

That is why more AI consumption does not automatically mean more productivity.

Token volume can rise while value remains flat.

Governance also costs money

There is another dimension that does not fit easily in a pricing table: models change.

Providers update infrastructure, policies, versions, and in some cases the observable behavior of their models.

For an informal application, that may not be a serious problem.

For a regulated company or critical process, it can be.

The organization may need:

regression evaluations;
versioned models or snapshots;
auditing of prompts and tool calls;
access controls;
data policies;
behavior monitoring;
rollback mechanisms;
human review for certain actions.

All of that is part of operating AI reliably.

And all of it has a cost.

The real shift: from buying tokens to operating capacity

During the first stage of adoption, it made sense to talk almost exclusively about models and rates.

The question was:

Which model is better and how much does it cost?

As AI becomes integrated into real processes, the question matures:

Which architecture delivers the outcome
with sufficient quality, control, and resilience
at the lowest sustainable total cost?

That requires looking simultaneously at:

model
+
infrastructure
+
agents
+
operations
+
governance
+
business value

The token remains important.

It simply stops being the center of the economics.

What an AI team should measure

Instead of a dashboard limited to tokens, a mature operation could track metrics such as:

Metric	What it tells us
Tokens per task	Intensity of model use
Cost per attempt	Price of executing one round
Attempts per task	Agent efficiency
Tool calls per task	Operational complexity
Success rate	Real system quality
Cost per successful task	Economics of the outcome
Latency per task	Experience and capacity
Human interventions	How much autonomy actually works
Estimated value per task	Economic impact
ROI	Whether the capability produces more value than it costs

The conclusion is simple:

Tokens measure activity. Outcomes measure usefulness.

And when AI becomes enterprise infrastructure, it makes sense to optimize the latter.

Source and inspiration

This article builds on the thesis presented in “We’re asking the wrong question about the cost of enterprise AI”, published in TechRadar Pro on August 18, 2026. The explanation, examples, conceptual formulas, and proposal to measure cost per successful task are an original didactic elaboration of that theme.

Original source: TechRadar Pro — We’re asking the wrong question about the cost of enterprise AI