Electricity into tokens: what AI compute costs

Every model call consumes compute. When a company uses a hosted model, the provider usually meters much of that use in input and output tokens. The price on the invoice is the end of a physical chain that starts with electricity and runs through a data centre, chips and the software used to serve the model.

Two things are happening along that chain. The cost of reaching a fixed level of capability has been falling quickly. At the same time, access to the newest chips and models can become more expensive when demand grows faster than capacity. Both matter to a company running agents, because an agent can make many model calls while completing one case.

This article follows the chain with figures available in September 2026. Measurements, market observations and forecasts are kept separate. Calculations are labelled illustrative and show their assumptions.

01How does electricity become a token?

A power plant and grid connection deliver electricity to a data centre. The site turns it into cooled, conditioned power for servers. Semiconductor factories turn wafers, memory and packaging into AI chips. Those chips run a model through a serving stack, which converts requests into tokens.

The links do not expand at the same speed. Software can improve in weeks. A new chip generation takes years to design and manufacture. Data centres take years to build, and the grid connection can take longer. Capacity grows at the pace of the slowest link.

The scale at the right-hand end is already large. Google reported processing more than 3.2 quadrillion tokens a month across its products and APIs in May 2026, seven times the volume reported a year earlier. That is one company’s self-reported volume, not a measure of the whole market, but it shows how quickly demand can move.

02How much electricity does a token use?

There is no universal figure. Energy use changes with the model, hardware, prompt length, output length, batching, utilisation and the boundary used for measurement. A figure that counts only the active AI chip will be lower than one that also includes host machines, idle capacity and cooling.

Google published one of the more complete public measurements. A median Gemini Apps text prompt used 0.24 watt-hours in May 2025. Its method includes AI accelerators, host machines, idle capacity and data-centre overhead. Google also reported that energy per median prompt had fallen 33-fold in twelve months. The company did not publish the number of tokens in that median prompt, and the results have not been independently verified.

Converting the prompt measurement into energy per token therefore requires assumptions. Combining it with public hardware and workload estimates gives the following order of magnitude:

The range is useful for scale, not for comparing providers. A token price also pays for the chips, networking, building, model development, serving software, reserved capacity and provider margin. Epoch AI models a one-gigawatt AI data centre at about $38 billion of up-front capital. Servers account for 56 percent of that estimate and the facility for 30 percent. Annual electricity is material, at about $0.6 billion, but smaller than the annualised cost of the hardware.

03Why can power still be the bottleneck?

The electricity charge may be a modest part of a token price, but access to electricity is slow to expand. The IEA puts a data centre at two to three years to build. A new grid connection in Europe can take two to ten years, depending on the country and location.

Demand is rising faster than the wider electricity system. The IEA says data-centre electricity use grew 17 percent in 2025, compared with 3 percent for electricity overall. Its central projection has global data-centre consumption rising from 485 TWh in 2025 to about 950 TWh in 2030. That projection is conditional: financing, returns, equipment supply and planning decisions can all change the outcome.

Equipment lead times add another constraint. Gas-turbine orders rose 70 percent in 2025, according to the IEA. GE Vernova reported 116 GW of gas-power equipment backlog and slot reservations in July 2026. Building generation next to a data centre may avoid a grid queue, but it still depends on scarce equipment, permits and fuel.

The Netherlands makes the timing visible. Data centres received 4.6 percent of Dutch electricity in 2024. In 2026, a court allowed TenneT to delay a 70 MW connection in Vijfhuizen until the regional grid is reinforced. NOS reported that sufficient reinforcement is expected between 2033 and 2035. The connection date, rather than the electricity price, determines when that capacity can operate.

04How fast can chip capacity grow?

Epoch AI estimates that global computing capacity from AI chips has grown about 3.3 times a year since 2022. That is exceptionally fast for physical infrastructure, but it is still slower than some measures of token demand. It is also a historical estimate, not a promise that the rate will continue.

Chip supply depends on more than final assembly. Advanced logic, high-bandwidth memory, packaging and lithography equipment all need to arrive together. ASML is the only supplier of EUV lithography systems used for the most intricate layers of advanced chips. New fabrication space takes years to build before those tools can be installed.

High-bandwidth memory is another limit because it sits next to the processor and moves model data fast enough to keep the chip working. The IEA expects the current shortage to last through 2027. Microsoft said its $41 billion of capital expenditure in the June 2026 quarter included the effect of higher component prices. These constraints do not stop chip production, but they make short-term capacity less responsive to demand.

05Why did some compute become more expensive in 2026?

One market measure moved against the familiar trend. SemiAnalysis’s index for one-year H100 rental contracts rose from $1.70 per GPU-hour in October 2025 to $2.35 in March 2026, an increase of 38 percent. The index covers negotiated rental contracts for one chip type and term. It does not show that all AI compute became 38 percent more expensive.

The movement is still useful. H100s were introduced in 2022, so newer hardware would normally push their rental price down. Strong demand, booked capacity and higher component costs pushed in the other direction.

What happens next is disputed. Dwarkesh Patel argues that frontier-lab revenue has been growing much faster than compute supply. If increasingly capable models earn more per unit of compute, labs can bid more for scarce capacity. Epoch AI reaches a related conclusion from estimated supply and demand: inference capacity at a fixed model size is growing about 3.4 times a year, while its estimate of potential demand grows much faster. Both are forecasts built on uncertain assumptions. They support a risk scenario, not a reliable price curve.

06Why can the same capability keep getting cheaper?

New hardware is only part of the answer. Models become smaller and more efficient, serving software increases throughput, and providers route simpler work to cheaper systems. The result is a rapid decline in the lowest price for a defined level of benchmark performance.

Epoch AI found that the lowest price to match GPT-4’s March 2023 score on the MMLU benchmark fell from $37.50 to $0.18 per million tokens by February 2025. The result is benchmark-specific and combines input and output prices in a fixed ratio; it should not be read as the price of every task GPT-4 could perform. Across the benchmarks Epoch studied, the rate of decline varied widely.

Provider costs have also fallen. Google reported reducing Gemini serving unit costs by 78 percent during 2025 through model, efficiency and utilisation improvements. A cheaper unit does not guarantee a lower total bill, however. More use, longer context, reasoning and retries can absorb the saving.

Agents make that demand effect concrete. Anthropic reports that, in its own research system, agents used about four times as many tokens as chat interactions, and multi-agent systems about fifteen times as many. Those ratios describe one system, not every agent. The general point holds: the cost of an agent depends on how it works through a case, not only on the provider’s price per million tokens.

This resolves the apparent contradiction. Yesterday’s capability can become cheaper while the newest capability and the hardware it needs remain scarce. A company can benefit from the first trend without trying to predict the second, provided it can measure quality and change models safely.

07How should a company budget for agent compute?

Start with the unit the business cares about: a completed case at the required quality. Token volume is an input to that number. It is not the result.

This is the operating discipline we use at Aitonomy. The business case is set per agent role before the build decision. In production, Aitonomy Control provides the managed runtime, governance, monitoring and traceability needed to compare actual work with that baseline. The cost data available still depends on the model provider and integration, so the monthly judgment uses completed cases, quality, exceptions and full cost rather than token volume alone.

The same discipline supports model changes. An agent role should move to a cheaper model only when the evidence shows that it still meets the required quality and supervision level. That is part of running an agent in production, not a one-off procurement choice.

For budgeting, assume neither that AI will keep getting cheaper nor that scarcity will keep making it dearer. Measure the cost of the work, keep the model choice open, and update the case with operating evidence.

Common questions