Can quantum computers fix AI’s energy problem? An audit from the matrix up

Two curves are converging in an uncomfortable way. The compute used to train frontier AI models has been growing four- to five-fold per year for over a decade. The efficiency of the chips it runs on has not. The difference is paid in electricity, and the bills are now large enough to bend national grid planning: the International Energy Agency (IEA) puts data-centre consumption at roughly 415 terawatt-hours (TWh) in 2024 — about 1.5% of global electricity — and projects around 945 TWh by 2030, with AI the dominant driver of the growth.

Whenever an exponential meets a wall, people reach for the most exotic technology on the shelf, and the most exotic thing on today’s shelf is a quantum computer. The intuition is seductive: deep learning is linear algebra at obscene scale, and there exist quantum algorithms that manipulate matrices with complexities classical methods provably cannot match. So — does the energy wall force quantum computing into the AI stack?

This article takes the question seriously enough to answer it properly. First the classical side: where the energy actually goes, why CPU → GPU → TPU (central, graphics and tensor processing units) was a progression of the same idea, and which architectures beyond CMOS (complementary metal–oxide–semiconductor, the transistor technology of every mainstream chip) are genuinely in sight. Then the quantum side: what HHL (the Harrow–Hassidim–Lloyd algorithm), QSVT (quantum singular value transformation), quantum kernels and quantum transformers actually do, what their fine print says, what Tang-style dequantisation took away — and then the audit: line up a real frontier model’s matrices against the conditions a quantum speedup requires, and see what survives. It ends with the error-correction bill, because on this blog it always does.

The energy bill, in numbers

Start with training. Meta’s Llama 3.1 405B — useful here because everything about it is public — consumed about 3.8×10²⁵ floating-point operations (FLOP) over 30.84 million hours on Nvidia H100 GPUs (graphics processing units). At the H100’s 700 W board power that is roughly 22 gigawatt-hours (GWh) for the GPUs alone, before cooling and power delivery overheads — on the order of the annual electricity of five thousand European households, spent once, to produce one artefact. GPT-4-class models are estimated at a comparable 2×10²⁵ FLOP, and the frontier has moved well past both.

Training is the headline, but inference is the bill. A single ChatGPT-style query costs on the order of 0.3 watt-hours (Wh) — trivial alone, but multiplied by billions of queries a day, served continuously, the lifecycle energy of a deployed model is dominated by inference within months. The energy problem of AI is not one heroic training run; it is the steady-state cost of running matrix multiplications as a public utility.

Where the energy actually goes: arithmetic is cheap, bytes are not

Here is the number that reframes the whole discussion. Mark Horowitz’s much-cited energy accounting for a 45 nm chip puts a 32-bit floating-point multiply at about 3.7 picojoules (pJ) — and a 32-bit read from off-chip DRAM (dynamic random-access memory) at about 640 pJ. Moving a word costs roughly 170× computing with it, and the ratio has grown at modern process nodes, where interconnect scales worse than transistors.

AI is not energy-hungry because multiplication is expensive. It is energy-hungry because von Neumann machines pay rent on every byte, and a neural network’s working set is measured in hundreds of gigabytes.

The standard way to make this quantitative is arithmetic intensity: FLOPs performed per byte moved, I = FLOPs / bytes. Every processor has a break-even intensity — peak compute divided by memory bandwidth — below which its arithmetic units idle, waiting for memory. For an H100 that is roughly 989×10¹² FLOP/s ÷ 3.35×10¹² bytes/s ≈ 295 FLOP per byte.

Now compute the intensity of the workload. Generating one token from a 405-billion-parameter model at batch size 1 performs about 2 FLOPs per parameter — 8.1×10¹¹ FLOP — while reading every weight once: 405 GB at FP8 (an 8-bit floating-point number format). That is an intensity of about 2 FLOP per byte, more than a hundredfold short of break-even. The arithmetic for that token takes under a millisecond of H100 compute; streaming the weights past it takes over a hundred milliseconds of memory time. The multipliers sit dark. Batching, caching and speculative decoding exist to claw this back, and they help — but the KV-cache (the attention layers’ stored key–value tensors) grows with context length and the attention matrix grows quadratically with it, so the workload keeps regenerating the problem the tricks fix.

Two more limits frame the classical picture from below and above. From below, Landauer’s principle prices an irreversible bit operation at kT ln 2 (Boltzmann’s constant times temperature) ≈ 2.9×10⁻²¹ J at room temperature; today’s arithmetic sits five to six orders of magnitude above it, so physics is not the binding constraint — CMOS is. From above, the efficiency trend that used to absorb demand growth has flattened: Koomey’s law — computations per kilowatt-hour (kWh) doubling every 1.6 years across the twentieth century — has slowed to a doubling roughly every 2.6 years since Dennard scaling ended. Demand grows ~4× per year; efficiency grows ~1.3× per year. The gap between those two exponents is the energy problem.

CPU, GPU, TPU: three shapes for the same integral

The progression of AI hardware to date is not a progression of faster transistors. It is a progression of architectures that spend a larger fraction of their die and their watts on the only two things a neural network does: multiply-accumulate (MAC — one multiplication plus one addition), and feed the multipliers.

The TPU’s systolic array is the purest expression of the lesson: since bytes cost 170× FLOPs, architect the chip so each byte is touched as many times as the mathematics allows. Google’s published comparisons attribute most of the TPU’s per-watt advantage over contemporaneous GPUs to exactly this — not better transistors, less movement. The same lesson drives the precision collapse: FP32 to BF16 to FP8 to FP4 (32-, 16-, 8- and 4-bit number formats) halves the bytes per operand each step, and a 4-bit multiply costs quadratically less energy than a 32-bit one. Neural networks tolerate it because they were never numerical analysis; they are statistics.

But notice what the whole progression concedes: CPU, GPU and TPU are one architecture — stored program, digital CMOS, von Neumann — with progressively more honest accounting. Each step recovers a constant factor. None changes the exponent of anything. When the workload grows 4× per year, constant factors buy quarters, not decades.

Is there a next machine? The classical candidates

Three technologies are regularly nominated as the successor. Two of them attack the actual problem — the cost of moving bytes. The third is routinely put in the same list and does not belong there.

1. Photonic processors. A mesh of Mach–Zehnder interferometers performs a matrix-vector multiply as light propagates through it — the multiplication is the physics, at close to zero marginal energy per MAC, and bandwidth is optical. The catch is at the edges: getting numbers in and out costs digital-to-analogue and analogue-to-digital conversion at every boundary, and the conversion energy plus laser wall-plug efficiency eats most of the theoretical win. Precision is analogue-limited to roughly 8 bits — tolerable for inference, hostile to training. Photonics is already commercially real where no conversion is needed: as interconnect, moving bytes between chips. As a compute substrate it remains a credible inference play, not a general successor.
2. Neuromorphic and in-memory computing. If moving data to the arithmetic is the cost, put the arithmetic in the memory. Memristor and PCM (phase-change memory) crossbars compute a MAC by Ohm’s law and Kirchhoff’s current law — the weight matrix physically is the resistor array, and a matrix-vector product is one analogue read. Digital near-memory designs make the same move conservatively: IBM’s NorthPole keeps all weights on-die next to the compute and reports an order-of-magnitude energy advantage over contemporaneous GPUs on vision inference. Spiking chips like Intel’s Loihi 2 add event-driven sparsity — energy spent only when a neuron fires. The shared weakness: analogue drift, device variability, and a software stack that assumes none of today’s tooling. These are real engineering programmes with silicon in the field, and they are the honest answer to “what comes after the GPU for inference.”
3. Quantum computing. Not a faster matrix engine, and it is a category error to slot it in this list — though everyone does, so let us do it carefully. A quantum computer does not reduce the energy per MAC; per physical operation, with cryogenics and control electronics, it is less energy-efficient than CMOS by orders of magnitude. Its only currency is complexity class: for certain problems it performs asymptotically fewer operations. Whether that currency is redeemable against deep learning’s linear algebra is a precise technical question — and the rest of this article.

Quantum linear algebra: what is actually on offer

The quantum claim deserves to be stated at full strength before it is audited, because at full strength it is remarkable.

A register of n qubits holds a vector in a 2ⁿ-dimensional complex space. A vector x ∈ ℝ512 that costs a classical machine 512 numbers can be carried in the amplitudes of 9 qubits as |x⟩ = Σᵢ xᵢ|i⟩/‖x‖. Every gate applied to those 9 qubits acts on all 512 amplitudes simultaneously. Exponential compression of the state, and linear algebra on the compressed representation directly — that is the raw material.

HHL: solving a linear system in log time

The 2009 Harrow–Hassidim–Lloyd algorithm made the raw material concrete. Given a system Ax = b with A an N×N Hermitian matrix that is s-sparse (at most s non-zeros per row) with condition number κ (ratio of largest to smallest singular value), HHL prepares the quantum state |x⟩ ∝ A⁻¹|b⟩ in time

O(log(N) · s²κ² / ε)

against the best general classical iterative solver — conjugate gradients — at O(N · s · √κ · log(1/ε)) for positive-definite systems. The dependence on the dimension N falls from linear to logarithmic. For N = 1012, that is the difference between a trillion steps and about forty. Later refinements (Ambainis; Childs–Kothari–Somma) improved the κ and ε dependence; the logarithmic-in-N core is the headline.

QSVT: the general theory

The modern formulation, quantum singular value transformation, is worth knowing because it subsumes nearly everything else. Embed a (suitably normalised) matrix A in the top-left corner of a bigger unitary U — a block encoding. QSVT then applies an essentially arbitrary degree-d polynomial p to the singular values of A, producing a block encoding of p(A), using O(d) invocations of U and one ancilla qubit. Choose p(x) ≈ 1/x and you have a linear solver — degree O(κ log(κ/ε)) suffices, which is where HHL’s modern O(sκ · polylog) complexity comes from. Choose p(x) ≈ eitx and you have Hamiltonian simulation. Choose a step function and you have projection and amplitude amplification. One framework — Martyn, Rossi, Tan and Chuang call it the “grand unification” — with a single price list: the cost of the block encoding, times the degree of the polynomial.

On top of this substrate sit the proposals aimed squarely at machine learning. Quantum kernel methods map data through a quantum circuit into a 2ⁿ-dimensional feature space and let a classical support-vector machine (SVM) work on the resulting kernel matrix — with a rigorous (if contrived) exponential separation on a discrete-logarithm-based dataset. Parametrised quantum circuits (“quantum neural networks”) train circuit parameters by gradient descent, the direct quantum analogue of deep learning. And quantum transformers compile the full attention–feedforward stack into QSVT primitives: Guo et al. showed the architecture can, in principle, run end-to-end in time polylogarithmic in the embedding dimension.

On paper, this is exactly the technology the energy wall is asking for: the dominant cost of AI scales linearly-to-quadratically in dimensions that a quantum computer touches logarithmically. Now the audit.

Read the fine print: four gates

Scott Aaronson’s 2015 caution about HHL-type algorithms — read the fine print — compresses to four conditions. Each is a gate between the paper complexity and a real speedup, and each must be passed simultaneously.

1. Loading. The complexity O(log N · …) starts after |b⟩ exists and A is block-encoded. If your matrix is a big table of learned coefficients sitting in DRAM, encoding it generically requires either a state-preparation circuit of depth O(N) or a quantum random-access memory (QRAM) — and the standard bucket-brigade QRAM construction uses O(N) quantum hardware cells. Either way, something scales with N, and the exponential advantage is spent before the algorithm begins. The gate passes only when the data is implicit: a matrix defined by a formula, a circuit, a Hamiltonian, a graph structure — description length polylog in N.
2. Conditioning and sparsity. The solver costs scale with sκ (modern) or s²κ² (original). A dense matrix has s = N: substituting it back erases the log. Block-encoding a dense matrix via its Frobenius norm costs a subnormalisation α = ‖A‖_F that inflates the effective κ the same way. The gate passes for sparse or highly structured matrices with κ = O(polylog N).
3. Readout. The algorithm delivers |x⟩, a quantum state — not x, a vector. Reading out all N amplitudes is tomography at Ω(N/ε²) measurements, which again erases the log. The gate passes when you only need functionals of the solution: one expectation value ⟨x|M|x⟩, a sample from the distribution |xᵢ|², an overlap with a reference state.
4. Repetition. Anything consumed per-example, per-token, per-layer multiplies the whole pipeline — including loading and readout — by the count. A subroutine called 10¹² times inherits none of the asymptotic grace of a subroutine called once.

The variational proposals face their own gate: barren plateaus. For wide classes of parametrised circuits the training gradient concentrates exponentially to zero with qubit count, and quantum kernels suffer the mirror-image disease — kernel entries concentrate exponentially, so estimating them takes exponentially many shots. Worse, recent theory suggests a bind: many of the circuit families provably free of barren plateaus are classically simulable. Trainable tends to mean simulable; expressive tends to mean untrainable. The escape routes between those two failure modes are an open research question, not an engineering roadmap.

The Tang test: what dequantisation took away

In 2018, Ewin Tang — then an 18-year-old undergraduate — examined the quantum recommendation-systems algorithm, one of the strongest candidates for exponential QML advantage, and found the speedup was not quantum at all. Given the same kind of access the quantum algorithm quietly assumed (the ability to sample rows and entries with probability proportional to their norms — the classical shadow of amplitude encoding), a classical randomised algorithm solves the same problem with runtime polylogarithmic in the matrix dimensions — polynomial in the rank, condition number and precision, but not in N.

The technique generalised fast. The Chia–Gilyén–Li–Lin–Tang–Wang framework dequantised quantum PCA, supervised clustering, support-vector machines, low-rank Hamiltonian simulation and low-rank linear solvers. The pattern of the boundary it drew:

A common misreading, worth correcting because it flips the conclusion: Tang’s results do not dequantise the sparse case — sparse, high-rank, well-conditioned systems are exactly where the quantum exponential still stands (linear-system solving in full generality is BQP-complete — BQP, bounded-error quantum polynomial time, being the class of problems quantum computers solve efficiently, and complete meaning as hard as any of them — so a classical polylog algorithm there would collapse complexity classes). What Tang dequantised is the low-rank regime. The fair summary: quantum computers are exponentially better at sparse high-rank linear algebra nobody can load data into; classical randomised algorithms are just as good at the low-rank linear algebra you actually can.

The practical sting for machine learning specifically: low-rank structure is not an edge case of learned models — it is their signature. Which brings us to the audit.

The audit: a frontier model’s matrices against the four gates

Take the matrix constellation of Llama 3.1 405B as the concrete, fully public stand-in for the frontier of large language models (LLMs): 126 transformer layers, model dimension 16,384, feed-forward dimension 53,248, 128-way attention with grouped key–value (KV) attention heads, context up to 128k tokens. Per layer, the feed-forward block alone holds dense weight matrices of 16,384 × 53,248 ≈ 8.7×10⁸ entries; the model totals 4.05×10¹¹ learned parameters, trained in BF16 and served at FP8 or below. Run each property through the gates:

A frontier LLM’s linear algebra is dense, low-precision, effectively low-rank, explicitly data-defined, and consumed in full at every step. Each of those five properties is individually the wrong one. The workload is not merely a poor fit for quantum linear algebra — it is the constructed counterexample.

The quantum-transformer literature, to its credit, says much the same when read carefully. The polylog runtime of the Guo et al. construction is polylog given block-encoded inputs; and PennyLane’s own analysis of the proposal identifies the embedding bottleneck as decisive — a transformer at embedding dimension ~10⁴ needs on the order of 10⁴ high-quality logical qubits before error correction, hardware that on any published roadmap is one to two decades out. We return to what “before error correction” conceals in a moment.

Five years out: which wall arrives first?

The question posed at the start was whether classical methods hit a wall soon enough that quantum computing becomes required. Project both curves to ~2031 and the answer is unambiguous.

On current trend (Epoch AI’s 4–5×/year), frontier training runs reach 10²⁸–10²⁹ FLOP around 2030 — three to four orders of magnitude past GPT-4 — demanding campuses drawing 1–5 gigawatts (GW): individual training runs consuming the output of a large power station for months. The walls that bind first are power procurement, HBM supply, and capital — runs costing tens of billions of dollars. These are economic walls, and the classical mitigations arriving against them are of exactly the byte-moving kind the first half of this article predicts: FP4 arithmetic, mixture-of-experts sparsity (a ~2-trillion-parameter model activating a twentieth of itself per token), co-packaged optical interconnect, near- and in-memory compute, and inference-time distillation. Constant factors, all — but multiplied together they buy the decade.

Against that, the quantum timeline on the supply side: IBM’s roadmap targets ~200 logical qubits in 2029 and ~2,000 after 2033; Google frames a useful error-corrected machine around the end of the decade. Even taken at face value, nothing on any published roadmap reaches within several orders of magnitude of the ~10⁴ logical qubits the most modest quantum-transformer sketch requires — and the demand side, per the audit above, fails four gates independently of hardware. A wall that is made of energy economics cannot be broken by a machine that does not reduce energy per operation. Within five years, quantum computing is not required for AI, not available for AI, and — on the merits of its own theory — not applicable to AI’s dominant workload. Classical AI will not hit a wall that quantum removes, because the walls it is hitting are not made of complexity theory.

The error-correction bill, and the crossover arithmetic

Suppose every gate above were somehow passed. There remains the bill this blog keeps itemising: quantum operations are not free, and error-corrected quantum operations are slow.

The arithmetic is short. A surface-code cycle on superconducting hardware takes ~1 µs (microsecond); a logical operation at the code distances useful algorithms need (d ≈ 27, ~1,500 physical qubits per logical qubit at Willow’s measured error-suppression factor Λ) spans tens of cycles, and non-Clifford gates — the gate type any universal algorithm needs — consume magic states from dedicated factory circuits that dominate the floor plan. A generous end-to-end estimate is ~10⁵ logical operations per second. An H100 executes ~10¹⁵ FLOP/s; a modest training cluster of ten thousand of them, ~10¹⁹. The clock disadvantage is ten orders of magnitude at the socket, fourteen against the cluster — and it must be repaid from asymptotic savings before a single joule is saved. That repayment schedule is brutal for small polynomial advantages:

This chart retires the most repeated number in quantum-for-AI discourse: that a Grover-type quadratic speedup would compress a three-month GPT-4 training run to hours. Run the arithmetic honestly: a quadratic speedup on ~2×10²⁵ FLOP of classical work means ~4.5×10¹² quantum operations, which at 10⁵ error-corrected logical ops per second is about a year and a half — five times slower than the cluster it was supposed to replace, and that is before any of the four gates above is priced in. Quadratic speedups are not a discount on classical work; they are a bet that only pays on jobs classical machines would run for decades. For error-corrected quantum advantage the workload needs quartic-or-better scaling — or an exponential separation, where the crossover collapses to trivial sizes. That is precisely why our own MERIQ assessment reserves its top advantage grade for proven super-quadratic speedups, and why the use case we build on stacks Grover on top of amplitude estimation to reach quartic end-to-end.

And the physical footprint, for completeness: the ~10⁴ logical qubits of the minimal quantum-transformer sketch, at the ~1,500 physical-per-logical of a 10⁻⁶ surface-code qubit, is ~15 million physical qubits before magic-state factories — against 105 on the best current chip, with per-gate energy dominated by a dilution refrigerator drawing ~25 kilowatts (kW). Quantum computing will not be the low-power option for anything whose classical competitor is a well-fed GPU. It wins only where the classical competitor is an exponential.

Where quantum genuinely enters the AI story

None of the above says “quantum computing is useless for machine learning.” It says the frontal assault — replacing the dense linear algebra of deep networks — is the one application the theory itself rules out. The survivors are narrower and more interesting:

1. Quantum data. Every gate in the audit assumed classical data that must be loaded. When the input is already quantum — states from a quantum sensor, a chemistry simulation, a materials Hamiltonian — the loading gate vanishes and provable separations exist: learning from quantum experiments can require exponentially fewer samples than any classical strategy. Machine learning on quantum systems, not instead of GPUs.
2. Structured cores inside classical pipelines. Sparse, implicitly defined, well-conditioned linear systems — PDE (partial differential equation) discretisations, lattice models, network flows — where the answer needed is an observable, not a vector. These pass all four gates by construction. They live in scientific computing and simulation-heavy machine learning, not in transformers.
3. Optimisation by different mathematics. Decoded Quantum Interferometry is the interesting recent datapoint: it attacks structured discrete optimisation (max-XORSAT-type problems: satisfy as many exclusive-or equations of a linear system as possible) by converting them to decoding, a mechanism with no classical analogue, and claims super-polynomial advantage over known classical methods for optimal polynomial intersection. It is not an AI-training subroutine — but AI-adjacent pipelines are full of discrete optimisation, and DQI is evidence that quantum optimisation can find advantage on structure, where Grover-style unstructured search cannot.
4. Sampling. Sampling from complex distributions is the one task where quantum devices already beat classical supercomputers by construction. Generative modelling and Boltzmann-style training consume samples; whether a useful distribution with this property exists inside ML is open — but it is the right shape of question.

Note what unites the survivors: in every case the quantum computer is aimed at a problem that is exponentially hard classically, and the data question is answered by construction rather than by QRAM. That is the same filter our MERIQ framework applies to any proposed use case — and generic “quantum machine learning (QML) on classical data” is the canonical example of a proposal that fails its data-basis criterion at screening, before hardware is even discussed.

An honest position, mid-2026

AI’s energy problem is real, structural, and classical. It is a data-movement problem running into the end of free transistor scaling, and the technologies that address it in the next five years attack bytes, not FLOPs: lower precision, sparsity, optical interconnect, near- and in-memory compute. Those are constant-factor answers, but the wall is made of constant factors.

Quantum linear algebra is real, remarkable, and aimed elsewhere. Its exponential advantages are confined — by theorems, not by engineering immaturity — to matrices that are sparse, well-conditioned, implicitly specified, and consumed as observables. A frontier language model’s matrices are dense, effectively low-rank, explicitly learned, and consumed in full, a trillion times a day. Tang-style dequantisation claims the low-rank regime for classical randomised algorithms; the error-correction clock taxes what remains by ten orders of magnitude and makes sub-quartic speedups economically void. No plausible five-year development changes any term in that sentence.

So the answer to the title is: no — and knowing why is worth more than the hype it replaces. The energy wall will be managed by classical architecture. Quantum computers will earn their place in the computational stack the way every specialised machine does: on the problems whose structure matches theirs — simulation, structured optimisation, sampling, quantum data — a territory that is smaller than “all of AI” and still large enough to matter enormously. Sorting which concrete problems fall inside that territory, with scored criteria instead of vibes, is exactly the assessment work we do at JoS QUANTUM.