Why does AI need graphics cards?

A language model is a very large set of numbers, its parameters, arranged in tables called matrices. The calculation that follows applies to a so-called dense model, which uses all its parameters for each word; mixture-of-experts models use only some of them, for example 13 billion out of 47 for Mixtral 8x7B from Mistral AI. To write a word, or more precisely a piece of a word, the model passes the preceding text through these tables: each parameter is multiplied by a number, then the result is added to the others. The Nvidia guide to this calculation sums it up as follows: each multiplication followed by an addition counts as 2 operations.

For a model of 70 billion parameters, this comes to about 140 billion operations per word. These operations are independent of one another and can be carried out at the same time. This is the work that graphics cards were designed for, originally to calculate the points of an image in parallel, and which they now do for neural networks.

The Nvidia H100 card, in its SXM version fitted 8 to a board in AI servers, carries a chip of 80 billion transistors, etched by TSMC. It has 16,896 compute cores and 528 cores specialised in matrix products, which Nvidia calls Tensor Cores. Its Tensor Cores perform up to 989,000 billion operations per second on 16-bit numbers, and twice that on 8-bit numbers. At the start of 2025, the generation of the H100 and its variants accounted for 77% of the computing power of all Nvidia AI chips in service, according to Epoch AI.

A computer or server processor can chain together varied tasks, but it carries out far fewer operations side by side. For a neural network, where the same operations are repeated on billions of numbers, the number of operations carried out at the same time is what counts.

To write each word, the card must read all the parameters of a dense model back from its memory. An H100 carries 80 GB of HBM3 memory, stacked around the chip, which it reads at 3.35 terabytes per second. A model of 70 billion parameters stored on 16 bits, or 2 bytes each, occupies 140 GB: it needs at least 2 cards.

Our calculation, as an order of magnitude, for a single request at a time on these 2 cards: reading the 140 GB takes about 21 thousandths of a second, and performing the 140 billion operations takes about 70 millionths of a second. The card therefore waits for its memory about 300 times longer than it computes. The Nvidia guide states that a matrix-vector product is always limited by memory; this is the shape of the computation when writing a single word for a single request.

This is why AI services group together the requests of many users: a single reading of the parameters then serves to write a word for each of them, and the electricity per request falls. Our analysis of AI installed on your own computer measures this effect. Memory is also the largest manufacturing item of the H100 baseboard according to Nvidia, with 42% of emissions, ahead of the chips, 25%.

An H100 SXM has a rated power of 700 W. It does not work alone: the Nvidia DGX H100 server, which combines 8 of them, consumes at most 10.2 kW with its processors, memory, network and fans, or 1,275 W per card. Measurements published by Microsoft researchers on the previous generation find a power that reaches or exceeds the rated power when the model reads the request, and lower when it writes the response.

The AI Impact Database uses 900 Wh for one card-hour, 70% of the card's share of the server's maximum power, 1,275 W, the average load used by Epoch AI. With cooling and the rest of the data centre, the hour comes to 0.99 kWh, or 0.45 kgCO2e with the world average electricity, as much as 3 km in a petrol or diesel car, including the manufacture of the car, and not counting the manufacture of the server. These are estimates: no measurement of the average consumption of an H100 in service has been published.

In 2025, Nvidia published the manufacturing footprint of its HGX H100 baseboard, which carries 8 cards: 1,312 kgCO2e, from raw material extraction to assembly, or 164 kgCO2e per card. Memory is the largest item, at 42%, ahead of the compute and network chips, 25%. ADEME, with its own model, estimates a single H100 card at 235 kgCO2e, and notes that Nvidia's method is not very transparent.

Hardware

Manufacture of an AI server, by component

Nvidia HGX H100 baseboard with 8 graphics cards, 1,312 kgCO2e, share of each component · Sheet published by Nvidia

Memory42%

Integrated circuits25%

Cooling18%

Assembly9%

Electromechanical4%

Other3%

Source: Nvidia, HGX H100 Product Carbon Footprint · projetcelsius.com/en/ai · CC BY 4.0

PNGCSV

The whole server emits more in manufacture than its baseboard and its 8 cards, according to a simplified GreenIT study. GreenIT estimates the manufacture, distribution and end of life of an entry-level server with 8 H100 cards at 8,341 kgCO2e, about 1,000 kg per card. Spread over 5 years of service at 69% utilisation, this manufacture represents about 35 gCO2e per card-hour, or about 8% more than the electricity for that hour with the world average electricity, and 17% with that of the European Union.

Epoch AI estimates, from Nvidia's financial results, that the company shipped about 4.5 million data centre cards in 2025, compared with 1.3 million in 2023. The maximum power of the cards shipped in 2025 alone reaches 4.9 GW, not counting servers or cooling. These are estimates, with an uncertainty that is often a factor of 2. Greenpeace East Asia estimates the electricity consumed to manufacture AI chips at 218 GWh in 2023 for Nvidia alone, and at 983.9 GWh in 2024 for Nvidia and AMD; the chart of electricity used to manufacture chips shows these 2 years and the share of the increase that comes from adding AMD.

For us at Projet Celsius, the graphics card illustrates 2 features of the footprint of AI. The first: with the world average electricity, the electricity of one card-hour emits about 13 times more than the share of server manufacture that falls to it, and about 6 times more with that of the European Union; the size of the model and the number of requests served set the number of cards used. The second: memory limits the writing speed, and it is also the largest manufacturing item of the H100 baseboard according to Nvidia (42%) as well as of the next generation, the HGX B200 (49%), whose manufacture emits 1.7 times more. These figures come from Nvidia's sheets, the only ones published for these cards.