AI Energy and Environment: Estimating, Accounting For and Reducing the Footprint

AI runs on electricity, and at today's scale that is a public question, not only an engineering one. Utilities plan grid upgrades around new data-center campuses, companies publish sustainability reports, regulators ask for disclosures, and customers ask "what is the footprint of this feature?" A professional AI engineer should be able to produce a defensible estimate, explain its assumptions, and know which design choices actually move the number.

Intuition: building the factory vs running the delivery fleet

Think of training as building a factory: a large, one-time burst of energy concentrated in weeks or months. Think of inference as running the delivery fleet that the factory supplies: each delivery (a query) uses a small amount of energy, but the fleet runs every day for the life of the product. Whether the factory or the fleet dominates depends on how many deliveries you make. For a popular model, the daily fleet bill, summed over a year or more of deployment, can exceed the construction bill.

Two more pieces complete the picture. The building that houses the machines has overhead: cooling, power conversion, lighting (captured by PUE). And the fuel mix of the local grid decides how much carbon each kilowatt-hour carries. The same workload can differ in emissions by a factor of ten or more depending on where and when it runs.

Where the energy goes

Training

Training energy scales with total compute. For a dense transformer, compute is about $6ND$ FLOPs ($N$ parameters, $D$ tokens), so doubling either doubles energy at fixed hardware efficiency. Training energy also includes failed runs, hyperparameter searches and ablations, which can be a large multiple of the final run and are rarely reported.

Inference

Inference energy per request depends on model size (active parameters), prompt and output length, batching efficiency and hardware. Reasoning models that generate thousands of hidden "thinking" tokens per answer use proportionally more energy per query. Total inference energy is per-request energy times request volume, and volume grows with adoption.

Beyond the accelerator

The GPU is not the only consumer. Servers include CPUs, memory, NICs and fans; racks include switches; the facility adds cooling and power-conversion losses. Published per-chip thermal design power (TDP) figures are a ceiling for the chip, not the energy of the job.

The core method: power times time

Energy is power integrated over time. For a steady workload:

where $\bar{P}$ is the average power of the IT equipment (servers and network) in kilowatts and $t$ is duration in hours, giving kilowatt-hours (kWh). The facility uses more:

Power usage effectiveness (PUE) is total facility energy divided by IT equipment energy. A PUE of 1.0 would mean zero overhead. Modern hyperscale facilities with efficient cooling achieve values close to 1.1; older enterprise server rooms often run far higher. PUE is an annual average and says nothing about how efficient the IT equipment itself is.

Operational emissions follow from the grid's carbon intensity $g$ (kg CO2-equivalent per kWh):

Water has two components. On-site water is mostly evaporated in cooling towers, measured by water usage effectiveness (WUE) in liters per kWh of IT energy. Off-site water is consumed at power plants generating the electricity, and depends on the generation mix.

Worked example 1: a training run

A team trains a model on 128 nodes, each with 8 GPUs (1,024 GPUs total), for 30 days. Measured average node power, including CPUs, memory and networking, is 8 kW (below the nameplate maximum because GPUs are not at full TDP every millisecond). Facility PUE is 1.2.

1. IT power: $128 \times 8 = 1{,}024$ kW.
2. Duration: $30 \times 24 = 720$ h.
3. IT energy: $1{,}024 \times 720 = 737{,}280$ kWh $\approx 737$ MWh.
4. Facility energy: $1.2 \times 737{,}280 = 884{,}736$ kWh $\approx 885$ MWh.
5. Emissions on a grid at 0.40 kg/kWh: $0.40 \times 884{,}736 = 353{,}894$ kg $\approx 354$ t CO2e. On a grid at 0.05 kg/kWh (largely hydro, nuclear or wind): $\approx 44$ t CO2e.
6. On-site water at an assumed WUE of 0.3 L/kWh: $0.3 \times 737{,}280 = 221{,}184$ L $\approx 221$ m³.

The same computation differs by a factor of eight in emissions purely by location. All inputs here are illustrative; in a real report, state each one and its source.

Worked example 2: an inference service

An 8-GPU server draws 6 kW on average while sustaining 20,000 output tokens/s across all users (high batch).

1. Energy per token: $6{,}000 \text{ W} / 20{,}000 \text{ tok/s} = 0.3$ J/token.
2. A 500-token answer: $500 \times 0.3 = 150$ J $= 150 / 3600 = 0.0417$ Wh of IT energy.
3. With PUE 1.2: $0.05$ Wh per answer.
4. At 10 million answers per day: $0.05 \times 10^7 = 5 \times 10^5$ Wh $= 500$ kWh/day, about 183 MWh/year.

Now change one assumption: if the same server only achieves 2,000 tokens/s because of low traffic (small batches), energy per token rises tenfold to 3 J. Utilization is the dominant variable for inference energy, which connects directly to the batching techniques in Inference serving at scale. Prompt tokens (prefill) also cost energy, which this simple per-output-token estimate folds into the measured server power.

Comparing the two examples: 885 MWh for training vs about 183 MWh per year for this service. With higher traffic, longer outputs, or several years of deployment, inference overtakes training. Published per-query estimates vary by orders of magnitude for exactly these reasons, so be wary of any single "energy per query" figure quoted without its assumptions.

Measuring instead of guessing

NVIDIA GPUs (Volta and newer) expose a cumulative energy counter through NVML. The pynvml bindings (installed as nvidia-ml-py) let you read it before and after a workload:

import time
import pynvml  # pip install nvidia-ml-py

def gpu_energy_joules(fn, *args, **kwargs):
    """Run fn and return (result, joules per GPU, seconds)."""
    pynvml.nvmlInit()
    handles = [pynvml.nvmlDeviceGetHandleByIndex(i)
               for i in range(pynvml.nvmlDeviceGetCount())]
    start = [pynvml.nvmlDeviceGetTotalEnergyConsumption(h) for h in handles]  # millijoules
    t0 = time.time()
    result = fn(*args, **kwargs)
    seconds = time.time() - t0
    end = [pynvml.nvmlDeviceGetTotalEnergyConsumption(h) for h in handles]
    pynvml.nvmlShutdown()
    return result, [(e - s) / 1000 for s, e in zip(start, end)], seconds

def footprint(joules, pue=1.2, g_kg_per_kwh=0.4, overhead=1.3):
    """Scale GPU energy to server and facility level; return kWh and kg CO2e."""
    kwh_facility = sum(joules) / 3.6e6 * overhead * pue
    return kwh_facility, kwh_facility * g_kg_per_kwh

Notes on the code: the counter covers the GPU board only, so overhead (here an assumed 1.3) scales to whole-server energy; measure it with rack power meters or the server's management controller if you can. Make sure the GPU work has finished before reading the counter (with PyTorch, call torch.cuda.synchronize() at the end of fn). Open-source tools such as CodeCarbon and Zeus wrap the same idea and add carbon-intensity lookups.

Grid and data-center growth

The International Energy Agency's Energy and AI report (April 2025) is the most cited global baseline. It estimated that data centers used about 415 TWh of electricity in 2024, roughly 1.5% of global electricity consumption, and projected in its base case that this would more than double to about 945 TWh by 2030, comparable to Japan's current annual electricity use. The United States and China account for most of the projected growth (together nearly 80% by the IEA's figures, as reported by Scientific American). AI-focused accelerated servers are the fastest-growing component.

How to read such figures responsibly:

• They are scenarios, not forecasts. The IEA presents ranges that depend on chip efficiency, adoption and power-supply constraints. Use the base case with its range, and cite the edition and date.
• Global share understates local impact. 1.5% of global electricity is modest, but data centers cluster: in some regions they account for a large and rising share of demand, straining transmission, generation adequacy and grid connection queues. Local constraints, not global totals, are what delay projects.
• AI is a subset of data-center demand. Data-center totals include conventional cloud, streaming, storage and enterprise IT. Be careful not to attribute the entire figure to AI.
• Efficiency and demand interact. Each hardware generation delivers more tokens per joule, but cheaper tokens encourage more use (a rebound effect related to the Jevons paradox). Total consumption depends on both.

Water

Cooling large heat loads often uses evaporative systems, which consume water. The effect is highly local: a facility in a water-stressed region matters more than one using reclaimed water in a wet climate. Researchers (Li et al., 2023, "Making AI Less Thirsty") have argued that AI's water footprint should be reported alongside carbon, including the off-site water used to generate electricity. Design trade-offs cut both ways: closed-loop liquid cooling and dry coolers can reduce water use but may increase electricity use in hot weather. Report water by location and season, not just as an annual total.

Carbon accounting

Scopes

The GHG Protocol classifies emissions into:

• Scope 1: direct emissions from owned sources (for example, diesel backup generators).
• Scope 2: emissions from purchased electricity, heat or cooling. This is where most operational data-center emissions sit.
• Scope 3: all other value-chain emissions, including embodied emissions from manufacturing chips, servers and buildings, and, for a cloud customer, the provider's emissions attributable to their usage.

Location-based vs market-based Scope 2

• Location-based uses the average carbon intensity of the grid where the electricity is consumed.
• Market-based reflects contractual instruments: power purchase agreements (PPAs) and renewable energy certificates. A company can report near-zero market-based emissions while the local grid still burns fossil fuels.

Both are legitimate and both should be disclosed. A stricter approach, 24/7 carbon-free energy matching, compares clean supply and consumption hour by hour rather than over a year, which better reflects the physical grid.

Embodied carbon

Manufacturing accelerators, HBM, servers and buildings has a footprint amortized over hardware lifetime. On a very clean grid, embodied emissions can be a large share of the total, which makes extending hardware life and using it at high utilization doubly important.

Worked example 3: per-run accounting summary

For worked example 1 on a grid at 0.40 kg/kWh, a defensible disclosure might read: "Facility energy 885 MWh (IT 737 MWh, PUE 1.2). Location-based operational emissions 354 t CO2e. Market-based emissions [value] under [named PPA]. Embodied emissions not included." Stating what is excluded is as important as the number.

Efficiency levers

Ordered roughly by typical impact for a team building on existing models:

1. Right-size the model. A distilled or smaller model, or a mixture-of-experts model with fewer active parameters, can cut energy per token several-fold if it meets the quality bar. Route easy requests to small models.
2. Serve efficiently. High batch utilization, prefix caching, speculative decoding and quantization (FP8, INT4) all reduce joules per token.
3. Avoid unnecessary tokens. Shorter prompts, caching repeated answers, limiting reasoning budgets where accuracy does not need them.
4. Choose efficient hardware. Newer accelerators usually deliver more work per joule; older hardware kept in service still carries its embodied carbon already spent.
5. Choose where and when. Run flexible jobs (training, batch inference, evaluation) in regions and hours with low carbon intensity (carbon-aware scheduling).
6. Facility efficiency. Low PUE, liquid cooling, heat reuse and water-aware cooling design.
7. Training discipline. Use scaling laws and small-scale experiments to avoid wasted large runs; checkpoint so failures do not repeat work.

Common pitfalls

• Using TDP times hours as measured energy. TDP is a ceiling; measure or use realistic average power, and include the rest of the server.
• Ignoring PUE, or double-counting it when a cloud provider's figures already include facility overhead.
• Quoting a single energy-per-query number without model size, token counts, batching and hardware.
• Counting only the final training run and omitting experiments, failed runs and inference.
• Reporting only market-based emissions, or mixing location- and market-based figures in one comparison.
• Treating global averages as local reality for grid and water impacts.
• Overstating precision. Report ranges and assumptions; these estimates are often uncertain by a factor of two or more.

In practice

• Instrument: log GPU energy (NVML), node power (management controller or PDU) and tokens served, and compute joules per token per endpoint as a standard dashboard metric.
• Write a short footprint note for major training runs: energy, PUE, grid region, location- and market-based emissions, water if known, and exclusions.
• Use cloud providers' carbon reporting tools as a starting point, but understand their methodology and boundaries.
• Fold energy into model selection: when two models meet the quality bar, prefer the one with lower joules per task.
• Keep claims balanced: AI's energy use is significant and growing, concentrated in specific places, and also highly sensitive to engineering choices you control.