Measuring the Carbon Footprint of Multi-Agent Reasoning Pipelines

Introduction

The shift from single-call LLM inference to multi-agent reasoning - debate, tree search, recursive critique, planner-executor decomposition - has produced measurable accuracy gains on hard benchmarks. These gains arrive with a less-discussed bill: the inference compute (and therefore energy and emissions) per task can rise by an order of magnitude or more. Despite a growing literature on training-time emissions [Strubell et al. 2019; Patterson et al. 2021], the carbon cost of agentic inference at production scale has received little quantitative attention.

This paper measures the energy and CO2-equivalent emissions of 11 multi-agent reasoning pipelines across 3 datacenter regions and proposes a carbon-aware routing policy that preserves most of the accuracy benefit at a fraction of the marginal emissions.

Threat Model and Scope

We consider deployment-time emissions only: inference electricity at the serving site, weighted by the marginal generation mix. We exclude embodied hardware emissions, network transit, and client-side energy. Let $E_{\text{task}}$ denote the energy used to complete a single benchmark task. Then

$$\text{CO}$$2(\text{task}) = E{\text{task}} \cdot \text{PUE} \cdot I_g

where $\text{PUE}$ is the datacenter power usage effectiveness and $I_g$ is the grid carbon intensity (gCO2eq/kWh).

Method

Pipelines

We instrumented eleven configurations: a single-call baseline ($B$); self-consistency with $k \in {3, 5, 9}$ samples; two-agent debate (3 rounds); planner-executor; tree-of-thought ($d=3, b=4$); recursive critic; and three hybrid variants.

Measurement

For each pipeline we logged input/output token counts, wall-clock latency, and per-GPU energy via NVML on a controlled in-house cluster (8x H200, PUE 1.18). For the two cloud regions we used published per-token energy estimates [Luccioni et al. 2024] cross-validated against our local measurements (Pearson $r = 0.94$).

We ran each configuration on a 1,200-question subset of MMLU-Pro and a 480-question subset of GPQA-Diamond. Carbon intensities were sampled hourly from ElectricityMaps over a 21-day window.

Results

Cost-Accuracy Frontier

Across both datasets the multi-agent configurations spanned a 3.4x-27.1x energy-overhead band relative to baseline. Tree-of-thought ($d=3, b=4$) had the highest absolute emissions (median 184 gCO2eq per GPQA question in a high-intensity grid) and the highest accuracy (+11.2 points over $B$). Self-consistency $k=5$ recovered 78% of that accuracy delta at 19% of the marginal emissions.

Diminishing Returns

Let $a(n)$ denote accuracy as a function of agent count $n$. We fit

$$a(n) = a_0 + \alpha , (1 - e^{-\beta n})$$

and found $\hat{\beta} = 0.41$ on MMLU-Pro and $\hat{\beta} = 0.29$ on GPQA. The marginal accuracy per additional agent falls below 0.4 percentage points by $n = 6$ on both datasets.

Carbon-Aware Routing

We propose a simple per-question router that escalates to a more expensive pipeline only when a confidence proxy (token-level entropy of the baseline) exceeds a threshold $\tau$.

def route(question, baseline_out, threshold=1.6):
    if baseline_out.entropy < threshold:
        return baseline_out  # cheap path
    return run_pipeline("self_consistency_k5", question)

With $\tau = 1.6$ the router preserves 71% of the accuracy gain over $B$ while incurring only 22% of the marginal emissions of always-on self-consistency. We bootstrapped 95% CIs over 1,000 resamples; the savings were significant at $p < 0.001$.

Discussion and Limitations

Our cluster measurements may not generalize to MoE-dispatch GPUs or to providers that aggressively batch heterogeneous workloads. Estimates rest on hourly grid intensity, which can underweight short emission spikes. We have not modeled cooling water consumption, an increasingly material concern in arid regions.

A broader concern is rebound: cheaper carbon-aware pipelines may simply be invoked more often, erasing the savings. Carbon budgets at the agent or organization level may be needed.

Conclusion

Multi-agent reasoning is not free. Reporting accuracy without reporting emissions risks a Pareto regression for the field. We advocate publishing per-task gCO2eq alongside accuracy on standard benchmarks and present a routing heuristic that captures most of the accuracy benefit at a fraction of the carbon cost.

References

1. Strubell, E. et al. (2019). Energy and Policy Considerations for Deep Learning in NLP.
2. Patterson, D. et al. (2021). Carbon Emissions and Large Neural Network Training.
3. Luccioni, A. et al. (2024). Power Hungry Processing: Watts Driving the Cost of AI Deployment.
4. Wang, X. et al. (2023). Self-Consistency Improves Chain-of-Thought Reasoning.
5. Yao, S. et al. (2023). Tree of Thoughts: Deliberate Problem Solving with LLMs.