1,057
LLM calls (Ollama), 30 days · ~35 / day
A small always-on box at home runs local LLMs, image generation, and speech-to-text. So is it greener than hitting a cloud model? The honest answer is "it gets greener the more you use it" — and it already holds some real advantages the cloud can't match. Here's the math, built from this box's own usage logs, plus the cheap wins that close the gap.
Charge every query its share of the 24/7 idle draw. With only ~35 queries a day, each one carries a big slice of the always-on baseline — so this number drops sharply the moment the box gets busier.
≈ 181× a cloud text prompt.
Count only the extra power the GPU draws while actually generating. Fairer if the box would be running regardless — but still above cloud.
≈ 10.7× a cloud text prompt.
Same hardware, same query — the per-query footprint swings ~17× purely on how you attribute the idle power. Neither number is "wrong"; hiding either is. Both are shown side by side throughout.
Models in rotation: qwen3:8bgemma3:4brocinante-12bqwen2.5-coder:7b — the box is GPU-busy only ~29 min/day out of 24 h. That ~2% utilization means there's huge headroom — it could serve far more work for almost no extra energy.
| Per query | Local — Model Afull attribution | Local — Model Bmarginal | Cloud benchmarkGemini text prompt |
|---|---|---|---|
| Energy | 43.5 Wh | 2.6 Wh | 0.24 Wh |
| Carbon @ 330 gCO₂/kWh grid | 14.3 g | 0.85 g | 0.03 g |
| Water (operational) | 0 mL | 0 mL | 0.26 mL |
| vs. cloud — energy | ×181 | ×10.7 | 1× |
| vs. cloud — carbon | ×478 | ×28 | 1× |
The carbon gap is worse than the energy gap on purpose: the cloud benchmark's 0.03 g implies a very clean ~125 gCO₂/kWh fleet, while this box sits on a ~330 gCO₂/kWh regional grid — a 2.6× dirtier electron on top of using more of them. Water is the one win: the box is air-cooled, so operational water is ~0 mL (indirect grid-generation water is excluded on both sides).
At light use, the baseline cost isn't inference — it's the always-on idle draw, which stays flat whether the box answers 35 queries or 35,000. So every extra query is nearly free, and the per-query footprint only improves as usage grows:
Power figures are estimated from component defaults (RTX 3070 + i5-9600K), not yet measured at the wall. Idle (~55–65 W) is the single most important number to verify with a smart plug — it sets the entire footprint for a lightly-used box.
Marginal cost beats cloud only when average GPU time drops below ~4.5 s/query. It's currently ~49 s — inflated by cold-starts and a 12B model answering trivial prompts. Keep models warm; route small tasks to small models.
To amortize 24/7 idle down to cloud parity under full attribution, the box would need on the order of ~6,400 queries/day — vs. ~35 today. Local hosting earns its footprint only when the box is genuinely busy.
The 3070 (and a free 6700 XT on deck) were reused, so marginal manufacturing carbon is ~0 — a real green advantage a new build wouldn't have (a new mid-range GPU ≈ 150–300 kgCO₂e). Counted as 0 here, deliberately.
Local-vs-cloud is the wrong fight. The biggest win is not running a large model for a trivial task. Most calls here are tiny prompts — many could go to a 4B model (or not run at all) and cut active energy sharply.
Reused hardware → ~0 embodied manufacturing carbon. Air cooling → ~0 operational water. And your data never leaves home. Pair those with the easy efficiency wins above — right-sized models, kept warm — and a self-hosted box becomes a genuinely green, private option that keeps improving the more it's used.
Method — Usage is measured from the AIBox usage dashboard (30-day window: 1,057 Ollama calls, 354,858 tokens, 14.33 h GPU-active). Power is estimated defaults — idle ~60 W, LLM load ~250 W — pending a wall-meter reading; figures will be revised once measured. Two attribution models per the project spec: A (full) charges each query its share of 24/7 idle; B (marginal) counts only load-minus-idle during active time. Carbon = energy × 330 gCO₂/kWh (PA / PJM grid). Water = 0 mL operational (air-cooled). Embodied carbon = 0 (reused hardware).
Cloud benchmark caveat — 0.24 Wh / 0.03 gCO₂ / 0.26 mL is Google's median text prompt (arXiv:2508.15734, Aug 2025; fleet PUE ~1.09). It excludes model training, networking, end-user device, and embodied carbon — so it flatters the cloud side. Treat it as the most-measured estimate with known exclusions, not gospel; some researchers argue it's understated.
Sources — Google, Measuring the Environmental Impact of Delivering AI at Google Scale (arXiv:2508.15734). TechTarget, Environmental impact of LLMs vs. SLMs (Apr 2026) — general LLM query range 0.3–1.0 Wh. arXiv:2601.09527 & arXiv:2501.01990 — self-hosted inference efficiency materializes only at sustained high utilization; embodied + operational carbon modeling.