Electricity Maps Blog
January 20, 2025
-
8 minutes
This blog post is part of a series of 7 articles uncovering all you need to know about marginal signals. Are you planning to integrate them into your solution or doing so already? Read more about what you should consider when using them in practice here.
This article is the last of the series and focuses on the challenges of validating these signals. We’ll also explore the discrepancies observed between different providers, and why ultimately these emissions factors cannot be used for tangible emissions reduction.
As we’ve uncovered in recent articles, marginal emissions factors are not suited for scope 2 carbon accounting, and optimizing electricity consumption based on marginal emissions factors may not reduce its carbon footprint even though they could theoretically reduce short-term emissions.
For companies using emissions signals, validation is essential to ensure trust in their accuracy and avoid misleading sustainability claims, especially as public scrutiny continues to intensify. However, as marginal emissions are not measurable, they cannot be validated against a ground truth that prevents tangible emissions reduction.
This blog post illustrates the challenges of validating marginal emissions factors, the discrepancies observed between different providers, and the implications of optimizing based on these factors.
On the surface, marginal emissions are simple: the emissions caused by the power plant ramping up (or down) in response to a change in consumption. We could measure marginal emissions factors if we knew the power plant with spare capacity that will adjust to changes in demand.
But according to PJM, the largest US grid operator, the reality is different. The concept of a marginal plant is true in simple illustrative scenarios but it is no longer valid on actual electricity grids that are much more complex.
The German grid operator 50Hertz opted for average emissions factors in their eCO2grid tool because “determining the correct power plant is extremely complex or even impossible”. Even grid operators, who should be the best positioned to measure marginal emissions factors, claim they can’t.
To validate a signal that is not measurable raises substantial challenges as highlighted by WattTime: “Marginal emissions are not directly measurable [...] Without ground truth, it can be challenging to determine which models are closer to the truth and quantify their accuracy”.
The grid operators go even further and claim that: “It is never possible to find out retrospectively whether the signal is correct”. For that reason, PJM “does not make any guarantees as to the accuracy of the information nor that it is fit for any purpose.”
In practice, this limits the ability to provide meaningful quality guarantees for the signal. At a time when sustainability claims come under heavy scrutiny, verifiability and auditability are key to reducing operational and reputational risk. Auditing a product feature based on marginal emissions is close to impossible and companies have already today been called out by the press and by experts for using marginal emissions.
This approach also jeopardizes efforts to achieve tangible emissions reductions when optimizing electricity consumption based on this signal or developing public-facing applications that rely on it.
Without a ground truth to validate marginal emissions factors, it becomes a challenge to source them given the multitude of methodologies and assumptions used. As highlighted by the Clean Energy Buyer’s Institute in their guide: “Depending on the specific methodology, data sources, and assumptions used, no one estimate is likely to match another. Even two approaches using the same methodology could use different input data or assumptions to estimate marginal emissions impact, leading to two different and sometimes inconsistent estimates. [...] This means that there is no single source of “truth” for marginal emissions impact”.
This conclusion is also shared by WattTime and NREL in their recent paper: “Multiple different models to estimate MEFs exist, but there is no single objective ground truth MEF dataset with which to compare the relative accuracy of two or more such models.”
We illustrate these challenges by comparing marginal emissions factors from three different providers for the same grid (PJM in Mid-Atlantic US) and over the same period (1st to 15th January 2024). The datasets used are:
Not all providers could be included here as some do not have openly available data but we encourage anyone to run the comparison with all datasets available to them.
The four signals are substantially different. The data from Electricity Maps and Avert show much smaller variations while the signal from PJM Data Miner is the only signal that shows negative marginal emissions. A negative marginal emissions factor means that when the grid demand increases, the emissions of greenhouse gases in the atmosphere decrease (or vice versa). This seems inconsistent at first but is an expected behavior of the signal. Singularity's data comes from the PJM marginal fuels posting yet yields different results than the marginal emissions rate published by PJM. It means that two marginal datasets from the same source that also happens to be the grid operator are not consistent.
While this introduces obvious difficulties for sourcing marginal emissions factors and for determining which vendor provides the most accurate estimate, it also raises a more fundamental doubt: Can this signal confidently be used for any purpose? Using the signal of one provider or another to calculate emissions would yield substantially different outcomes and using it for optimization would lead to contradictory operational decisions.
In this section, we reuse the example of January 2024 in the PJM grid to more concretely illustrate the implications of using emissions factors that can’t be validated and reveal significant inconsistencies from one provider to another.
On the 1st of January 2024, an electricity consumer shifting its load with PJM marginal emissions signal would have shifted it from 3 am (blue peak highlighted in brown) to 5 pm (blue low highlighted in yellow). Calculated on PJM marginal emissions, this would have avoided 460kg of CO2 for every megawatt-hour shifted.
According to the data sourced from Singularity API though, this would have actually increased CO2 emissions by 185kg for every megawatt-hour shifted. In other words, had the consumer been making decisions based on Singularity-sourced marginal emissions factors, they would have done the absolute opposite optimization and shifted consumption from times in yellow to times in brown.
Let’s consider another use case with the calculation of emissions avoided by renewable energy generation. If a wind farm was generating 1 MW of clean electricity at 3 am on the 1st of January 2024, it would have avoided 860kg of CO2 over one hour according to PJM. However, this would have avoided only 430kg of CO2 (hence half of it) according to the PJM data sourced from Singularity API. This number of avoided emissions can hardly be used for any meaningful emissions reporting given the uncertainty that lies around it.
In both cases, the lack of ground truth and validation of marginal emissions factors poses serious challenges to their use.
As illustrated in Figure 3, the difference between marginal emissions factors available in PJM Data Miner and Singularity API can get even larger than on January 1st, 2024, and even exceed 1500gCO2/kWh in absolute value. Using one signal or the other leads to substantially different results for any use case from emissions reporting to load shifting.
These discrepancies illustrate how in the absence of a ground truth, the accuracy of a chosen signal remains unknown. This generally limits the trust in marginal signals. Besides the operational risk of optimizing in the wrong direction, using such a signal introduces reputational risks when exposed to auditors and public scrutiny.
Marginal emissions factors cannot be measured, and even grid operators are not able to validate them historically. In the absence of any ground truth, it is not possible to confidently assess the accuracy of a marginal signal. Providers use a multitude of methodologies and assumptions that often lead to inconsistent signals without the possibility to identify which one is the most accurate.
We encourage readers to perform their own evaluation and validation of marginal emissions factors. PJM marginal emissions factors are available for free on their data miner. A subset of our historical marginal emissions factors can also be provided upon request.
Given the importance of combatting climate change, emissions reporting and claimed emissions reduction are rightfully coming under increased public scrutiny. In these times, and to ensure our actions lead to tangible emissions reduction, verifiability and auditability are key. This is why regulations and standards require the use of average emissions factors and prohibit the use of marginal emissions factors. At Electricity Maps, we encourage transparency by publishing our data openly on our public app and data portal. All this data is also available through our commercial API.
Visit our free app to understand electricity grids and their CO2 emissions around the world.
