| Submit your comments on this article |
| International-UN-NGOs |
| Audit Shows Pervasive Inaccuracies In Washington DC Think Tank's Climate Campaign |
| 2008-06-05 |
| A newly released audit report reveals that the popular online power plant emissions ranker Carbon Monitoring for Action, run by the Center for Global Development (CGD), is utilizing erroneous and questionable CO2 emissions data. The new report, titled "Climate Campaign Built on Questionable Data," provides quantitative evidence cataloguing the depth and breadth of errors and gaps in CARMA's numbers, which have faced criticism before. The report also raises questions about CGD's lack of disclaimer and disclosure policies. CGD has known of data quality issues since early December of 2007. Given the policy importance of climate management (and the unmistakable role that a rallied public plays in the issue), the auditors provide guidelines for generating reliable and actionable CO2 data. "The first signal of CARMA's dubious data emerged when Hong Kong's Castle Peak power station publicly refuted CARMA's numbers," said the lead author of the report, environmental consultant Shakeb Afsah. Afsah, a policy analyst trained at Harvard's Kennedy School of Government and the Indian Institute of Technology in New Delhi, is best known for his work with the World Bank, implementing environmental rating programs in Indonesia and the Philippines in the mid nineties. CGD's explanation of the Castle Peak errors led Afsah to suspect broader problems in CARMA's database. When CARMA was launched on Nov. 14, 2007, Castle Peak presented the group with company data showing electricity production at 14.15 MWh for the year. CARMA had calculated power generation at 28.2 million MWh for Castle Peak - twice what company data showed. The CGD corrected the error. Its communications department then wrote: "So far, out of the 50,000 plants listed only one meaningful discrepancy has been identified." NOT TRUE - The audit reveals that 90 percent of CARMA's CO2 estimates have discrepancy with US Government's data if a +-5 percent margin of error is used. And more than half of CARMA's CO2 estimates exceed a 25 percent margin of error when contrasted with the USEPA data. These differences are difficult to reconcile. Examples abound. USEPA data shows Florida's Manatee plant generating 5.7 million tons of CO2. CARMA sets the number at 11.7 million. The Mountainview Power plant in California, Hays Energy in Texas, RS Nelson in Louisiana. are just a few plants with CO2 emissions values differing by more than one million tons between CARMA and USEPA data. Given such immense differences, both data sets cannot be correct. The contradictory numbers pose an additional problem: the public has no clear understanding of which ranking system, data set and website should be trusted. The audit report shows that CARMA's model accounts for the discrepancy between USEPA and CARMA numbers. CARMA's numbers are simply too outlandish. CARMA estimates that the Laramie River power station in Platte County, Wyo. generated 24.8 million MWh of electricity in 2007. The plant's maximum generation capacity, by all accounts, is 70 percent less. CARMA has other illogical estimates for Laramie River. It shows the coal based power station generating 815 lbs of CO2 for every megawatt hour of electricity. Coal fired power plants generally have carbon intensity ranges between 1800 and 2200 lbs CO2/MWh. CARMA's inordinately low numbers raise red flags for analysts. These errors impact rankings. USEPA ranks the Laramie River plant 24 among US carbon emitters. CARMA ranks this plant at 58. The discrepancy has serious reputational implications for a company. The Rockport power station in Spencer County, Ind. sits on CARMA's distinctive "Dirty Dozen" list. But the USEPA ranks Rockport at 22, reserving the "Dirty Dozen" label for another company. Three other plants - the Cumberland plant in Tennessee, the Sherbourne County plant in the Becker, Min. and the Bruce Mansfield plant in Shippingport, Pa.-- are also incorrectly labeled "Dirty Dozen" by CARMA. The relative ranking of power plants is the driving force behind activism incentives that CARMA aims to create. If this critical information itself is questionable, it raises doubts about CARMA's usability for climate protection initiatives. There are clear signs that CARMA's methodology doesn't resonate well with the CO2 monitoring and verification protocols like those recommended by the World Resources Institute. CARMA's technical inconsistencies extend beyond CO2 estimates; its carbon intensity data seems implausible. CARMA attributed many natural gas power plants with a carbon intensity of 6,000 lbs/MWh - nearly three times the average for coal-fired plants. Natural gas simply cannot generate so much CO2 during the combustion process. Nabiel Makarim, former Minster of Environment in Indonesia, who first conceived the idea of environmental rating and disclosure program in 1993, considers accurate information vital. "Data credibility is essential for environmental disclosure programs. Such programs have zero-tolerance standard for inaccuracies." Stakeholders on both sides of the contentious issue of carbon emissions have read the pre-release draft of the report. They have uniformly expressed support for disclosure and reiterated Makarim's concerns. "For decision makers to arrive at sensible climate policies, they must have available reliable and verifiable data on which to base legislation and regulations," said Ned Leonard, Vice President for Policy at the American Coalition for Clean Coal Electricity. Environmental economists who read the audit report also expressed that CO2 emission rankings should be based on actual measured data and not statistical estimates. Why have the principles of quality been compromised? The surging popularity of climate issues has created a tempting environment for public recognition, but popularity doesn't preclude the need for accuracy. Terms CARMA employs, like "Power plant voyeurism," stimulate the public around negative information. This is not information-sharing. CARMA's public promises are not being kept. Even geographical coordinates have been fudged. CARMA claims to have marked the exact locations of 50,000 power plants, but 41 percent of U.S. power plants are missing from their database. The plants that are marked have faulty geographic coordinates, placing, for example, Washington D.C.'s Benning power plant in an apartment complex. Afsah hopes that his due diligence work results in more disciplined data-based analysis and stronger on-site monitoring and verification of CO2 emissions. He says that CARMA fosters an asymmetric focus on large power plants when in many cases these plants have the lowest carbon intensity. Smaller power plants with worse carbon intensities get relegated to an irrelevant subset by CARMA. This lopsided effect of CARMA's ranking system could shape distorted policies. CGD severed contact with the audit's authors after reading an initial draft in December-07, to the authors' distress and disappointment. As a publicly relevant policy instrument CARMA and its backers at CGD should be answerable to those with legitimate concerns. CGD's behavior raises questions about the accountability channels available to the public when such organizations run policy programs using private data. A public agency would face higher accountability standards. On Dec. 7 of last year, Dennis de Tray, Vice President for Special Initiatives at CGD, communicated that CARMA's David Wheeler would "deal with many of the issues" raised about the program's methodology. More than five months later, and five days after a pre-release version of the audit hit the internet, CARMA released a methodology paper. Power engineers quickly confirmed that important operational variables, like the supply stack that reflects the mix of power plants in a region, are not included in CARMA's estimation model. Such omissions inevitably create questionable estimates. Such concerns were voiced early on by the Australian spokeswoman for the New South Wales Energy Minister Ian MacDonald: "The US study has utilized a number of assumptions, some of which are highly questionable. For example, the quoted tonnes are inaccurate because they are based on capacity only and ignore actual generation, fuel type and efficiency. Simply using the size of a power plant is not appropriate for comparing rates of emissions. The study also appears to have assumed how the power stations operate, rather than researching the actual operation data." Now that CARMA's methodology is out, authors of this report look forward to full debate on this matter. But unfortunately CARMA has again withheld information - the database used to create CARMA's estimation model, comprising information on around 3,000 power plants, remains outside the public domain. As such, the model remains impossible to replicate. Auditors don't expect perfection, but are CARMA's numbers accurate enough to rank power plants or geographical aggregates? CARMA's discrepancies have impacted county aggregate rankings - Walker County, Ala. is ranked first for CO2 emissions by CARMA, but USEPA ranks it 115. CARMA ranks California at 13, but USEPA sets it at 25. Similarly Michigan ranks at nine by CARMA but 13 by USEPA. Which ranking is right? The auditors hope that their report initiates a profound discussion on public disclosure and data accuracy. In the meantime, dedicated CO2 monitors can use data from the USEPA, an institution that has adopted many of the basic steps that would make CARMA more honest and more valuable, including disclosure about where statistical models were used in place of hard data, what the margin of error is in those occasions, and full "open sourcing" of the model so the public can review and critique the data and updates on the database. |
| Posted by:3dc |
| #4 Bad CARMA. |
| Posted by: charger 2008-06-05 11:55 |
| #3 better link also, the average for 08 is below the long term average, not each month |
| Posted by: mhw 2008-06-05 10:20 |
| #2 for what its worth, May 08 was substantially cooler than May 07 and in fact, average worldwide temperatures every month since Jan 08 have been below the long term lower troposphere average temps according to the Huntsville algorithm method of temperature estimation (there are other algorithms which don't show as much of a temperature decrease). |
| Posted by: mhw 2008-06-05 09:35 |
| #1 OK CARMA, so when are you going to stop lying to us? |
| Posted by: Richard of Oregon 2008-06-05 09:22 |