Climategate is the popular name given to the revelation of e-mail and software from the Climate Research Unit (CRU) at East Anglia in Britain. http://www.climate-gate.org… It’s not known whether the material posted on the web was obtained by a hacker or by a whistleblower inside the organization. The bottom line on the scandal is that scientists sponsored by governments should be required to make their work public so it can be checked by other scientists.
Peer review is the process by which scientists check the work of other scientists. The process is imperfect, but it helps ensure the accuracy of published scientific papers. The Climategate revelations showed that CRU scientists did not want their work checked by climatologists skeptical of their theory that CO2 is causing a climate crisis. The revelations include a great deal of trash-talking about dissenting climatologists at other organizations, promises to subvert the peer review process by which scientific articles are qualified for publication, and a long narrative of attempts to reconstruct software used to compute published temperature data. CRU scientists had failed to keep records of how they had come up with the results they had published, so not could no one else verify their results, they could not do it themselves.
There is a remedy for the problems. Governments should impose a condition on climate research grants and aid related to climate research that source data collected or analyzed under the grant, and all software developed under the government support shall be posted on the Internet within one month of publication or announcement of the results by any means.
The revealed documents includes a README file of a scientist, "Harry," trying to reproduce the climate data published by CRU, documenting enormous difficulty doing so. The file is posted at http://www.anenglishmanscastle.com…. CRU’s mission is to obtain temperature data from various sources around the world, validate and correct the data, and convert it into a gridded format useful for scientific and practical purposes. The validation and correction steps are important because the raw data includes clerical errors, instrument errors, and errors due to the heat effects of new construction near the individual collection stations. Gridding converts the temperature data from the randomly located collection stations to regular increments of latitude and longitude using interpolation techniques. CRU performs all of the processing functions.
Small errors are important because the total amount of global warming examined is on the order of only a degree per century. Moreover, scientists look for natural experiments in which local conditions may have local climate effects. For example, rapid growth of a city many increase local pollutants or local CO2 levels, and scientists like to examine the possible local effects on temperature.
Britain has a Freedom of Information Act (FIA) similar to that in the United States. FIA requests can be filed to obtain certain documents and other data developed at government expense. In Britain, someone filed a request for the data used to support claims of CO2 global warming. CRU had great difficulty complying, Climategate revealed, because the software and data files were such a mess that they could not reconstruct the results they had published. he tale of woe begins with a guy copying 11,000 files and trying, unsuccessfully, to make something of them. He discovers, for example, that there are alternate files with the same name and no identification of which file is the one that should be used, or why.
NASA has similar responsibilities for climate data in the United States, and a similar FIA request was filed for supporting climate data. After nearly three years, NASA has still not complied with the request, and a lawsuit is now threatened to attempt to force compliance. http://www.thenewamerican.com…
I suspect that most of the problems of compliance at CRU and NASA are due to professional incompetence, not a conspiracy to cover up errors they know to have been made. What has been revealed at CRU clearly shows incompetence. There is nothing novel about poorly written software. A product of human nature and schedule pressures is the method of hacking at software until it appears to work, then calling it done. In the commercial world, demands from users limit incompetence through calls for bug fixes, and ultimately user abandonment of one vendor in favor of another. Those mechanisms do not apply to climate data.
In the case of climate research, the tendency will be to hack at the software until it meets the expectations of developers, in this case the global warming believers at CRU. They could be innocently making a dozen small errors that tend to inflate temperatures in recent times, and no one would question the results, because their expectations are met.
The remedy lies in immediate public disclosure. If the software must be posted regularly, which it will have to be because new results are released regularly, then peer pressure will greatly encourage sound software engineering practices like the use of software configuration control systems. Moreover, the details of the methodologies employed for processing and analysis will be subject to peer review.
CRU deals mainly with data rather than climate models, however disclosure applies to climate modeling software as well. The basic physics of carbon dioxide only accounts for about a third of the global warming that it is claimed to cause, and that’s not enough to cause a climate crisis. The models contain multiplying factors that are not verified by experimental measurement. All of the mechanisms should be subject to peer review and public scrutiny. A few institutions have made their climate model code public, but only a very few.
Aside from the concerns for good science and good professional practice, the public has a right to access what it paid for, for no reason beyond the fact that they paid for it. There are exemptions allowed in FIA legislation. The exemptions are for national security, independent proprietary data, and information sealed in lawsuits. None of the exemption apply to climate research. The requests to CRU and NASA were not denied under exemptions, they just not fulfilled. Requiring disclosure before publication or within a month after publication will guarantee that the public gets what it has a right to.
Climate research strongly affects public policy, so while good professional practices are important in all areas, disclosure of the details of climate research is especially important.