Governments should require that funded climate data be posted
Debate Rounds (3)
The purpose of this resolution is address one of the issues raised by Climategate, the scandal in which e-mail and software at the Climate Research Unit (CRU) in East Anglia. http://www.climate-gate.org... It's not known whether the CRU data was exposed by a hacker or by a whistleblower, but however revealed, issues persist. The scientists were revealed to be trash-talking about climate crisis skeptics, and apparently conspiring to subvert the peer review process. Those issues are put aside here to discuss another problem, the concealment of software and data from the scientific community.
The revealed documents includes a README file of a scientist, "Harry," trying to reproduce the climate data published by CRU, documenting enormous difficulty doing so. the file is posted at http://www.anenglishmanscastle.com.... CRU's mission is to obtain temperature data from various sources around the world, validate and correct the data, and convert it into a gridded format useful for scientific and practical purposes. The validation and correction steps are important because the raw data includes clerical errors, instrument errors, and errors due to the heat effects of new construction near the individual collection stations. "Gridding" converts the temperature data from the randomly located collection stations to regular increments of latitude and longitude using interpolation techniques. CRU performs all of the processing functions.
For research on global warming, small errors are important because the total amount of global warming examined is on the order of only a degree per century. Moreover, scientists look for "natural experiments" in which local conditions may have local climate effects. For example, rapid growth of a city many increase local pollutants or local CO2 levels, and scientists like to examine the possible local effects on temperature.
Britain has a Freedom of Information Act (FIA) similar to that in the United States. FIA requests can be filed to obtain certain documents and other data developed at government expense. In Britain, someone filed a request for the data used to support claims of CO2 global warming. CRU had great difficulty complying, Climategate revealed, because the software and data files were such a mess that they could not reconstruct the results they had published. he tale of woe begins with a guy copying 11,000 files and trying, unsuccessfully, to make something of them. He discovers, for example, that there are alternate files with the same name and no identification of which file is the one that should be used, or why.
NASA has similar responsibilities for climate data in the United States, and a similar FIA request was filed for supporting climate data. After nearly three years, NASA has still not complied with the request, and a lawsuit is now threatened to attempt to force compliance. http://www.thenewamerican.com...
I suspect that the problems of data compliance at CRU and NASA are due to professional incompetence, not a conspiracy to cover up errors they know to have been made. What has been revealed at CRU clearly shows incompetence. Moreover, there is nothing novel about incompetently written software. A product of human nature and schedule pressures is the method of hacking at software until it appears to work, then calling it done. In the commercial world, demands from users limit incompetence through calls for bug fixes, and ultimately user abandonment of one vendor in favor of another. Those mechanisms do not apply to climate data.
In the case of climate research, the tendency will be to hack at the software until it meets the expectations of developer, in this case the global warming believers at CRU. They could be innocently making a dozen small errors that tend to inflate temperatures in recent times, and no one would question the results, because expectations are met.
The remedy lies in immediate public disclosure. If the software must be posted regularly, which it will have to be because new results are released regularly, then peer pressure will greatly encourage sound software engineering practices like the use of software configuration control systems. Moreover, the details of the methodologies employed for processing and analysis will be subject to peer review.
CRU deals mainly with data rather than climate models, however the resolution applies to climate modeling software as well. The basic physics of carbon dioxide only accounts for about a third of the global warming it is claimed to cause, and that's not enough to cause a climate crisis. The models contain multiplying factors that are not verified by experimental measurement. All of the mechanisms should be subject to peer review and public scrutiny. A few institutions have made their model code public, but only a very few.
Aside from the concerns for good science and good professional practice, the public has a right to access what it paid for, for no reason beyond the fact that they paid for it. There are exemptions allowed in FIA legislation. The exemptions are for national security, independent proprietary data, and information sealed in lawsuits. None of the exemption apply to climate research. The requests to CRU and NASA were not denied under exemptions, they just not fulfilled. Requiring disclosure before publication or within a month after publication will guarantee that the public gets what it has a right to.
Climate research strongly affects public policy, so while good professional practices are important in all areas, the situation addressed by the resolution is exceptionally important.
The resolution is affirmed.
I am not going to defend either side of the climate-change controversy. I will leave that for another debate. What I will show is the fallicy in assuming that a full public disclosure of of the raw data, the processes, the software, and the findings will somehow further the further the progress toward determining the truth of the facts. Furthermore, I will show how unlimited public access to all, and especially preliminary data will only serve to further fire the flames of rhetoric that serve to obscure what may well be the most crucial issue of our time. I am certain, no matter on which side of the climate-change question your allegience currently lies, by the end of Round 3, you will be convinced to vote CON on the resolution as proposed.
My first refutation of the resolution is simple. There is no reason to believe that public disemination of the raw data, the processes used to qualify and quantify that data, the software used to accomplish those processes nor the daily findings that result from those processes would change the Public's perception of the issue. In fact, the polititions who must vote to fund this research (at least in the US) are very attuned to the pulse of the voters and their support or opposition to spending taxpayer money for it and, for the most part, those taxpayers do not rely on data or facts, they rely on someone to tell them how they feel about the question. Both the "Liberal" and the "Conservative" sides of the issue have their pundits to tell them whether to support the research or not, whether to believe what the other side tells them or not and no amount of facts are going to make a perceptable differencein the way a person feels.
This is not an argument of opinion, it is an argument based upon historical precedents:
In November of 1963, an assasin shot and killed the President of the United States in Dallas, TX. The Warren Commission reviewed the evidence of the investigation into that shooting and the background of the assasin, Lee Harvey Oswald and issued its findings almost a year later, in September of 1964.
The report spelled out the facts of the case and drew the conclusion that Oswald had acted alone. But conspiracy theorists had made up their minds that there were more than one shooter, that Oswald was acting under orders of the USSR or Castro's Cuba or The CIA or the FBI or whatever else they could dream up. Finally, after a long investigation by the Assinations Records and Review Board, they published their findings in September of 1998, confirming what the Warren commission has stated 34 years earlier.
Did that put an end to the conspiracy theory?
In 1947, the US Air Force launched Project Sign, later to become known as Project Blue Book which listed the results of investigations of thousands of reports of UFO's from the Roswell, New Mexico incident through January 1969. In January of 1970, those files were made available to the public, on the assumption that the facts would end the arguments over the validity of UFO sightings and Alien Invasion fears.
In 2009, the first draft of the National Health Care Reform Bill was published both in the Congressional Quarterly and on the Internet. Well, I guess that stopped all of the misconceptions about the bill, including the "government coming between you and your doctor" and the "Obama Death Panels."
Also in 2009, the CDC issued the warning against H1N1 or "Swine" flu, urging people to take precautions. Later, they came out with a vaccine that their test data proved safe. Now, in December, after over 10,000 people have died from H1N1 inthe US, people are still convinced the vaccine is "bad" and refuse to get vaccinated or even have their children vaccinated.
If governments require raw data (which almost always contains "flaws"), processed data (just another term for changing data or simply eliminating some of it), the software, (source code for analysis?) and findings within 30 days, both sides, pro and con, will have a field day pulling one line quotes, massaging figures, adding adjectives and just plain lying, and using the data they know nobody is really going to research to prove them wrong. They (both sides) will use this data to inflame the public and people, being people, are usually more afraid of change than keeping the status quo, so nobody wins and, many times there is a good chance the public is the ones that are going to get hurt.
My case for requiring the posting of publicly-funded data related to climate research boils down to three points:
C1. It will allow review of the data for error by other scientists.
C2. It will put peer pressure on the originating scientists to use better methodologies, such as software configuration control.
C3. The public has a right to the products of work paid for by taxpayers.
Con did not address any of my three points. Instead, he introduced two negative contentions.
N1. Pro claimed that public controversy would continue regardless of whether data is disclosed or not. I never claimed that disclosure would end public controversy. I don't doubt that there are people among the public who will not alter their positions regardless of what is revealed. We should not care about that. Dissent is protected, even if it is not well-founded. However, until there is disclosure of what climate crisis advocates have done, there is no possibility of achieving a consensus on it, either by the public or among scientists.
We may not ever get a public consensus, but there is a possibility of getting closer agreement among scientists. That will not happen until disclosure of research data and methods is accomplished.
N2. Con goes on to claim, "I will show how unlimited public access to all, and especially preliminary data will only serve to further fire the flames of rhetoric that serve to obscure what may well be the most crucial issue of our time." So for example, if the official position is that the earth is the center of the universe, allowing access to data that shows the earth revolves around the sun would similarly, "fire the flames of rhetoric that serve to obscure what may well be the most crucial issue of our time." "Our time" in that case being the Sixteenth Century. It is precisely because a scientific issue is important that data ought to be disclosed, not suppressed.
Con implied I wanted "preliminary data" disclosed. The resolution makes it clear that disclosure of source data and processing software is required only one month *after* publication. No preliminary data need ever be disclosed. It sometimes happens that a particular line of scientific inquiry proves ultimately fruitless, in which case no results are published and nothing ever need be disclosed as a consequence.
When a result is published it is believed by the originator to be reliable. In the case of climate data, it is used as a basis for public policy decisions. If it supposed to be the basis for decision making, then it is appropriate that the means by which it was derived be disclosed at that time, or soon thereafter.
Returning to my contentions, to which Pro offered no rebuttals:
C1. The revealed CRU e-mails show an intent to further subvert the peer review process. Peer review is performed by qualified scientists. Climate crisis advocates have a well-established pattern of attempting to conceal data. The most notorious example is the bogus "hockey stick" in which global temperature were claimed to have been stable for a thousand years, until they rose exponentially in the past few decades. The hockey stick graph was included in the 2001 IPCC report as proof of CO2 caused global warming. The graph was doubted from the outset, because it did not show the Medieval Warm Period or the Little Ice Age, major periods of natural climate variation.
With enormous effort, skeptical scientists finally managed to extract the source data from those behind the hockey stick construction and to show the specific errors in data processing that produced the spurious result. Organizations such as the Harvard-Smithsonian Institute for Astrophysics subsequently conducted a massive review of the scientific literature and firmly established the existence of the past climate variations. The UN removed the hockey stick from the 2007 IPCC report. The process of uncovering and correcting the errors took close to a decade, because the originating scientists would not divulge their data or detailed methodology.
CRU was heavily involved in preparing the data for the bogus hockey stick.
The present resolution would prevent such concealment and would enable skeptical scientists to conduct a proper review of important research. The errors in the derivation of the hockey stick might have been revealed before it was included in the 2001 IPCC report.
Currently there is considerable controversy over recent temperature data that show sharp recent temperature rises. Allegations include claims that much of the recent proof of global warming is derived from the rings of three trees in Siberia, which is claimed to be given a high weight in multiple sets of climate data. Originators of the data are extremely reluctant to reveal their data and methods. The current resolution would help resolve the issue.
Note that NASA has refused to comply with FIA requests made in 2007. The present resolution would have required contemporaneous disclosure.
C2. The accepted professional practice for industry is to use a software configuration control system. This is applied both to program code and to data files. So if one wishes to recover the results at some specific time in the past, the configuration control system will automatically reconstruct the software and data sets for the desired day. CRU could not do that, so they could not comply with FIA requests even if they wanted to.
To my knowledge, no one has claimed that being unable to reproduce past results is an acceptable practice in the scientific world. Forcing immediate direct disclosure solves the problem in one sense, because outsiders can then track the data. However, once scientists at CRU and elsewhere realize they will be forced to disclose, by far the easiest way to comply is to do what they should have always done -- implement a configuration control system.
The US military systematically requires its contractors to implement such systems, and they sometimes require that the government be able to access the system remotely in real time. In other words, the military does not allow disorganized software development. While the resolution does not require such high standards of scientists, the resolution strongly encourages improved practices. The benefit is that the taxpayers get higher quality work for their money.
C3. Why has Con failed to address the public's right to get access to what they have paid for? I have allowed that there are certain exemptions to the general rule of disclosure. Climate research does not fit any category of exemption. FIA requests have not been denied under any claim of exemption, the requests are just arbitrarily delayed or ignored. The resolution would put an end to concealment through delaying tactics.
The CRU e-mails include internal requests to destroy past e-mail files so they could not be uncovered by FIA requests. CRU also admits to having destroyed original climate data, although they claim they did so to save storage space. Immediate disclosure puts an end to the destruction of scientific data to conceal it.
Any one of my three contentions is sufficient to support the resolution. So far Con has addressed none of them.
The resolution is affirmed.
Perhaps I am misreading the resolution but what I see is that once a finding is published or "announced by any means," within 1 month the source (I read this as "Raw") data that was collected or analyzed (wouldn't analyzed data also have to have been collected?) would be posted on the Internet, open to the public (else how would the "public" know what it is they are getting for their money?) apparently unabridged.
To assume this would eliminate (or even curtail) errors and/or ambiguous data or skewed results is pure conjecture and unsupported. Would that this data and the software used were made available to all bona fide investigators, meaning scientists, researchers and technicians employed by or actively engaged in climate-change research for peer review and co-operative research, with Penalties for releasing information prior to peer review and consensus under the umbrella of National Security (since, indeed, the security of the nation is at stake based upon the path we take in response to the question of global climate change) then I believe a better understanding would be possible than placing the future of our country in the hands of the pundits who, admittedly (or not) have their own agendas.
If all communications between parties engaged in climate-change research were to be made public, free and uninhibited discourse between these parties would be severely curtailed. Although it is impossible to secure e-mail 100%, it should still be considered as private or even as protected as "snail mail" or the verbal conversation of which it is replacing and not subject to public scrutiny.
No one doubts that every instance of computer software is without possible flaws, or glitches, but to recreate identical results from identical data requires more than just a copy of the software. One has to duplicate perfectly the entry of that data and the parameters surrounding the software within the specific computer as well as being equally conversant in the operation of the software as its creator. This is especially true of software that is targeted toward a specific problem (analyzing and modeling climate change data within certain parameters) rather than "vanilla" software (such as a word processor or graphics generator) that will eventually be used in many divergent applications by many users.
Most software designers, as well as most scientists, I understand, are a proud group, who feel insulted by uninformed criticism upon their work by pundits who are not conversant in the field in which their attacks are directed. To expose this work or product to public scrutiny, rather than limiting it to peer review, is to place unjustified pressure on these professionals to limit their research to "non-controversial" areas or to withhold crucial data to avoid unwarranted attack or even to "fudge" data to assuage the critics.
There is no argument against the public having a right to the PRODUCTS of work paid for by taxpayers. But to insist upon public disclosure of every keystroke and instance of data falling outside of the standard deviation model is to invite equally public defamation of the publishing party as well as "ammunition" by those whose agenda is to disrupt and discredit the entire field of study by any means at their disposal. Again, I aver, there is a direct correlation between public perception of an issue and the amount of money approved to research that issue. While the Congress controls the purse strings, it is the constituency who ultimately control the congress. This, of course, includes that portion of the constituency who have a vested interest in maintaining the status quo, regardless of the long-term effects.
Con understands correctly that the resolution requires that raw data be published within one month within the results of analysis. He asks, "wouldn't analyzed data also have to have been collected?" Analyzed data is what is published, so it is always disclosed. Thus for example, the raw historical temperature data showing the existence of the Medieval Warm Period (MWP) would be required to be posted on the Internet along with the processing software that removed the MWP to yield the hockey stick graph in which there was no climate change prior to the last few decades. Under the current rules, only the result in which past climate change was removed is revealed, and it took scientists a decade to dig out the steps by which the MWP was made to disappear.
Con claims that, "To assume this would eliminate (or even curtail) errors and/or ambiguous data or skewed results is pure conjecture and unsupported." Con's clam is false because (a) I gave an important example, the hockey stick controversy, in which a serious error was revealed once the requirements of the resolution were met, and (b) the peer review process, long the basis for scientific publication, is enabled by the resolution. The resolution requires that information available under the Freedom of Information Act in the US and Britain be produced on a timely basis, rather than be subject to indefinite delay. It is not unsupported conjecture that peer review uncovers errors in general, nor is it conjecture that the process embodied in the resolution uncovered the major error in the hockey stick graph, with the result that the hockey graph was removed from the 2007 IPCC report.
It is also not conjecture that CRU, perhaps innocently, erased the climate source data requested under a British FIA request, and that after nearly three years NASA has not responded to a FIA request for source data. If the method of the resolution is put into effect, the data disappearances and indefinite delays will be ended.
Con suggests that the data and software only be given only to "bona fide investigators" rather than to "pundits." This suggestion does not respond to the history I have cited in which CRU and NASA did not in fact provide the data for review by bona fide climate scientists, claiming they lost the data or that they are unable to reconstruct the analysis or using various other excuses. In the Climategate e-mails, CRU scientists speak directly of subverting the peer review process so that their work will not be subject to scrutiny. The CRU and NASA scientists have no worries whatsoever about the general public or "pundits" receiving the data, because only scientists are capable of unraveling what CRU referred to as "tricks" used in processing the data. finally, there is nothing that prevents a scientist from also being a pundit. CO2 crisis advocates are often prominent pundits. Just produce the data for all to see, without a bunch of nonsense obstacles.
The Medieval Warm Period was about as warm as the present, but it was made to disappear entirely from the historical record of climate through the the use of mathematical techniques claimed to be good science. It took substantial expertise to discover the errors. Every effort was made to keep the hockey stick data from qualified scientists. (The history of the hockey stick and it's unraveling is well covered in Plimer's book. http://www.amazon.com... )
The way that peer review is currently avoided, as it was avoided in the case of the hockey stick graph, is to provide the papers and the data only to believers in climate crisis, who then provide only a cursory review. The work is not made available to review by skeptics prior to publication, and after publication the data is not produced voluntarily for review. FIA requests must be formally filed, and those are often ignored or resisted.
Con implies that there is some threat to national security involved if data is published. There is none. For example, there are currently about 770 scientific papers supporting the existence of the Medieval Warm period, and they include work form Russia, China, and every corner of the earth. Con cannot site a single matter of national security involved.
Con claims, "If all communications between parties engaged in climate-change research were to be made public, free and uninhibited discourse between these parties would be severely curtailed." The resolution does not require the publication of any e-mail or correspondence. It does not require disclosure of preliminary results of any kind. The resolution requires on that when results of climate analysis are published voluntarily by scientists, that the supporting raw data and processing software be posted within one month of publication. CRU wanted to keep their e-mail about subverting peer review secret for fear a hacker or whistleblower would reveal it, but the present resolution would not affect private e-mail. The resolution only concerns data and software, and then only when results are announced.
Con argues "One has to duplicate perfectly the entry of that data and the parameters surrounding the software within the specific computer as well as being equally conversant in the operation of the software as its creator." Yes, that is why software configuration control systems are used. http://en.wikipedia.org... It is a solved problem. There is no version of good science that does not require reproducing results.
There is no concern that unqualified people in the general public will critique climate research software. The general public cannot comprehend it. What CRU and NASA are worried about are the critiques of well-qualified climate scientists. And they should be worried, because in the past they have been caught cooking the books.
Con concludes, "But to insist upon public disclosure of every keystroke and instance of data falling outside of the standard deviation model is to invite equally public defamation of the publishing party ..." But, of course, the resolution does not come close to requiring every keystroke. Rather, climate scientists advocating crisis theory made the Medieval Warm Period and the Little Ice Age disappear from climate history. During the Medieval Warm period, grapes were grown in Scotland and Greenland was named for its greenery. During the Little Ice Age, the Thames froze over every year and winter festivals were held on the ice. But after processing by "sound scientific methods" a graph was produced that showed nothing happening with global climate until the last few decades. The data and methods were withheld from skeptical scientists and from the public who paid for the bogus research.
The resolution only requires that scientists paid by the government to perform climate research disclose what raw data they started with, and how they process it to get the results they voluntarily choose to publish. I'm sorry if they don't like to show their work, but most of us had to do that starting in grade school. climate research is too important to let it be concealed, only to have it ultimately drawn out by lawsuits under the Freedom of Information Act. Let's do it up front and get on with it.
The resolution is affirmed.
Pro has stated he is having trouble figuring out what I was claiming with respect to the contentions. I understand his confusion since I meant to say I was refuting his claims of results that most likely would result. I apologize for the error and will be more careful in the future. Too bad the spell checker cannot flag those kind of errors.
Pro contends that by posting on the internet the data supporting the publication of findings by a particular scientist or group, that the raw historical data be included in the posting along with the software used to analyze and plot that data, be included. On the surface this sounds good but here is the problem: If a group was to come across a small data set that refuted their other findings, but that that data was obtained in an unsatisfactory manner (such as some measurements being taken during an unusually warm period due to other than usual conditions, they would have to show the erroneous data and explain it omission in the overall results. Here is where the pundits come in: I can hear it now, "They just threw out data they didn't like!...This proves their entire theory is wrong and it is probably been cooked to get more tax money!" In debate we call that a Biased Sample Fallacy coupled with the Red Herring Fallacy. In debate, we have the opportunity to point these fallacies out and moving on, letting the judges decide. In real life, once the pundits latch on to something they think they can run with, in this case, not the "missing data" but the tax money. Scientists can counter the arguments all they want, but the anti's will still be the loudest. It is for this reason the raw data and the process of analysis (which may include the non-consideration of some data) being presented to the public would be counter-productive, thus confirming Con's position. I said I wouldn't get into the debate of global climate change per se so I will admit that the same scenario could be played out to the opposite side but the result would be the same...degenerating the vital investigation into this question into a debate of semantics rather than allowing the truth to emerge. Regardless of which side one is on, it is the truth should be the goal, not just an individual's (or a political party's or special interest group's)preset agenda. This whole question has become a political football, no longer a scientific investigation into a very important subject.
Pro has, in fact, admitted that [quote]" There is no concern that unqualified people in the general public will critique climate research software. The general public cannot comprehend it."[end quote] therefore solidifying Con's position that there is no good reason to release the data he suggest to the public, but to restrict it's exposure to qualified scientists, other than to allow non-professionals, with no prestigious standing at risk, to cherry-pick data to foist upon the unsuspecting and admittedly gullible public as the absolute truth. This alone, should be enough to vote in favor of Con.
Pro contends that the "hockey stick graph" contained a "major error" that caused it removal from the 2007 IPCC report. However, it was not the major error but the removal of the questionable data was portrayed by certain highly placed politicians who were admittedly opposed to any suggestion that global warming could be a real occurrence.
The fact that the "Medieval Warming" and the "Little Ice Age" were left out due to the suspect nature of the data and the relatively slow temperature rise shown by even the suspected data over several hundreds of years compared to the comparatively rapid rise over less than one hundred years, did not discourage the critics of the issue one bit, with the pundits again exclaiming how the figures were deliberately falsified to simply satisfy some unknown benefactor holding the enormous purse strings of the federal budget.
Pro has said that all the data should be produced "for all to see, without a bunch of nonsense obstacles." But then, since the public wouldn't comprehend it, it would still become, not a scientific question but a political football with each side pointing fingers and claiming the other guy was wrong and the "unqualified people in the general public" would be the ones left to make the decision of which side was "right."
Pro's contentions have all been supported by questionable and debatable outcomes based on illegal and unethical "evidence" that has been sensationalized to create an adversarial atmosphere surrounding the entire field of research. It is not my contention that either side is correct - just that the public exposure of esoteric (at least among the general public) methods of analysis is counter-productive to determining the truth behind the science. Therefore, Con refutes the resolution as written.
2 votes have been placed for this debate. Showing 1 through 2 records.
Vote Placed by RoyLatham 4 years ago
|Agreed with before the debate:||-||-||0 points|
|Agreed with after the debate:||-||-||0 points|
|Who had better conduct:||-||-||1 point|
|Had better spelling and grammar:||-||-||1 point|
|Made more convincing arguments:||-||-||3 points|
|Used the most reliable sources:||-||-||2 points|
|Total points awarded:||6||0|
Vote Placed by Nails 4 years ago
|Agreed with before the debate:||-||-||0 points|
|Agreed with after the debate:||-||-||0 points|
|Who had better conduct:||-||-||1 point|
|Had better spelling and grammar:||-||-||1 point|
|Made more convincing arguments:||-||-||3 points|
|Used the most reliable sources:||-||-||2 points|
|Total points awarded:||1||3|