The biggest scam in history

[PaulG.]

After the NRA thread, I'm almost afraid of my email client taking action against me, but I'll post anyway, just because this also illustrates the issue discussed in both threads, basically, of lying to further your goal.

While it's fun to make fun of Al Gore, he did not claim he invented the Internet. He's a politician, and politicians are at least smart enough to know not to say things like that (unless the politicians are actually scientists who invented something). He said he had a part in the creation of the Internet and he was actually backed by some of the people who did, who verified that he helped push and fund the relevant projects.

"During my service in the United State Congress, I took the initiative in creating the Internet." -Al Gore, 4/9/1999

If George W. Bush had said the exact same thing the media and the moonbats would have gone nuts and howled in collective glee. Editorials would have gone on for months. Al Gore said this, and whether or not he was taken out of context doesn't matter. He said it. Al Gore was very instrumental in legislation that helped build the Internet infrastucture. Even taken out of context the quote is actually partially true. It just sounded silly and self-serving. That's why I use the quote and make fun of Al Gore for saying it.

[Gary Rubin]

"During my service in the United State Congress, I took the initiative in creating the Internet." -Al Gore, 4/9/1999

If George W. Bush had said the exact same thing the media and the moonbats would have gone nuts and howled in collective glee. Editorials would have gone on for months. Al Gore said this, and whether or not he was taken out of context doesn't matter. He said it. Al Gore was very instrumental in legislation that helped build the Internet infrastucture. Even taken out of context the quote is actually partially true. It just sounded silly and self-serving. That's why I use the quote and make fun of Al Gore for saying it.

I've worked with people who are involved with funding and overseeing development of government programs. They really do seem to have trouble distinguishing (at least in the language they use) between doing the work and being in the program management/funding chain. I've heard people say "We have experience with X". When you ask questions about that experience, it becomes clear that what they really should have said is that they performed programmatic oversight of someone who was doing X.

I think it's not terribly unlike the people who have used LV-build apps and think that means they are experienced LabVIEW users.

[hooovahh]

I've worked with people who are involved with funding and overseeing development of government programs. They really do seem to have trouble distinguishing (at least in the language they use) between doing the work and being in the program management/funding chain. I've heard people say "We have experience with X". When you ask questions about that experience, it becomes clear that what they really should have said is that they performed programmatic oversight of someone who was doing X.

I think it's not terribly unlike the people who have used LV-build apps and think that means they are experienced LabVIEW users.

This is not limited to politicians or project managers. I had a professor for a networks class (mostly CAN) and in the first day of class he had a power point presentation telling us about how he made a drive by wiring for a fork lift, and he made some robots and stuff.

Turns out he didn't do any thing, it was a senior design class and a thesis that he advised for that in his mind he helped enough to claim he made it.

Our final project was to simulator a drive by wire system for a motor cycle (lets not get into why this is a bad idea just yet) We had a node to simulate the signals, and we had a node displaying the signals with a dash display, and images of the front and back of the bike to show turn signals and lights.

His next term he added out stuff to his power point presentation and ended up telling the class he made it again. Coincidentally enough, there were some students who were in both classes and they called him out on it.

[PaulG.]

A plethora of climategate stories.

Al Gore taking cover.

Why can't we investigate both?

Edited December 4, 2009 by PaulG.

[Michael Aivaliotis]

All. Let's keep this conversation civilized please.

The Earth's been here for millions of years and it will still be here for millions more, even after we've become extinct. So at least, for the short amount of time we have, let's try to give each other some respect.

[PaulG.]

BBC is on board. It reveals the most shocking revelation yet IMO. But why am I not shocked?

As programmers we should all start taking this scandal personally.

[crelf]

BBC is on board. It reveals the most shocking revelation yet IMO. But why am I not shocked?

As programmers we should all start taking this scandal personally.

From the BBC article: Graham-Cumming said... "If you look at (the code) -- what's done here by these alleged CRU files - it's not the thing you'd expect to see in certainly a commercial industry. You would not see this sort of source code because it's not clearly documented. There's not audit history of what's happened to it. So it would be below the standard in any commercial software."

Ha! I can point to many many apps (some very old ones that I've written myself) that aren't well documented nor contain an audit history that are still commerical Point is: the CRU code *isn't* commercial software. That doesn't excuse it from good standards, but let's not try to suggest it's common practise to apply the standards of on industry segment to another.

[PaulG.]

Point is: the CRU code *isn't* commercial software. That doesn't excuse it from good standards, but let's not try to suggest it's common practise to apply the standards of on industry segment to another.

(slowly shaking my head)

Let's not lower this discussion to talking about "standards".

This is about ethics.

Science is the search for knowledge. Science is not the "truth" corrupt entities with an agenda want us to believe by manipulating data and facts for monetary and political gain.

[Anders Björk]

I really do not get what fuss is about there is one scientist that has behave badly with data analysis, that is very bad in it self. But The IPCCs report are not based on this only cheating scientist is based on a much much much greater material.

Also that fact is that we have burning coal and oil in rate that is not sustainable at all.

Edited December 6, 2009 by Anders Björk

[Daklu]

I really do not get what fuss is about there is one scientist that has behave badly with data analysis, that is very bad in it self. But The IPCCs report are not based on this only cheating scientist is based on a much much much greater material.

This is not a case of one random scientist behaving badly. It is a case of several powerful scientists in the area of climate research conspiring to silence critics and manipulate data to achieve a political goal. The IPCC is not a scientific body; it is a political organization. If you read the links you will find many reports of obvious flaws in the IPCC's "peer review" process.

Also that fact is that we have burning coal and oil in rate that is not sustainable at all.

Sustainable energy is a separate (though related) issue from AGW. I happen to agree we should be actively researching energy alternatives. The correct way to pursue that goal is NOT by feeding the public fear-based propaganda; it's by finding a real argument that changes the minds of your opponents.

-----------------

Incidentally, AGW may be real, it may not be--but it's clear to me we need to step back and look at the issue more thoroughly and more transparently before we take action.

Edited December 7, 2009 by Daklu

[Cat]

BBC is on board. It reveals the most shocking revelation yet IMO. But why am I not shocked?

As programmers we should all start taking this scandal personally.

I have written code that has the exact term "fudgeFactor" for a constant.

I have written code without an audit track.

I've probably written code that if someone wanted to pay another LV programmer big bucks to analyze, they'd probably find some esoteric error in it somewhere. Well, okay, maybe it wouldn't be *that* hard.

I have written code that included perhaps not quite the most professional comments, mostly made late at night (or early in the morning) after too much espresso, or perhaps getting a little over-the-top when completely fed up with a user's anal retentive requirements (I'm thinking about a dialogue box that has two options: T="Continue" F="I don't know. Ask me again." I would have gotten rid of it after the beta release, but the users love it. Most anything is funny after a week on a submarine...)

Unless I missed it, no where in that article does it say the code doesn't do what it's supposed to do. No where does it say that the algorithm behind the code is incorrect or was used to "fudge" data. It does however sound like the programmer needs a little help with his/her self-esteem...

I write software that people use to analyze data. I bear no responsibility for what they do with the results my software gives them.

It seems that a scientist deliberately altered data -- regardless of what the code may have told him. That is wrong and extremely upsetting. But I consider that on a completely different level from someone writing "unprofessional" code. It would be a bit hypocritical of me to do otherwise.

[PaulG.]

But I consider that on a completely different level from someone writing "unprofessional" code. It would be a bit hypocritical of me to do otherwise.

We all have written "unprofessional" code and comments. But I would like to think most of us would take the extra step and bring our professionalism up a notch or two if we were working on a project of such importance. The CRU is either collectively arrogant or extremely naive to think their work would never be scrutinized by the general public.

Edited December 7, 2009 by PaulG.

[crelf]

We all have written "unprofessional" code and comments. But I would like to think most of us would take the extra step and bring our professionalism up a notch or two if we were working on a project of such importance.

depends on who it is important to - I know that the code I write can be considered more important to my customer than it is to me, and maybe that's what's happened here - the coder/scientist(s) invovled were just implimenting alogrithms, and weren't the end-users. I'm not saying that makes it right, I'm just saying that there may be an explination that's not just that the scientisit/programmers invloved are evil.

The CRU is either collectively arrogant or extremely naive to think their work would never be scrutinized by the general public.

Maybe the scientist and/or developer didn't think it would ever be scrutinized - maybe they thought "okay, we know that this is a fudge factor, and is wrong, but we need it in there to get the software and/or data analysis up and running for now with some dummy data, and we'll go back an fix it later, once we've got the real data in", and then it got lost in the haze. Totally true that a better code review and change tracking system with the appropriate processes to execute them would have caught this.

That said, you're right - the CRU we *insane* to release anything to the public without reviewing it first. The fact that there's fudge factors anywhere is inexcusable (especially with the gravity of th results), that they then released the code with the fudge factors in there is (you said it perfectly) either collectively arrogant or extremely naive. Or maybe it was reviewed and someone who wanted to take the CRU down from the inside say the issue and didn't red flag it. Or... actually, there's a bunch of reasons that this could have happened - but none that I can think of that are excusable.

[Daklu]

I bear no responsibility for what they do with the results my software gives them.

Addressing this comment purely from a philosophical standpoint, you bear much responsibility if the software you write (intentionally or unintentionally) biases the data one way or the other--moreso if you lock it up and don't let anyone else review it.

[asbo]

That said, you're right - the CRU we *insane* to release anything to the public without reviewing it first. The fact that there's fudge factors anywhere is inexcusable (especially with the gravity of th results), that they then released the code with the fudge factors in there is (you said it perfectly) either collectively arrogant or extremely naive. Or maybe it was reviewed and someone who wanted to take the CRU down from the inside say the issue and didn't red flag it. Or... actually, there's a bunch of reasons that this could have happened - but none that I can think of that are excusable.

My understanding is that all of these documents were not released so much as leaked - either by hackers or a whistleblower - and found on some Russian FTP site. I think CRU very much intended to keep this stuff under wraps.

[Gary Rubin]

Maybe the scientist and/or developer didn't think it would ever be scrutinized - maybe they thought "okay, we know that this is a fudge factor, and is wrong, but we need it in there to get the software and/or data analysis up and running for now with some dummy data, and we'll go back an fix it later, once we've got the real data in", and then it got lost in the haze. Totally true that a better code review and change tracking system with the appropriate processes to execute them would have caught this.

Or the "fudge factor" is a correction to the formulas to get models to match the observed data.

If you are coding something up using "y=mx", with the assumption that all of your data starts at the origin, you might find that "y=mx+b" actually fits reality better. That constant "b" could be called a refinement to the model, or it could be called a "fudge factor", especially by someone who doesn't necessary understand the origins of the equations.

I'm not saying that this is what happened - just pointing out that a "fudge factor" can be used for non-nefarious reasons.

[Cat]

Addressing this comment purely from a philosophical standpoint, you bear much responsibility if the software you write (intentionally or unintentionally) biases the data one way or the other--moreso if you lock it up and don't let anyone else review it.

Philosophically speaking, I agree completely.

That being said -- we have some very smart people who work here who can come up with all sorts of in-depth analysis algorithms that I don't necessarily completely understand (I'm still puzzling over how our algorithm for a lofargram really works). They hand me an algorithm and I turn it into code. How do I know that algorithm isn't biased somehow? I don't.

[PaulG.]

My understanding is that all of these documents were not released so much as leaked - either by hackers or a whistleblower - and found on some Russian FTP site. I think CRU very much intended to keep this stuff under wraps.

You are correct. But I don't think anyone knows for sure if the database was hacked or leaked by a whistlblower. By my understanding the CRU has fought every (European equivalent of the) Freedom of Information Act request for data. It was also revealed last week that CRU deleted much of the old, raw data that was stored on older formats, i.e. paper, etc. All they have of the older data are "value added" data. (Jon Stewart opined: "Value added data? What is that? Numbers fortified with art?" but I digress)

Recently NASA was threatened with a FOIA lawsuit for their climate data. I don't know who is funding CRU, but NASA is US taxpayer funded. I think it's part of the scam that you have to file a lawsuit to get climate data from NASA. The American taxpayer owns that data and should have as much access to it as they wish. NASA refusing to disclose climate data is like NASA refusing to post images from the Hubble.

[crelf]

we have some very smart people who work here who can come up with all sorts of in-depth analysis algorithms that I don't necessarily completely understand... They hand me an algorithm and I turn it into code. How do I know that algorithm isn't biased somehow? I don't.

Right - and even if there's a comment in there that says "fudge factor", if you're just the coder then it might not be your place to question it if your jobe is purely implimentation. Again, I'm not saying that what's happen is right, I'm just saying that there might be more than what we're seeing here.

[Phillip Brooks]

Have a webcam? Submit your climate debate question to CNN via YouTube.

They will probably only accept questions that allow them to show associated background video of devastation and suffering, but hey; who knows?

If you do, share a link to your video question!

(There are already many ClimateGate questions posted; mark them up or down as you see fit)

[Gary Rubin]

Right - and even if there's a comment in there that says "fudge factor", if you're just the coder then it might not be your place to question it if your jobe is purely implimentation. Again, I'm not saying that what's happen is right, I'm just saying that there might be more than what we're seeing here.

Didn't I just say that?

[crelf]

Didn't I just say that?

Paraphrasing is the sincerest form of flattery

[Anders Björk]

This is not a case of one random scientist behaving badly. It is a case of several powerful scientists in the area of climate research conspiring to silence critics and manipulate data to achieve a political goal. The IPCC is not a scientific body; it is a political organization. If you read the links you will find many reports of obvious flaws in the IPCC's "peer review" process.

I do not agree that most of the scientist in the IPCC are fussy in their science and the science they have as basis for their statements are not well founded. It is not that political. Well IPCC are composed of scientists, all peer-review have its flaws , it science problem in general. The fact is the US har increase it CO2 emmisions with 26% since Kyoto, therefore companies in US has not save as much energy as you would have done if US would have ratified that treaty. 12 years have been lost already. So that Climate Gate is so large in US and not in Europe is not surprising, it is threat for many industrialists. They do not think of the benefits or the alternative cost..

It is large risk if the climate would change as in IPCCs worse case scenario, the cost are very very high, enormous! The cost to do change although the decision basis might(a very low risk) be wrong is much much less.

Sustainable energy is a separate (though related) issue from AGW. I happen to agree we should be actively researching energy alternatives. The correct way to pursue that goal is NOT by feeding the public fear-based propaganda; it's by finding a real argument that changes the minds of your opponents.

-----------------

Incidentally, AGW may be real, it may not be--but it's clear to me we need to step back and look at the issue more thoroughly and more transparently before we take action.

In many European countries the adoption of the Kyoto protokoll has made us move faster against greener energy like windpower and biogas. The heavy industries have also seen the that benefits for saving energy is very large and have short pay-off time.

Edited December 7, 2009 by Anders Björk

[Cat]

Or the "fudge factor" is a correction to the formulas to get models to match the observed data.

If you are coding something up using "y=mx", with the assumption that all of your data starts at the origin, you might find that "y=mx+b" actually fits reality better. That constant "b" could be called a refinement to the model, or it could be called a "fudge factor", especially by someone who doesn't necessary understand the origins of the equations.

I'm not saying that this is what happened - just pointing out that a "fudge factor" can be used for non-nefarious reasons.

This is exactly what my "fudgeFactor" is. It was developed years before I started on this part of the project, so last time I was testing at that lab I insisted they take me thru the derivation. Turns out it was off by quite a bit (new sensors with different sensitivities, different gain settings, etc). So now I have a new, improved (and thoroughly documented) value to throw into the equation. But I still call it "fudgeFactor" because that is more interesting sounding than "refinement to the model".

[Grampa_of_Oliva_n_Eden]

I do not agree that most of the scientist in the IPCC are fussy in their science and the science they have as basis for their statements are not well founded. It is not that political. Well IPCC are composed of scientists, all peer-review have its flaws , it science problem in general. The fact is the US har increase it CO2 emmisions with 26% since Kyoto, therefore companies in US has not save as much energy as you would have done if US would have ratified that treaty. 12 years have been lost already. So that Climate Gate is so large in US and not in Europe is not surprising, it is threat for many industrialists. They do not think of the benefits or the alternative cost..

It is large risk if the climate would change as in IPCCs worse case scenario, the cost are very very high, enormous! The cost to do change although the decision basis might(a very low risk) be wrong is much much less.

In many European countries the adoption of the Kyoto protokoll has made us move faster against greener energy like windpower and biogas. The heavy industries have also seen the that benefits for saving energy is very large and have short pay-off time.

I believe it was an old vampire movie that featured a scene where someone pointed out the garlic to keep vampires away. Another replied "there are no vampires around here!" and the first replied "see how well it works!"

This is not just a joke in an old movie. Here in the US it is common practive and is concidered good edicate (?sp? behaviour) to say "God bless you" when someone sneezes. Strangers that would normally say nothing to you will say "God bless you." if you sneeze in public. From what i understand, that practices goes back to the dark ages when it was observed that shortly after people started sneeizing they would become ill and often act posessed afterwards. The thought at the time was that the soul left the body at the moment of a sneeze and the sound of the sneeze was the soul screaming. Since the body was temporarily devoid of a soul the person was subject to demonic possesion so say "God bless you." was a way to help prevent demonic possesion. My point is that un-founded beliefs can become part of our lives if nobody stops to question the pratice.

Human nature makes me nervous when a change in behaviour is attributed to a supposed "truth".

I can see a desperate move on the side of those that want to push forward with all of the possible implications if global warming is shown to be real. If they can get the rules set in stone soon they can point at the lack of global warming as a sign "Look how well it is working!". The last thing we should do in rush into something that has so many far reaching reprecusions.

THe move by the EPA yesterday (I question the timing) that CO2 and five other green-house gases are now recognized as being harmful to humans. Under existing US law the EPA is now obligated to regulate these gases.

Will they?

I can't say.

But if they do we COULD find out who Cass Sunstien is and his out-look on the methane producing cattle we insist on eating.

[set Kring's Law = suspended]

The fermentaion of beer produces CO2 and COULD be regulated.

[set Kring's Law = active]

THe remainder of this post was omitted because I already sound crazy enough.

Ben