The Hockey Stick

This isn’t going to be full accounting of the Hockey Stick controversy. You can read the post at the American Institute of Phsyics or Wikipedia if you want more gory details, or Mann’s book if you want a first-hand account.

I am instead going to look a one “skeptical” account of the controversy and point out a few places where it goes wrong. (I’ll leave it to you to decide whether it should be considered an example of denialism.)

This account can be found on an anonymously authored blog called The Skeptical Mind. I can find no indication of when the piece was posted, so when I point out below that the piece fails to discuss various subsequent developments, I can’t say whether this is because those papers hadn’t yet been published, or because the author simply ignored them. But this is irrelevant for deciding whether we should accept his account (though it presumably is relevant for deciding whether the author should be considered a “skeptic” or a “denier.”)

I’ll call the author of the post SM, since I don’t want to keep saying “the author.”

1) SM starts off by laying out the background of the Hockey Stick controversy, and this is where the trouble begins.  SM tells us that the question of whether the Earth was warmer during the Middle Ages than it is today is very important for the validity of the IPCC’s warnings about climate change:

Why is this claim so important?

Because if a similar or greater warming phase has occurred in the very recent past, before human CO2 emissions had caused CO2 levels to rise, then clearly any such recent warming must have been natural and was not caused by CO2. And if any recent similar warming phase was natural then clearly the current phase of warming could also be a natural phenomena

This is wrong.

Even if the MWP (Medieval Warm Period) was globally warmer than today, that doesn’t imply that “the current phase of warming could also be … natural.” It might be the case that there was something that caused the Earth to warm in the Middle ages, but that something is not playing a role today. If that were the case, then we should conclude that even though earlier warming was natural, today’s warming is not.

Is there such a cause?  And if there is, is it playing a role today?

There are a handful of likely contenders: increased solar activity, decreased volcanic activity, sustained changes in ocean currents.

Is it plausible that any of these are causing of the current global warming trend?

Solar activity has been holding steady while global temperatures have been skyrocketing. Volcanoes have likewise not been unusually quiet. And we don’t have long-term El-Nino-like currents driving warming either.

And yet we know that it’s a basic physical fact that carbon dioxide retains heat, and we know for certain that we’ve been increasing atmospheric CO2 considerably.

So regardless of the temperatures in the MWP, we have reason to be concerned today. Indeed, if you look at the the statements issued by various professional science organizations (e.g., the AAAS, the American Physical Society, or the American Chemical Society), you won’t find them relying on the paleoclimate studies. The physics is enough to cause serious concern.

2) SM then claims that the paleoclimate reconstructions of Mann, Bradley, and Hughes (MBH) constituted a major upheaval of previously established climate science:

Mann completely redrew climate history, turning the Medieval Warm Period and Little Ice Age into non-events. In the new Hockey Stick diagram the Medieval Warm Period and Little Ice Age have disappeared, to be replaced by a largely benign and slightly cooling linear trend in climate until 1900 AD after which the Mann’s new graph showed the temperature shooting up in the 20th century in an apparently anomalous and accelerating fashion.

In every other science when such a drastic revision of previously accepted knowledge is promulgated, there is considerable debate and initial scepticism, the new theory facing a gauntlet of criticism and intense review. Only if a new idea survives that process does it become broadly accepted by the scientific peer group and the public at large.

This never happened with Mann’s `Hockey Stick’. The coup was total, bloodless, and swift as Mann’s paper was greeted with a chorus of uncritical approval from the increasingly politically committed supporters of the CO2 greenhouse theory. Within the space of only 12 months, the new theory had become entrenched as a new orthodoxy.

To demonstrate the significance of this supposed revolution, SM offers the following comparison of a graph from the 1990 IPCC report and the “Hockey Stick” graph.
Early Hocky Stick

So, the claim is that when the global warming alarmists had to somehow get rid of the inconvenient MWP, Mann et al. came along and conveniently (and illegitimately) did precisely that. How plausible is this claim?

First, notice how odd it is for someone to place more trust in an older, clearly less-sophisticated, summary of scientific data than a more up-to-date summary. Indeed, if I recall correctly, no one even remembers the origins of the 1990 graph, let alone the data and methods that were used to produce it. And it’s not like paleoclimatology is a huge well-established field that is unlikely to change its mind over the course of a decade or two. I’d wager that more data was produced between 1990 and 2000 than was ever produced prior to 1990.

Second, it’s not true that the MBH paper offered a “new theory” that did away with the MWP and the Little Ice Age. Non-scientists are likely to look at the above Hockey Stick Chart and only notice the blue line in the middle, but scientists look at the chart and see the grey region surrounding that line. That’s a depiction of the uncertainty. Given that explicit uncertainty in the study, it’s not even clear (to my untrained eye at least) that the two charts above are incompatible with each other. The MBH paper was surprising, but hardly constituted a scientific revolution.

Third, it’s also not true that the MBH study was “greeted with a chorus of uncritical approval” by the climate community. It was treated as an important contribution, but it was only one more piece in the ongoing research to uncover the past climate of the Earth. There were legitimate scientists offering legitimate criticisms; it looked very much like most scientific discussions: arguments, data, challenges, questions, and so on. Science is a messy business, and it takes time for things to settle down enough to know what we can rely on.

However, a fair bit of rhetorical weight was put on the MBH graph. It was, for example, featured rather prominently in the Summary for Policymakers of the 2001 IPCC report, even though though paleoclimate research was only one of a dozen points highlighted for policy makers. And notice that the claims only speak about the Northern Hemisphere (not the whole globe), and only say that the claims are “likely”:

New analyses of proxy data for the Northern Hemisphere indicate that the increase in temperature in the 20th century is likely (7) to have been the largest of any century during the past 1,000 years. It is also likely (7) that, in the Northern Hemisphere, the 1990s was the warmest decade and 1998 the warmest year.

The “7” following the claims that the increase is “likely” unprecedented in the last millennium take us to a footnote reminding us that “likely” means a 66%-90% chance. So the IPCC report is saying that there’s about a one-in-four chance that the claim could be wrong. Does this sound like a statement of a “new orthodoxy” or a claim that is supposed to undergird all the rest of the science in the report? It doesn’t to me.

3) SM then goes on to discuss the McIntyre and McKitrick paper(s) criticizing the MBH paper, and presents it as a devastating critique:

[Y]ou use random numbers as input data, then apply the statistical technique you are testing to the random numbers then if the techniques are sound you should get a set of random numbers coming out the other end of the calculations. There should be no false shape imparted to the random noise by the statistical techniques themselves, if what you get out is random numbers then this would prove that the techniques you were testing were not adding anything artificial to the numbers. This is what McIntyre and McKitrick did using the techniques that Mann had used in the Hockey Stick paper. And the results were staggering.

What they found was that 99% of the time you could process random data using Mann’s techniques and it would generate a Hockey Stick shape. This meant that Mann’s claim that the Hockey Stick graph represented an accurate reconstruction of the past climate was in tatters.

Here are some examples. Below are eight graphs. Seven were made by processing random numbers using Mann’s techniques. The eighth is the actual Hockey Stick chart from Mann’s paper. See if you can spot which is which.


Sounds pretty awful, right? I mean, who would believe those climate scientists after something like this? How could Mann and his buddies possibly reply?

Well, first, they pointed out that those seven graphs of red noise above were cherry picked for their strong hockey stick appearance. If you got the impression that most of the analyses of noise looked like this, you were misled. In fact, far less than 1% of the analyzed noise data looked like this.

Second, it’s not clear that the claim that the analysis was actually performed on pure red noise the way M&M claimed. Scientists tried without success to verify the claim that pure red noise would produce “hockey sticks” so consistently.

But further analysis did indeed reveal that M&M were correct that the BBH statistical analysis introduced a likely bias towards a warming trend. The important question (which doesn’t seem to occur to Sceptical Mind) then becomes just how strong is this bias?

Several statisticians and scientists looked into this, and the consensus view (reported e.g., by von Storch and Zorita) is that the bias in the MBH technique “does not have a significant impact but leads only to very minor deviations” when it is applied to white noise, red noise, and actual and simulated climate data.

So a mistake of sorts was made, but it was insignificant. And MBH knew even before these analyses that the issue in question (namely, which time period to “center” the principle component analysis over) would not be relevant to their study. It is very often the case that scientists use mathematical techniques that are not completely rigorous, but which work well enough for their purposes. Often times (as with the Dirac Delta function, and with this analysis of the MBH technique) mathematicians can later come in and either justify the technique or analyze whether the lack of rigor made any difference.

4) SM next reports on McIntyre and McKitrick attempt to reconstruct the MBH graph from the MBH data set, and points out that M&M fail to get the same result as MBH.

In the graph below the dotted line is the original Hockey Stick chart as published by Mann and as adopted and promoted by the IPCC. The solid line shows the past temperature reconstruction if the data used by Mann is averaged using the correct statistical analysis techniques rather than Mann’s unconventional ones. As can be seen the familiar Medieval Warm Period re-emerges and the 1990s cease to be the hottest of the millennium, that title is now claimed by the early 1400s.


Except, if you look at the graph, you’ll see that it doesn’t display the Medieval Warm Period at all. The MWP is generally dated from about 950 – 1250 CE. The M&M chart only goes back to 1400. So what’s up with all that warming in the 15th and 16th centuries?

The answer, as it turns out, is that M&M’s statistical analysis of the data is wrong. To understand why they’re wrong requires us to go into principle component analysis (but I won’t go very far).

In short, PCA (principle component analysis) is a way of looking for trends when you have many different kinds of data. This is very helpful for studies like MBH 1998 because it allows you to take tree ring studies, coral samples, lake bed cores, and so on, and try to extract significant trends in temperature from all this data.

Now, one of the questions that researchers face when using this method is how to go about deciding which “principle components” (that is, the dimensions of data that show a trend) they should include in their analysis. This can be particularly difficult when you have many different studies that cover different spans of time.

What MBH did was to consider all the studies that covered a particular block of time, find the principle components that were relevant, and use that as the basis of their statistical analysis. They were able to show that the top two principle components were all they needed to consider; the other dimensions of data didn’t change the results significantly.

When M&M went through the data, however, they looked for the principle components for essentially the entire data set, and then they used the top two components for their analysis. This meant that their analysis essentially ignored the vast majority of the data.

Now, one can use a wider technique like M&M’s, but if you do, you’ll need to include more than the top two components. You need to go deeper to find the trends. If you look to see how many components are statistically significant, you find that you would need five components to get the sort of accuracy that MBH required in their paper.

So basically, M&M screwed up. Their mistake was pointed out in the peer-reviewed science literature. And Sceptical Mind didn’t notice.

5) SM also reports M&M’s claim that the SBH result crucially depends on the inclusion of a particular set of data from cores of bristlecone pines.

This claim was also analyzed in the literature and found to be mistaken. For example Wahl and Ammann reach the following conclusion:

The results presented here show no evidence for removing the MBH Northern Hemisphere temperature reconstruction from the list of important climate reconstructions of the past six centuries, on the basis of alleged “flaws” in its use of proxy data or underlying methodology. Indeed, our analyses act as an overall indication of the robustness of the MBH reconstruction to a variety of issues raised concerning its methods of assimilating proxy data, and also to two significant simplifications of the MBH method that we have introduced. The shape of a single-bladed “hockey stick”-like evolution of Northern Hemisphere temperature over the last 600 years is strongly confirmed.

Their analysis shows that M&M’s truncated data would fail to meet the criteria for being good proxy data that was set out in the MBH paper (and that is widely accepted). That is, the M&M analysis should be viewed as a statistically insignificant spurious result, not a reliable record of the past climate.

I’ll close by reiterating that this isn’t supposed to be anything like a complete account of the Hockey Stick controversy.  At best, it should be read as a set of reasons for being very skeptical of a set of denialist talking points.  I suggest that the above five points give us good reason to distrust Skeptical Mind’s account of the controversy, and to be especially skeptical of SM’s assessment about the IPCC claims being “in tatters” and so on.

Of course, my pointing to critical scientific papers doesn’t decisively settle any questions about paleoclimatology.  For one thing, I’m no climate scientist; I’m sure my understanding of the science is flawed to some degree.  However, I do like to think that that I have a bit of expertise in history and philosophy of science, and in the study of denialism and pseudoscience.  So while you shouldn’t trust me on the science, you might consider trusting me when I say that the arguments on the Skeptical Mind page should are closer to denialist tactics than they are to legitimate scientific criticism.

For another thing, it would be a mistake to suppose that the debate ended with the papers I cited above.  There have been replies to the replies, and replies to the replies to the replies, and so on.  That’s how science works.

As is often the case, there was no complete grand resolution; instead scientists just turned to more fruitful questions and projects.  Mann changed his statistical analysis to one that is less problematic (and still finds a hockey stick in his studies).  Many other proxy summaries have been published (which also display the hockey stick pattern).  Science moves on and improves.  If you want to understand where we stand now, I’d suggest reading the latest IPCC report, rather than worrying about debates that are over a decade old.  But also remember that very little of scientists’ worries about climate change rests on conclusions from paleoclimatology.

*** *** ***

So, what lessons should we draw from this? Here are some suggestions:

a. Don’t trust anonymous bloggers more than trained scientists.

b. Sometimes people say false things that sound convincing, and even when the story is true, it might not be relevant. Reading that MBH made all these mistakes seems horribly important, until you find out that (a) they didn’t really make all of those mistakes, (b) the mistakes they did make didn’t change the overall result, (c) that many competent scientists have looked at these criticisms and found that they fail to undermine the science reported in the IPCC.

c. It is often quite difficult to figure out what the relevant facts are. If you aren’t willing to invest a lot of time and effort digging into research papers and analyses of data, you might want to trust the judgment of the vast majority of scientists (e.g., the AAAS, the American Physical Society, and the American Chemical Society).

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s