Reply to comment by von Storch and Zorita on “Hockey sticks, principal components, and spurious significance”

[1] Von Storch and Zorita [2005] (hereinafter referred to as VZ) concur that the Mann et al. [1998] (hereinafter referred to as MBH98) principal component (PC) method “very often shows a hockey stick shaped pattern even if the data was by construction free of such structures” - the Artificial Hockey Stick (AHS) effect. We did not claim that the AHS effect applied to all situations. We did claim that it affected MBH98 [McIntyre and McKitrick, 2005a, 2005b] (hereinafter referred to as MM05a and MM05b, respectively), where the AHS effect interacted with flawed bristlecone proxies. [2] VZ provided a simulated example where the AHS effect does not “matter”. Unfortunately, VZ pseudoproxies dramatically overestimate the correlations of MBH98 proxies to gridcell temperature. This results in both the construction of a much stronger “temperature signal” in VZ simulations than is justified for MBH98 15th century proxies and the exclusion of proxies with strong nonclimatic trends, like bristlecones, which was a focus of our articles, making their example irrelevant. We discuss what is necessary to create a relevant simulation. [3] VZ pseudoproxies were not constructed by using actual correlations to gridcell temperature for MBH98 North American tree ring and other proxies. Instead, they relied on information from another study [Jones and Mann, 2004], using a different data set. As a result, they endowed their pseudoproxies with correlations to gridcell temperatures (0.3–0.7) that are much higher than actual MBH98 proxies. The mean correlation of the 15th century MBH98 North American tree ring proxies to gridcell temperatures is −0.08(!), although the mean correlation to precipitation (0.29) is close to VZ standards, as shown in Figure 1. [4] VZ also failed to model pseudoproxies with low temperature correlation but strong nonclimatic trends. A significant fraction of MBH98 proxies (bristlecones) have correlations to CO2 levels exceeding 0.4 but a mean correlation to gridcell temperatures of only 0.01 (see Figure 1 and auxiliary material). Without them, there is no significant hockey stick shape in any high-order PCs nor in the NH temperature index itself (see MM05b). [5] The inaccurate modeling of pseudoproxies is exacerbated by inaccurate replication of the MBH98 PC method itself. MM05a pointed out that MBH98 computed PCs by applying a singular value decomposition (SVD) directly on decentered data and that use of PCs calculated using the correlation or covariance matrix (of the decentered data) would have mitigated the AHS effect (this alone would have reduced the hockey stick index for the 15th century North American PC1 from 1.6 to 0.9). VZ said that the MM05a Monte Carlo study calculated PCs on the correlation matrix; in fact, following MBH98, we used the decentered data matrix in our simulations. From this, we surmise that VZ calculations were not done by SVD on the decentered data matrix; this will have a material impact on their yield of hockey sticks. [6] While it is widely assumed [e.g., von Storch et al., 2004] that the MBH98 multiproxy method is inherently robust to defects in a few individual proxies, we have found exactly the opposite: their method is extraordinarily sensitive to the inclusion of even a few hockey stick–shaped proxies with nonclimatic trends. VZ attributed the AHS effect to the fact that, using red noise simulations, “no real structures steer the eventual selection of the eigenvectors,” and argued that this would not apply if the pseudoproxies had a strong temperature signal. However, the AHS effect is not limited to red noise situations. It is even more powerful if there are some series with a strong nonclimatic trend; then a few “bad apples” strongly “steer” the selection of eigenvectors, especially if there is a weak temperature signal. [7] To demonstrate this, we carried out new analyses for this reply (see Figure 2). MM05a showed that the hockey stick shape of the MBH98 North American PC1 was entirely due to 16 bristlecone series. In our new calculations, we sequentially pruned the number of bristlecone sites in the NOAMER network from 16 to 0, reserving the most dominant series (Sheep Mountain) for last, carrying out an MBH98-type PC analysis on each network. The PC1 hockey stick index (HSI), as defined by MM05a, was essentially unchanged (>1.4) down to only 3 sites (Figure 2 (top)) with the weighting of bristlecone sites in the PC1 and PC2 (Figure 2 (bottom)) remaining nearly constant. As other bristlecone sites were pruned, eigenvector weightings on these 3 sites increased dramatically, essentially taking up all the weight previously allocated to all 16 sites, remarkably demonstrating the “steering effect” of a few series. [8] Even two bristlecone sites were sufficient to steer the selection of eigenvectors; the HSI declined slightly to 1.1 with very high weights allocated to only 2 sites. Even one hockey stick–shaped site was sufficient to steer the selection of eigenvectors under the MBH98 method, yielding a seemingly “significant” hockey stick shaped PC2 (HSI - 0.92) with a dominant loading from only one site. In each case, even with only one site, the affected PCs appear to be “significant” under a naïve application of Preisendorfer's Rule N [Preisendorfer, 1988]. [9] Thus, the steering effect of the AHS method is not limited to a red noise situation, but also affects a network of real data including only one proxy series with a nonclimatic hockey stick shape. The VZ assumption that all proxies have a correlation of 0.3–0.7 to gridcell temperature eliminates such “bad apples” with their powerful “steering” effect. However, it is the combination of the steering effect of these proxies, together with the very low correlations to temperature of the majority of proxies and the use of SVD on the decentered data, that made the MBH98 method vulnerable to generating a hockey stick shaped PC1 series, an effect which is precluded by the very construction of the VZ simulations. As a result, the hockey stick–shaped PC1s emerging from the VZ simulations are likely to be attenuated relative to ones generated from a more realistic simulation of MBH98. [10] After tree ring PC calculations, there is a further MBH98 stage in which tree ring PCs are formed into a network with other “proxies” (20 in the AD1400 network). In the actual MBH98 AD1400 network, the mean correlation to gridcell temperatures of these 20 proxies is only 0.13, as compared with the unrealistically high VZ correlations of 0.3–0.7. In VZ simulations, the strong temperature signal from these pseudoproxies will counteract the attenuated hockey stick coming from their tree ring network. A more accurate simulation of MBH98 would feature a stronger hockey stick shape in the tree ring PC1 and essentially white noise in the remaining proxies. McIntyre and McKitrick [2005c] (hereinafter referred to as MM05c) report on this combination, which regularly yields spuriously high RE statistics relative to NH temperature. [11] MM05a reported that a distinct result of the AHS effect was a spuriously high RE statistic combined with negligible R2 (∼0.0) and other cross-validation statistics. Thus a test of whether VZ pseudoproxy simulations accurately replicate the MBH98 environment is whether they yield a seemingly high RE statistic (say >0.4) together with a cross-validation R2 statistic of ∼0.0. This outcome can be obtained by simulated PC1s generated from red noise (MM05a), or with simulated PC1s combined with a white noise network (MM05c). [12] Unfortunately, VZ did not provide cross-validation statistics from their simulations or digital output or source code from which such statistics could be calculated. However, noting the relatively high pseudoproxy-temperature correlations in VZ, we conjecture that, if their RE statistics are high, so are their R2 statistics. Unless they can show that they have constructed an example with high RE and low R2, they have not captured the particular conditions in which the AHS can indeed be expected to “cause significant harm when reconstructing past climate.” [13] The Artificial Hockey Stick effect does not affect all situations. VZ have provided an example, but one which is irrelevant to evaluating MBH98. Had they examined a network of pseudoproxies with negligible correlation to temperature mixed with a few pseudoproxies influenced by a nonclimatic trend, using SVD on a decentered data matrix, they would have regularly obtained hockey stick–shaped NH temperature reconstructions with a spuriously high RE statistic. Auxiliary material for this article contains a figure and a table. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.