Editorial: (Post-normal) Statistical Science

Speaking into my microphone, I tried to explain why a monitoring network for monitoring ozone might not be adequate for the things that really matter, namely the extremes. 16 experts with impeccable scientific credentials sat behind their microphones around a rectangular array of tables, listening to me. Seven others were on speakerphones from remote locations. Over a 2-day period, our multidisciplinary panel, convened by the US Environmental Protection Agency, was to review a 1700-page draft document containing just about everything that been discovered about ozone since the last such review in 1996. Later, we are to help the Environmental Protection Agency to formulate new air quality recommendations for photo-oxidants including ozone. The panel sat in a corner of a large hotel ball-room. The rest of the room was filled with an audience of about 80 people, including about 50 of the draft's authors along with representatives of various special interest groups who had come to listen and, in some cases, to present a report in no more than 5 minutes. The proceedings were being recorded. The meeting was open to the public. Discussion focused on what was in the draft and what was not, on what was certain and what was not, and on how to incorporate that uncertainty in formulating public policy based on science and how not. We were engaged in a process that exemplifies post-normal science (PNS)! I am not sure who coined that phrase, but I first read it in an article on the World Wide Web by Funtowicz and Ravetz (undated). It refers to a category of processes involving scientists and science that lead to conclusions and commonly to policy decisions. They write: ‘in relation to policy, “the environment” is particularly challenging. It includes masses of detail concerning many particular issues’: like our 1700-page draft! The tasks differ from those of normal science owing to complexity, specifically radical uncertainty, a plurality of legitimate perspectives and very high risk. PNS embraces issue-driven scientific inquiry related to environmental controversies; the facts are uncertain, values in dispute, stakes high and decisions urgent. Everything, the choice of what to measure, the modelling approaches and the formation of theoretical constructs, is shaped by societal values. Rather than knowledge discovery as in traditional science, the ultimate goal of its post-normal successor is consensual agreement, what one person called ‘convergence’ at our panel meeting. Objectivity has given way to intersubjectivity. The quality of their eventual outputs—conclusions and maybe policy decisions—depends on the processes that produced them rather than just on the experiments as in normal science. The Environmental Protection Agency has been exemplary in developing and refining such processes, through which scientific debate can be channelled for policy making. PNS has a handmaiden, mandated science, as described in Salter (1988). Its realm lies in the intersection of values, public policy and science according to Gerald van Belle who in a recent lecture characterized mandated science as taking place in public view, often adversarial, having a multiplicity of stakeholders and involving concern for accountability (the bottom line!). Interest groups have well-honed strategies for exploiting PNS's uncertainty: raising doubts (e.g. confounding and measurement error); asking the wrong questions; making numerous freedom-of-information access requests and using other forms of harassment; reanalysing the data, perhaps with different models or assumptions. There is plenty of scope here for a post-normal statistical scientist! The widely held belief in global climate change illustrates well a post-normal scientific conclusion, one that led to the Kyoto accord, a policy decision. In fact, Oreskes (2004) concluded on the basis of her survey of 928 abstracts that ‘none of the papers disagreed with the consensus position’ and that ‘there is a scientific consensus on the reality of anthropogenic climate change’. That prompted the Royal Society's Vice-President, Sir David Wallace, to send the media a letter asking them to ignore ‘individuals on the fringes, sometimes with financial support from the oil industry, who have been attempting to cast doubt on the scientific consensus on climate change’. His position was supported by Lord May, the Society's President, according to an article in the UK's Daily Telegraph (2005). Here we have intersubjective agreement! However, high levels of uncertainty have left that consensus open to controversy along with the accord. According to Matthews (2005), an attempt to replicate the Oreskes study by social anthropologist Benny Peiser found that only about 1% of those abstracts supported the hypothesis of anthropogenic warming and most did not even mention it. Peiser reported those conclusions in a letter that he claims Science‘refused to publish’. The plot thickens! According to Matthews, Science also refused to publish a similar finding of Dennis Bray (German Institute for Coastal Research). Even the Intergovernmental Panel on Climate Change is not spared. McIntyre (2005) asserted that the hockey stick diagram prompted the Panel to change its 1990 conclusion that the mediaeval period was warmest in the last millennium, in favour of the 1990s. However, he wrote that he and his co-author, Ross McKitrick, have shown that the hockey stick diagram resulted from flawed analysis, in particular from data mining methods that would have produced hockey stick patterns even in a random series (see for example McIntyre and McKitrick (2005)). That analysis and others like it might have benefited from critical analysis of statisticians before its publication. After all, statistical science is tailor made for PNS and its complexities, uncertainties being admitted through probabilities and values through utilities. Not surprisingly, statistical scientists did become involved, after its publication. One, Bjorn Lomborg, an Associate Professor of Statistics (Department of Political Science, Aarhus) wrote what the Economist (September 6th, 2001) described as ‘… one of the most valuable books on public policy—not merely on environmental policy—to have been written for the intelligent general reader in the past ten years’. (See Lomborg (2001).) Another, Ian Castles, former President of the International Association of Official Statisticians, criticized the Intergovernmental Panel on Climate Change's ‘Year 2000 special report on emission scenarios’ on statistical shortcomings, according to Agerup (2004). In fact, statistical scientists should be involved in mandated science and PNS so that those in other disciplines do not usurp their legitimate roles. Yet meeting this requirement has not been easy as the climate example demonstrates. I wonder why. Are we blocked from doing so? Or does our nature lead us to shun roles on centre stage, preferring the wings instead? Or perhaps we do not have the requisite skills for handling the complicated interactions that are involved. In any case, along with opportunities our discipline faces big challenges in this era of PNS. For one thing, investigators are increasingly forced to trade core research, which is vital to the development of the discipline, for research that is mandated by the granting agencies. I am reminded that just a few months ago a man who has made important contributions to the field of environmetrics confided that he was very reluctantly changing fields to take advantage of the comparative abundance of funding that is available for genome research. I doubt that he is alone! Some veterans seem able to squeeze out some core research by disguising it under their mandated funding. However, the effect on new researchers is difficult to predict. To be sure, they will enjoy their high levels of support and therefore accelerated career advancement. Yet their work will be heavily constrained by its targeted nature and urgency. I wonder whether they will be able to reach their full potential and to take full advantage of their intellectually fertile years. The bottom line is the need to monitor the influence of PNS on our discipline and to ensure that young investigators along with graduate students are well schooled about its positive and negative effects. Currently little guidance seems available. The Royal Statistical Society's code of conduct, for example, gives good general guidelines on professional practice, but these can be difficult to interpret in the context that is addressed by this editorial. For example, given our keen interest in global climate change, the Society's rule 5 will leave us in doubt about exactly what and how we can contribute to the debate: ‘Fellows shall not purport to exercise independent judgement on behalf of a client on any product or service in which they knowingly have any interest, financial or otherwise’. What should we purport to do instead? Little specific guidance concerns conduct in adversarial proceedings although we are ‘allowed to engage in controversy’ so long as we do not ‘cast doubt on the professional competence of another without good cause’ (rule 10). In any case, the code addresses only professional conduct, not the more subtle issues concerning our obligations to nurture and protect statistical science against the strong currents of PNS and mandated science. A partial solution lies in ensuring that statistical education is sufficiently broad to acquaint statistics graduates with the challenges that are presented by PNS while equipping them with the enhanced skills that are needed to cope with them and even selectively to take advantage of the opportunities. In particular, the statistical consulting sequence that is commonly found in statistics graduate programmes might be expanded to include multidisciplinary meetings where a multiplicity of legitimate views are presented in an adversarial context. Certainly, a good education must equip students to address situations involving radical uncertainty, decision-making with a multiplicity of legitimate perspectives and interdisciplinary inquiry. Graduates should be able to take their legitimate place at the head table of science and to offer clear and intelligent criticism as well as comment. This brings me back to my microphone. It is time to discuss the transfer of causality that can occur due to a confluence of collinearity and measurement error. Is ozone really the cause of all those documented health impacts?

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.