Report of an Expert Panel to Review the Socio-Economic Models and Related Components Supporting the Development of Canada-Wide Standards (CWS) for Particulate Matter (PM) and Ozone To the Royal Society of Canada

Abstract Canada-Wide Standards (CWS) for particulate matter (PM) and ozone may be the most ambitious environmental standards ever proposed in Canada. They have attracted considerable attention and debate. This report addresses the validity of the socioeconomic modeling aspects of the Canada-Wide Standards development process. Socioeconomic considerations are addressed in one of eight principles underlying the development and attainment of CWS, according to a CWS subagreement signed by the Canadian Council of Ministers of the Environment (CCME). Principle 3.1.7 states that measures to attain agreed-upon Canada-Wide Environmental Standards will be determined in a sustainable development context, recognizing environmental and socio-economic considerations. This work has been previously published by the Royal Society of Canada, Ottawa (Ontario), in June 2001, appearing as "Review of Socio-Economic Models and Related Components Supporting the Development of Canadian Standards for Particulate Matter and Ozone," ISBN 0-920064-73-6. A CCME Framework for Socio-economic Analyses in Setting Environmental Standards (CCME, 1998) describes procedures and information requirements for socioeconomic assessments of potential or proposed environmental standards. This framework states that while it may not be possible or necessary to carry out all of the analytical steps because of time, data, or resource constraints, a partial assessment can produce information that is useful for policy deliberations. The framework also notes that socioeconomic findings are not intended to be prescriptive concerning decisions about environmental standards because other input factors such as toxicity, epidemiology, ecological consequences and geographical distribution of effects, and other equity considerations are also necessary and important to an informed choice with respect to standard setting. Socioeconomic considerations are also specified under Government of Canada regulatory policy requiring federal regulatory authorities to demonstrate that the benefits of regulatory requirements are greater than their costs. Regulatory authorities must "ensure that the benefits outweigh the costs to Canadians, their governments and businesses. In particular, when managing risks on behalf of Canadians, regulatory authorities must ensure that the limited resources available to government are used where they do the most good" (Government of Canada, 1999). This implies not only that benefits should be greater than costs, but also that benefits minus costs, or net benefits are to be maximized, which means an attempt should be made to make standards efficient. The objective of the expert panel process was to provide an independent, expert review and critique of the socioeconomic analyses (SEA)—in this case a cost-benefit analysis (CBA)—conducted in developing the Canada-Wide Standards on PM and ozone. Through a review of the models and associated data and assumptions used in the analyses, the panel was asked to produce a report to address the following questions: What are the strengths, merits, limitations, gaps, and the degree of uncertainties of the proposed approaches, models, and their inputs and outputs? By what means could the models and analytical approaches be improved, so as to minimize uncertainties and maximize the relevance, reliability, and utility of outputs? What other approaches and/or tools could be used to conduct these analyses? The benefits and costs associated with Canada-Wide Standards for PM and ozone are highly uncertain and controversial. Uncertainties are associated with each step in the analysis of benefits and costs—including the link between emission reductions and ambient air quality, the extent to which human health and the environment are affected by changes in ambient levels of pollutants, the economic values (as measures of preferences) associated with improvements in environmental and human health, accuracy of the emissions inventory and projections of what this inventory and other factors will be in a future baseline, and the scope and magnitude of economic costs associated with emission reductions, both to industry and to society. With uncertainties so pervasive, analysts are required to make many choices and assumptions in estimating costs and benefits. For example, while it is clear that the epidemiological association between PM and excess mortality is consistent and robust, there are many remaining gaps in current understanding of the relative toxicity of PM components and gaseous copollutants and the magnitude of potential life-shortening effects. These uncertainties introduce possible biases into the estimation of PM-related health benefits. The panel draws the following major conclusions from its review of the CBA undertaken for the development of CWS for PM and ozone and from the academic and policy literature relevant to this topic. In drawing these conclusions, the panel views the use of a structured approach to the examination of costs and benefits as a positive development in Canadian regulatory policy analysis. The CWS socioeconomic analysis (SEA) was in fact limited to a cost-benefit analysis (CBA). Because the CWS implementation of the SEA process was judged by the panel to be limited to a CBA, it was reviewed as such. CBA is just one of many available decision support tools. The requirements of CBA in the CWS process depend on whether the purpose is to select an ambient air quality standard or to guide and evaluate the implementation process. The extent to which the results of the CBA can inform the CWS decision process is limited for various reasons including the following: Provinces are required to establish implementation plans to ensure that CWS will be met. Therefore, when doing a prospective CBA, the measures to be implemented (i.e., revised emission control regulations) are necessarily undetermined, as are the compliance levels for the revised control regulations that will be achieved in various provinces. The distributional impacts of both costs and benefits have not been assessed. The panel acknowledges that the CWS Development Committee for PM and Ozone describes the cost estimation as "preliminary and, in some instances, a cursory analysis used to provide a macro level order of magnitude perspective on the costs associated with the various optional levels for PM and ozone CWSs" and notes that caution should be exercised in their interpretation (Canada-Wide Standards Development Committee for PM and Ozone, 1999). Deficiencies in Canadian data and modeling capabilities and limited time and resources restricted the scope of the analysis that could be undertaken for the CWS process. Credible CBA should be conducted to support the development of Canada-wide standards. The panel recommends that CBA be used to inform decision makers about the projected costs and anticipated benefits of CWS. CBA needs to be designed to distinguish between the costs and benefits of meeting alternative PM and ozone standards within the limits of current science. There are potential overlaps in the estimation of costs and benefits for PM and ozone because emission control strategies will impact both PM and ozone levels and it is not clear which components of the air pollution mix are responsible for the various health effects. These uncertainties in the CBA need to be clearly communicated. At its best, CBA provides the decision maker with a systematic identification, estimation, and measure of uncertainty of monetary values for the relevant costs and benefits of interest to decision makers and stakeholders. To be fully informative, the CBA results provided to stakeholders and decision makers need to adequately analyze and explain the major sources of uncertainty in the inputs of the CBA model projections and their likely effect on model outputs. In view of the importance of the proposed regulatory decision, the CBA performed for the CWS for PM and ozone is deficient in relation to the state-of-the-art for CBA. If the CWS CBA was intended to provide an adequate basis for balancing costs and benefits and for influencing where the CWS should be set, this CBA was not up to the task. If the objective was strictly to confirm that costs were not exorbitant for CWS that were deemed to be both technically feasible and associated with some substantial benefits, then this CBA provided contributions toward those judgments. When judged against the elements of process and structure of CBA required for credibility as indicated in conclusions 4 and 5 that follows, the panel finds that the CWS CBA does not satisfy these requirements and does not meet a reasonable level of quality for a CBA to support a decision of this import. While the CWS CBA has some value as a scoping analysis and provides a limited degree of guidance for decision-makers, it requires substantial improvement to meet the criteria for credibility. The process for using CBA in CWS needs improvement. A Discussion Paper on Particulate Matter (PM) and Ozone Canada-Wide Standard Scenarios for Consultation prepared by the Canada-Wide Standards Development Committee for PM and Ozone (May 1999) states that "in selecting PM and ozone CWS level scenarios for stakeholder considerations, an attempt was made to balance the anticipated benefits of improved air quality with the technological feasibility and costs of achieving those improvements." But it is apparent that the standard CBA procedure of comparing incremental benefits for tighter standards with incremental costs was not done for the CWS CBA process. Timelines for the analysis were exceedingly short, and the CBA effort appeared to be underfunded, resulting in short-cuts that substantially reduced the credibility of the analysis. Reporting and communicating the CBA results was also ineffective, particularly in terms of conveying a clear understanding of what was done and why it was done as it was. The panel has identified several CBA elements of primary concern that require attention in order to ensure the credibility of CWS CBA. The elements of primary concern for assessing credibility of CBA are the following: Accuracy of emissions inventory data. Accuracy of cost estimates. Use of state-of-the-art air quality models. Sufficiency of air quality monitoring. Use of reasonable baseline assumptions for regulatory regime. Inclusion of well-documented environmental conditions. Inclusion of demographics in the CBA. Adequate consideration of economic growth. Selection of dose-response functions based on current weight of evidence. Selection of valuation functions based on current weight of evidence. Explicit expressions of uncertainty (measurement, model, and statistical). Compatibility of scenarios with the form of standard (8-h and 24-h averaging times). Inclusion of distributional analysis of costs and benefits (identification of sensitive subgroups, affected sectors). Internal consistency of analyses (linking costs with benefits consistently). Discussion of nonquantifiable endpoints. Explanation for the choices of benefits and costs included. Performing CBA that meets these requirements will involve substantial investment on a continuing basis. Critics of the adequacy of the CWS CBA should support the generation of a knowledge base adequate to perform credible CBA. Generation of that knowledge base will require substantial investment of money, infrastructure and expertise. The panel has outlined its view on the limitations of the CBA undertaken for the development of CWS for PM and ozone. Given those concerns, the panel provides the following two conclusions on the measures of benefits and costs as calculated within the CWS CBA. As in all CBA the estimates of benefits and costs are uncertain. Emerging analyses (in particular, of the tax interaction effect and the value of a statistical life) suggest that the costs associated with reducing emissions may be underestimated and the human health benefits overestimated. However, there are additional uncertainties that temper the impact of these emerging studies on the CBA. Most notably, the cost analysis performed in the CWS process is based on engineering estimates (resulting in overestimated costs) and the benefit measures do not include ecosystem effects (potentially large, but highly uncertain). The direction of the bias in net benefits depends on the weight placed on these factors. The overall direction of the errors in benefits estimation is undetermined. The premature deaths reduced and broader range of health effects avoided by reducing PM ambient levels to the CWS PM standard are likely underestimated in the CWS analysis. However, the dollar value estimates for mortality reductions (based on value of statistical life, VSL) are very likely overestimated. The overall effect of these potential biases on the benefits realized from emission reductions is not clear from the current evidence. Based on these conclusions, the panel makes the following recommendations. Capacity building. Given the identified deficiencies in the CWS CBA, the panel recommends the following: Canada should build a capability for conducting CBA for CWS by improving emissions inventories, air quality modeling capability, air quality monitoring networks, socioeconomic modeling, human health data gathering, and developing economic analyses of health-environment interactions. This capacity building will require long-term financial support to build the infrastructure, as well as government and industry commitment to making these improvements. Data and models should undergo continuing development and refinement with reporting and documentation at periodic intervals that are integrated within the timeframe for decision-making. This includes particularly the air quality valuation model (AQVM) used to estimate the health benefits of air quality improvements. As detailed later in this report, the AQVM may already be out of date in its choice of dose-response functions for estimating mortality risk reductions and for valuation of this health endpoint. The decision to update AQVM to include more recent work depends on the criteria of study inclusion, such as degree of peer review, number of confirmatory studies, etc. The panel recommends that such criteria used to develop the AQVM be reviewed and updated, as necessary. Improve Canadian capacity for air quality modeling. Collaborations with other North American agencies and research groups should be encouraged and supported with long-term funding. Inclusion of risk–risk trade-offs (estimation of damages associated with risks from pollutants that increase as a result of the pollutants of interest being reduced, e.g., UV-B radiation increasing as a result of ozone concentrations being decreased). All CBA model specifications and input values (e.g., risk coefficients, health event valuations) used for the purposes of regulatory decision making should be fully transparent and readily accessible to all interested stakeholders and researchers. An explicit procedural and consultative framework should be developed for CBA to inform the decision process. Informed decisions require dialogue and consultation between decision makers, stakeholders, and CBA analysts in an open, transparent process. The decision process should require consideration of results of CBA along with other inputs to the decision. Funding should be allocated, and roles and responsibilities within the CWS process should be defined—including an external expert advisory body to review approaches, progress, etc. Formal guidelines for considering evidence and making and communicating decisions should be developed. Communication. Improved two-way communication concerning the assumptions, limitations, and uncertainties associated with the methods and results of CBA is needed between analysts and policymakers and between policymakers and the public. Clear communication of the conceptual underpinnings and limitations of valuation techniques and the interpretation of the results of cost and benefit studies is needed to correct prevailing misconceptions about the conduct and interpretation of these studies. Cost-benefit analysis and other types of socioeconomic analyses. Socioeconomic analysis (SEA) includes a wide variety of social and economic analysis methods, of which cost-benefit analysis (CBA) is one example and is typically the foundation for other socioeconomic analyses. However, the panel recommends that CBA be conducted separately from broader socioeconomic analyses, including plant closures, unemployment, regional economic impacts, competitiveness, or inflation for broad-based rules. Such analyses often ignore labor and capital mobility and are not commensurate with values used in CBA and therefore, when provided alongside CBA estimates, the results of broader SEA models may give rise to double counting of benefits or costs. If effects are expected to be borne disproportionately by only a few sectors, these types of analyses are useful, but they should be presented as contributions to the assessment of the distribution of impacts. A more promising, but more resource-intensive, approach is the expansion of the CBA to a general equilibrium analysis to capture the costs of the tax interaction effect. (see Section 7.4) As for competitiveness analysis, shifts to imports may have positive environmental effects that would need to be taken into account, and in this sense, a broader SEA or general equilibrium framework would be useful. Cost-benefit framework for analysis of environmental quality regulations. The panel endorses the use of a cost-benefit framework for the analysis of environmental regulation while recognizing the empirical limitations of CBA. The panel recommends: Continued development of methods for accurate assessment of costs and benefits, including methods for the analysis of general equilibrium (including tax interaction) effects and international trade impacts of regulatory change. Continued development of and communication regarding alternative decision-making frameworks, including multi-attribute methods, to be used as methods to "triangulate" with traditional CBA. Investments in human capital in the area of CBA of environmental regulation so that policy makers and the Canadian public can be confident that cost and benefit measures accurately reflect Canadian values and public preferences as well as Canadian institutional arrangements. Notes This work has been previously published by the Royal Society of Canada, Ottawa (Ontario), in June 2001, appearing as "Review of Socio-Economic Models and Related Components Supporting the Development of Canadian Standards for Particulate Matter and Ozone," ISBN 0-920064-73-6. ∗See U.S. EPA, September 2000. Guidelines for Preparing Economic Analyses. 240-R-00-003. ∗The theoretically appropriate measure of opportunity cost is the change in producer surplus. Producer surplus is defined as the area above the firm's supply or marginal cost curve but below price. As such it is a measure of the gains the producer has for selling at prices higher than marginal cost. Regulatory policies typically shift the marginal cost curve by increasing costs of production. However, since the firm can respond to regulatory change in many ways, not only through direct implementation of emissions reduction technology, the cost of implementing emissions reduction technology is thought of as an upper bound on the cost estimate. †While the benefit measures are referred to as "willingness to pay" or "willingness to accept," most benefit measurement techniques do not actually directly ask individuals to reveal this amount. Individuals may reveal their willingness to pay or willingness to accept through market transactions or other forms of behaviour. There is considerable confusion in the popular literature about the concept of "willingness to pay" (a theoretically appropriate welfare measure) and the method of benefit estimation, contingent valuation, that actually asks individuals what they would be willing to pay. ∗A special but important issue arises here when regulatory impacts have an intergenerational time scale (e.g., costs borne today but benefits received by the next generation). Even a consumption-based discount rate could reduce future impacts to trivial levels over a long time frame. In terms of discount rate policy, it could be argued that intergenerational effects deserve "special" treatment reflecting societal trade-offs across generational income distributions. One simple example is the argument that a discount rate reflecting the long-term rate of economic growth can reflect an "equal" treatment of the generations, reflecting the presumed greater economic ability of future generations to shoulder burdens relative to ourselves. ∗A CWS for PM2.5 of 30 μg/m3, 24 h averaging time, by year 2010. A CWS for ozone of 65 ppb, 8 h averaging time, by year 2010. ∗Rowe, R. D., et al. 1995. The New York Electricity Externality Study. Dobbs Ferry, New York: Oceana. †Cropper and Freemen selected six VSL studies of 21 as "best" for use in policy analysis. Four are wage risk studies and two are contingent valuation studies. The wage-risk estimates range from $3 million to $9 million (1996 CDN dollars), and the contingent valuation estimates range from $4 million to $5 million (1996 CDN dollars). The arithmetic mean of all six selected VSL estimates is about $5 million (1996 CDN dollars). ∗Tables 1A–4H of Section 2 of Compendium of Benefits Information. †Tables 5A–9H of Section 2 of Compendium of Benefits Information. ‡Tables 10A–12H of Section 2 of Compendium of Benefits Information. ∗The assumptions are based on the analyses performed for the Atmospheric for the in and the Environment Canada review of proposed U.S. Environmental for U.S. and work for the and the Canada-Wide for 2000. use the particulate matter or to to all forms of matter the This in a variety of and and it is often by an or since the need not be PM2.5 to that of that has an that and is often as the of the while are referred to the to that of the distribution that has than and includes the PM2.5 of the use the particulate matter or PM when do not need to between the various of the of emissions have been for regional studies in but are not available for Canada wide studies. ∗The is in an attempt to the effects of that there is a that these types of air quality model can the of ozone more than et ∗The also the and over the the is the next to the in which the air is the to a of about At over the of the to about to the of input of to the the most emissions of pollutants into the are they are from or have to the ∗The benefits by and are overestimated since they that the of would to a of in ozone This is an since it is often that ozone is actually the or generation of regional in the and in the developing along with ozone input from the or regional air quality have a context, and this should not be ∗The AQVM notes that levels of in are about of levels in the area according to and from their study of the costs of the and that monetary measures of change in to compliance costs over a partial equilibrium analysis would on a of economic and and would this would be by the general equilibrium analysis makes assumptions about affected and all in the as they could be A trade environment will to compliance and in Canada and the will use control ∗The in the of risk reductions could because the of pollution control today will not benefit they to the effects of pollution air pollution case or because the today to a that risk of only a (e.g., can be made to for the effect of to the model, a to reduce of at should what or would pay to reduce current of at to ignore here the body of literature using an value This approach provides a for air pollution reductions but is on market about pollution and to all types of effects, health being only has the would of not using that these studies were published the recent literature the traditional estimates of the from et that are other of where the preferences of some have been by the decisions of as a ∗The welfare states the economic view most in what is and what is for a the can only be and The of that does not depend on the reasons one has for What for is that individuals the and the are to be to each of to those that benefit the a for in the attainment of some for example, would be the report of the Committee on the argument is that preferences are not to and economic value is not to one is the to this is that there are some that should not have a placed on of the economic value of have used this the monetary value of those up to to a bound on the value of and may be one and the on and for in and for and

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.