Chapter 5, Risk Commission Final Report, Vol. 2

The regulatory reform debate in the 104th Congress highlighted the role of benefit-cost analysis and cost-effectiveness analysis in regulatory decision-making. Each of the last five presidents has issued an executive order requiring estimation and consideration of the benefits and costs of major regulatory actions, but the questions of whether and to what extent various regulatory decisions should be determined by economic considerations remains controversial.

Risk assessment results can be used as the basis for estimating costs and benefits the results of both risk assessment and economic analysis can contribute to or determine a regulatory decision. Risk assessment and economic analysis can involve large investments of resources and multiple assumptions however, and they produce uncertain results. Their results contribute only part of the information that must be considered in making decisions about the best ways to protect human health and the environment.

In view of the important and complementary roles of risk assessment and economic analysis, the Commission decided to consider the strengths and limitations of economic analysis, although we were not explicitly mandated to do so. We relied on an invited issue paper by Alan Krupnick, Michael Toman, and Ray Kopp of Resources for the Future (see Appendix A7 for abstract) and on invited testimony and comments received from Lester Lave of Carnegie Mellon University, Richard Morgenstern of Resources for the Future (on leave from EPA), Nicholas Ashford of MIT, Douglas MacLean of the University of Maryland, and John Graham of the Harvard School of Public Health.

This section briefly addresses the role of benefit-cost analysis (BCA) and cost-effectiveness analysis (CEA) (together referred to as "economic analysis") in regulatory decision-making. Some health and environmental statutes require the consideration of costs and benefits in risk-related decision-making; others explicitly exclude their consideration, while still others are silent. Like risk assessment results, the results of economic analyses have often been communicated solely in numeric terms accompanied by little information on assumptions, nonquantified benefits and costs, and the analyst’s confidence in the results. The 1996 Economic Report of the President recognizes the important role of cost and benefit considerations in risk management decision-making, while highlighting the need to take uncertainty into account and to include factors that cannot be monetized or quantified.

The role of economic analysis in regulatory decision-making is controversial. There is concern that economic analysis places too much emphasis on assigning dollar values to aspects of health and the environment, that are difficult—if not impossible—to quantify. There is also concern that regulatory decisions about health and environmental protection might be made strictly on the basis of whether their quantifiable benefits outweigh their monetized, quantifiable costs.

The tools of economic analysis should be recognized as legitimate and useful ways to obtain information for the Risk Management Framework and for regulatory decisions that will affect health, safety, and the environment, but not as the sole or overriding determinant of those regulatory decisions. Information about costs and benefits that are intangible and that cannot be assigned monetary values should be addressed and considered explicitly. Assumptions and uncertainties should be specified.

Economic analysis plays an important role in the options stage of our Framework for Risk Management (see Section 2). Like risk assessment, the tools of economic analysis have strengths and limitations. And like social and political considerations and information on risks to health and the environment, economic analysis can provide important input to risk management and regulatory policy decisions. Considering incremental costs and benefits in regulatory decision-making can help to clarify the tradeoffs and implications associated with alternative regulatory policies and help regulatory agencies to set priorities. Economic analysis can contribute to making better use of society’s limited resources.

The objectives of CEA and BCA differ. CEA can help identify the risk management option that achieves a specified regulatory goal with the smallest cost or least reduction in overall social well-being. That is, CEA begins with an assumed health or environmental protection goal and then explores and compares the methods that could achieve that goal to identify the least costly one (while acknowledging that costs and benefits might be inequitably distributed; see page 96). For example, if the health-based goal is to lower the current ambient ozone standard to 0.1 ppm, CEA could be used to help to choose among options that are expected to attain the 0.1 ppm standard but use different approaches, generate different costs, and may have different probabilities of success. Tengs et al. (1995) used CEA to compare different life-saving medical interventions against a common measure, years of life saved.

CEA also can be used to assess different means of achieving intermediate regulatory goals. Suppose, for example, that several alternatives can be pursued to reduce automobile exhaust emissions as part of a larger ozone control strategy. CEA can be used to rank the cost per unit of emissions reduction of those alternatives. Policy makers could then compare the vehicle policies with other options to determine the least cost way to achieve the larger goal of ozone reduction. Disadvantages of CEA are that the most cost-effective option might not be the one that provides the most efficient allocation of resources and that only costs, not benefits, are considered.

BCA has a different role: it can be used to help formulate risk management policies and priorities and identify risk management goals that maximize net benefits across various levels of protection. For example, BCA can assess the benefits and costs of alternative health-based standards with different levels of health protection. Consider the following hypothetical example:

Possible Standard	No. Annual Health Effects Averted	Incremental Benefit	Annual Cost of Controls ($ million)	Incremental Cost	Incremental Cost ($ million/effect averted)

status quo (100 ppm)	—		—		—
50 ppm	500	500	50	50	0.1
20 ppm	950	450	150	100	0.2
5 ppm	990	40	500	350	9
1 ppm	999	9	2,000	1,500	170

In this is example, BCA could assist EPA in selecting the standard that it should adopt by translating health effects into dollar-equivalent units with such methods as "willingness-to-pay." The willingness-to-pay concept reflects the economic principle that environmental quality and risk reductions ultimately are things people value, just as they value conventional consumer goods. Althouh it is subjective and can be unreliable, economists use this method to estimate how much people will give up to gain environmental improvements. It is only one approach that can be used to value costs and benefits, however. In this hypothetical example, if economic analysis indicated that the public is willing to pay up to $5 million per averted health effect, the economically "efficient" standard would be between 5 and 20 ppm. BCA applied in a strict quantitative sense can be used only to the extent that costs and benefits can be monetized. This approach might be rejected if willingness-to-pay is unknown, benefits are nonquantifiable, or BCA is considered inapplicable. CEA, in contrast, could compare the costs of implementing different methods of control with the number of deaths or health effects that would be prevented by those controls. The policy maker would have to decide which cost is acceptable and select a standard that is consistent with that cost and in keeping with other desired goals of the decision-making process.

The advantage of BCA, in principle, is that it can be used to help make choices among policies and actions with quite different benefits and costs, guided by what members of society are thought to be willing to pay to reduce risks. It is no small challenge to compare, for example, costs and benefits of reducing lead derived from paint contamination in houses with those of ambient ozone reduction. In some cases, benefits and costs might be nonquantifiable because of the absence of reliable data, not because they are intrinsically nonquantifiable. In such cases, it is better to rely on qualitative analysis than to produce an indefensible quantitative analysis. When there are believed to be substantial benefits (or costs) that cannot be monetized, a BCA should be supplemented by discussion of the nonquantifiable elements, as emphasized in the 1996 Economic Report of the President. Effective methods of including nonquantifiable benefits in economic analysis are needed and should be pursued. At a minimum, good practice would include listing what the analyst believes are potentially important nonquantifiable benefits (and costs).

An example of a method for evaluating both quantifiable and nonquantifiable benefits is a study of environmental damages caused by the generation of electricity (Rowe et al. 1995). Benefits were divided into four categories: benefits quantifiable; damages probably de minimis, so quantification not justified; quantification possible but more resources required for analysis; and quantification not possible. The first category included the health benefits of reducing air pollution because the epidemiologic, cancer risk, and valuation literature regarding air pollution is relatively rich. The benefits of reducing acid deposition on crops, vegetation, and forests were placed in the second category. The third category included impacts of surface water chemical discharges on fisheries; monetization of the effects was thought to be possible for some chemicals, but many assumptions would be needed and the effects were unlikely to be large. The effect of greenhouse gases on climate was a prominent example in the fourth category; instead of monetization, a sensitivity analysis was provided, which indicated that every dollar of damage per ton of CO2 emitted was equivalent to 0.1 cent per kilowatt-hour when electricity is generated by coal. Other category four examples are the effects of air pollution on wildlife and the effects of acid deposition on cultural and historic materials.

A BCA of a proposed policy should also be supplemented with information on its distributional consequences. In an assessment of aggregate benefits and costs there is no accounting for who bears the risks and could benefit from risk reduction and who bears the costs of implementing the policy. BCAs based on aggregate benefits and costs do not explicitly weigh consequences by income category or ethnic group (see next section). Equity considerations can be considered in BCA, but doing so requires agreement on how to weight different social groups. No objectively correct weights can be substantiated.

CEA, in contrast, does not require that benefits be monetized, although they can be monetized when appropriate. (Nonmonetized benefits cannot be aggregated.) CEA requires only that the "effectiveness" of a policy be defined by some physical measure (such as tons by which pollutants are reduced, or number of cancer deaths avoided). The cost of different policies per unit of effect can be compared. CEA cannot inform the debate over the goals of a policy, but it can provide information about the cost per death or effect averted; it is up to the policy maker to decide how to use that information to make a decision. CEA, however, should be used cautiously in the analysis of a program with more than one favorable effect—for example, it saves lives, reduces illness, and provides ecological or aesthetic benefits—as it is difficult to compare these on the basis of cost per single beneficial effect. Only if the other favorable effects can be monetized and subtracted from the costs, can a net cost-per-life-saved calculation be made; similarly, an estimate of the net costs of ecological or aesthetic benefits can be made by deducting estimates of reduced morbidity and mortality risk.

A recent review of the conduct and use of economic analysis in support of EPA regulations indicates that economic analysis has so far played only a minor role in actual decision-making (Morgenstern 1997), primarily because:

Despite its limitations, BCA can provide useful information to help evaluate the favorable and unfavorable effects of proposed regulatory policies and should continue to be used as appropriate to inform but not as the sole criterion for decision-making. Benefit-Cost Analysis in Environmental, Health, and Safety Regulation stated that, "benefit-cost analysis is neither necessary nor sufficient for designing sensible public policy. If properly done, it can be very helpful to agencies in the decision-making process" (Arrow et al. 1996). Because estimates of costs and benefits are highly uncertain, BCA cannot be used to "prove" that the benefits of a policy outweigh its costs, or vice versa, Nonetheless, information about the incremental costs and benefits associated with options for a regulatory decision can serve the public interest and, in fact, is mandated in the Unfunded Mandates Reform Act of 1995 and in Executive Order 12866. Moreover, BCA can be an important element of a more inclusive set of decisional criteria for assessing the potential value of regulation. In particular, to ascertain that the benefits of regulations justify their costs, as stipulated in Executive Order 12866, it is important not only to identify and measure the incremental costs and benefits that can be quantified but also identify those which are less quantifiable.

Economic analyses have been criticized because they are often blind to issues of environmental equity and fail to make explicit who bears the costs of a regulatory decision and who reaps the benefits.

Economic analyses should present information, where practicable, that can be used to provide a firmer basis for evaluating any inequitable distributions of costs and benefits.

BCA generally does not address the equity implications of the policies that they seek to evaluate. For example, if implementing a policy that affects health, safety, or the environment decreases the welfare of the poor and increases the welfare of the wealthy, but the benefit to the wealthy outweighs the loss to the poor (in dollars, not percent income), BCA might show the policy to lead to an improvement in aggregate social welfare.

BCAs need not incorporate equity considerations quantitatively. Deciding how different groups should be weighted for equity in economic analysis is highly value-laden. However, if groups or individuals within a societal group potentially affected by a policy are likely to experience the impact differently then that should be identified and communicated to risk managers, regulatory decision-makers, and stakeholders, and considered as policies are formulated. For example, the implementation of a policy that reduces permissible pesticides will result in some segments of the population enjoying reduced health risks when consuming the affected fruits and vegetables; it will also result in some who will no longer to be able to afford those fruits and vegetables. Evaluating such differences quantitatively would be problematic, but revealing them qualitatively would provide important information that could be considered in the regulatory decision.

The results of economic analyses, like the results of risk assessments, are often expressed as single numbers unaccompanied by any information on the precision or uncertainty that might be associated with them. The inconsistency among agencies and programs in estimating, for example, the cost per life saved in association with a regulatory decision in part reflects the uncertainty associated with valuing such a quantity.

Like health risk assessment, economic analysis involves multiple assumptions and produces uncertain results. Estimates of the costs and benefits associated with alternative regulatory and nonregulatory options rely on data to the extent that they are available, relevant, and reasonably precise, but also rely on judgments, values, assumptions, and extrapolations.

The primary sources of uncertainty associated with the results of economic analyses should be identified, characterized, stated explicitly, and communicated clearly. The results of economic analyses should not be expressed as though they are precise measures of actual economic costs and benefits.

Many sources of uncertainty associated with the results of economic analyses. For example, the results of health risk assessments contribute substantial uncertainty and the uncertainty associated with an upper-bound point estimate of individual risk can range over several orders of magnitude. Economic analysis relies not on point estimates of individual risk, but on the entire probability distribution of potential costs or benefits for an entire affected population, which cannot be accurately extrapolated from an upper-bound point estimate of individual risk. Economic analysis relies on information about the central tendencies (mean or median) of costs and benefits for a population as a whole as well as measures of dispersion, so that aggregate expected net benefits can be evaluated. Determining central tendencies and measures of dispersion requires information on the probability distributions underlying the important components of costs and benefits. If a scientific assessment of risk provides information only on the upper bounds of hazards the economic analysis will either overstate the net benefits to the general population or be relevant only to the tail of the risk distribution. However, relying only on central tendencies might misrepresent net costs or benefits to particular subpopulations. Avoiding these inconsistencies requires changes in approaches to both health risk assessment and economic analysis, as discussed later on page 99.

Other sources of uncertainty in economic analyses used in an environmental context are associated with valuing the benefits of environmental assets. Environmental assets are features of the natural environment that people are willing to support financially to avoid their degradation. They include recreation areas, endangered species, visual range, open space, and wetlands. People might value preventing degradation of those assets because they use the services that the assets provide ("use value") or simply because of their existance ("non use value"). Quantitative estimates of value in both cases can be highly variable and often controversial, which may partly explain why natural resource damage provisions in existing laws have been little used.

Cost estimates are also highly variable and imprecise, and they can vary according to the bias of the organizations affected. Regulatory agencies often must base their cost estimates on incomplete information from parties with economic interests at stake. The Office of Technology Assessment (1995) evaluated how agency estimates of the costs of new regulations before enactment differed from the actual costs incurred. For example, industry comments suggested that implementing the workplace standard for vinyl chloride would cost industries $1 billion; actual costs were about $250 million. OSHA predicted that implementing the workplace standard for cotton dust would cost industries about $280 million a year; actual annual costs were about $80 million. Neither of those estimates anticipated process and technology changes that substantially decreased costs, increased efficiency, and reduced exposures.

In general, according to MIT Professor Nicholas Ashford’s testimony to the Commission, costs are initially overestimated for several reasons: costs are often provided by the regulated industries, the ability of regulated industries to learn more cost-effective means of compliance is neglected, economies of scale are ignored, and preregulatory cost estimates neglect the impressive effect that regulations can have on stimulating new technologies. Of course, estimating the economic impact of a new regulation before it occurs is inherently very difficult, relying of necessity on assumptions, judgments, and speculation.

Examples of documented cost underestimation are more difficult to identify, because of a dearth of retrospective analysis. Nevertheless, a number of analysts believe that it occurs with some frequency. For example, recent Clean Air Act rulemakings associated with operating permits did not adequately allow for affected emitters’ opportunity cost that resulted from delays in receiving new permits. The Resource Conservation and Recovery Act’s rule making on assessing the toxicity of waste materials inadvertantly included large volumes of lower-risk materials, increasing the actual costs of the rule compared with EPA’s estimate.

Given the assumptions and uncertainties, it is misleading to express the results of economic analyses as single numerical estimates of costs or benefits. In some cases, probabilistic techniques could provide some sense of the distribution of possible outcomes. More generally, qualitative information included as narratives that assess a few alternative scenarios and their relative plausibility would be helpful. In all cases, however, it is essential to identify the primary sources of uncertainty.

Monetized valuation of benefits for regulatory purposes is inconsistent across regulatory agencies and programs.

To achieve more nearly consistent benefit valuation among regulatory agencies, the value of mortality risks should be stated explicitly and valued with best estimates or ranges of estimates and with consistent use of procedures and basic assumptions. Development of federal guidelines for benefit valuation involving stakeholder input should be considered.

Although several successive administrations have issued executive orders that require consideration of costs and benefits in rulemaking, those administrations have explicitly refused to establish a consistent basis for valuing reduction in death risk (or "statistical life" saved) associated with various policy options. As a result, under current guidance agencies may choose not to value death risks (or "lives") explicitly and avoid subjecting their regulations to comparison with a benchmark for cost effectiveness.

Inconsistency in valuation takes several forms, including whether an analysis includes explicit values for death risk reductions, how such values are incorporated, and what values are chosen. For agencies that do explicitly value death risk reductions, the implied value of a statistical life ranges from $1 million to $10 million. For agencies that do not explicitly value death risk reductions, but instead base decisions on an "acceptable" cost per life saved, the implicit value of a statistical life can be far higher. One study of EPA regulatory decisions that affected cancer risks found regulations promulgated that cost more than $50 million per life saved. An Office of Management and Budget study of such behavior, involving a broader range of causes of death, found even higher costs per life saved, as did a recent Congressional Budget Office study of drinking water standards. In such cases, the decisions were probably driven by statutory or technological requirements. Another way of valuing lives or social costs is by the ratio of false negatives (failing to identify a chemical as a carcinogen) to false positives (inappropriately identifying a chemical as a carcinogen, thereby leading to regulation and loss of its beneficial uses), as illustrated by the Lave-Omenn value-of-information model for carcinogenicity test strategies (Lave et al. 1988, Omenn and Lave 1988, Omenn et al. 1995; see "Value of Obtaining Additional Information" on page 91).

Encouraging agencies and programs to value death risks with consistent procedures that lead to the best estimates or ranges of estimates of such values under specified conditions could reduce interagency and intra-agency inconsistency and possibly facilitate more cost-effective decisions across the agencies. "Best estimates"* could be devised within an interagency process that takes into account consensus and the range of uncertainty around published values, including the extent of comparability of various types of risks. Too-rigid protocols that reduce economists’ flexibility to choose the data and analytical approach that best fit the problem should be avoided, however.

Risk assessors are unfamiliar with the information about risks that is needed for economic analysis. As a result, the questions asked and the results of risk assessments often do not match the needs of economic analysis.

Risk assessors and economists who must rely on the results of risk assessments to estimate benefits should collaborate more to reduce the inconsistencies between scientific and economic approaches to characterizing risks and risk reduction alternatives. Risk assessors and economists should expand their methods to reduce mismatches.

Implementing the Commission’s Risk Management Framework and using information on both risks and economics to make decisions require some consistency between risk and economics-related assumptions and conclusions. At present, risk assessors operate in a world essentially isolated from that of economists, and economists often have little knowledge of risk assessment. Furthermore, risk assessors and economists are generally attempting to answer different questions. Incompatible and contradictory practices will have to be reconciled if risk assessment and economic analysis are to be used together to support effective risk management decision-making.

For example, the results of risk assessments are used in economic analysis to estimate benefits, but risk characterization end points are often inconsistent with economic valuation starting points. The traditional methods of evaluating health effects for use in health risk assessment can conflict with the needs of economists who are asked, at least implicitly, to provide information on individual preferences for avoiding health risks. For example, a 10 percent improvement in lung function is not meaningful to most people. They do not demand greater lung function; they want fewer sick days. Health risk assessments seldom evaluate risks in terms of sick days, and no available economic studies can be used to value a 10 percent improvement in lung function. In addition, adverse effects other than cancer are generally regulated by comparing a chemical’s exposure concentration to its reference standard, or "safe," concentration. Without a linear dose-response relationship, there is no basis for estimating a probability of risk (such as one extra cancer death out of a million people exposed over a lifetime). Economists’ methods for evaluating risks require that risks be expressed as probabilities. Closer collaboration between economists who are familiar with the valuation literature and scientists who are estimating concentration-response functions might help to overcome such mismatches in estimating risk and economic value.

Another conflict between the needs of economists and the results of risk assessments arises because health risk assessments generally focus on individual risk estimates rather than population risk estimates. Economists estimate benefits for the population at large, for two reasons. First, if costs are to be compared with benefits, it would make no sense to compare total costs with benefits experienced by only one person, especially the hypothetical "maximally reasonably exposed" individual. Second, even if one were performing a CEA in which abatement costs per risk to the maximally exposed person were being estimated, the resulting estimates could be very misleading for the decision-maker. Suppose that two abatement strategies had equal cost, but one was related to a very high individual risk and low population risk (because few people were exposed to the pollutant of concern), and the other associated with exposing many more people but with low individual risk. A CEA based on individual risk would lead to adoption of the first strategy instead of the strategy based on the population risk, which could be considered the more relevant measure.

Inconsistency also results from the traditional risk assessment practice of relying on conservative assumptions to account for uncertainty about exposure or toxicity. That tradition purposely skews risk estimates upward to build in a margin of safety that is intended to protect a population from health risks (estimating average risk reductions instead might result in protection of only part of a population), and provides only one point in the upper end of a risk distribution. According to standard practice, a BCA is an attempt to describe the distribution of risks (or the distribution of risk reductions) in the population and defer to a decision-maker to determine what is an adequate level of protection and which strategies deliver that level of protection. Computing cost-effectiveness measures on the basis of an upper-bound estimate of risk will result in a lower-bound estimate of the actual cost. Using distributions of risk estimates instead of upper-bound point estimates might overcome this inconsistency.

Finally, mismatch can result because risk assessment relies more on expert opinion and economic analysis relies more on the expressed preferences of nonexperts for products or activities associated with risks, where those preferences are conditional on individual risk perceptions; economic estimates of damages are based on individuals’ willingness to pay to avoid risks. Nonexperts’ individual risk perceptions often disagree with expert opinion (see Identifying Risk Communication Needs on page 39). Resolving these inconsistencies will require judgments regarding the appropriate weighting of the opinions of experts and of informed, nonexpert people. Interaction and collaboration between nonexpert stakeholders and technical people may lead to convergence of views.

The use of margins of exposure by EPA to compare cancer and noncancer risks (see Need for a Common Metric on page 43) has been criticized as being unsuited to economists’ needs for specified, extrapolatable (not necessarily linear) dose-response curves down to very small exposures. That problem has always existed for noncancer effects, because they are thought to exhibit a threshold (no effect below a particular dose). Without a dose-response relationship, there is no basis to calculate incremental benefit and incremental cost as exposure concentrations are reduced. Putting aside the issue of defining that threshold, economists could combine "willingness-to-pay" methods and biological insights to put values on margins of exposure for various types of adverse effects. Having the margins decline due to increases in emissions and exposures would be a negative effect. Taking action to increase margins of exposure between exposures known to have adverse effects and exposures actually experienced in various occupational and environmental settings would be a benefit. Presumably, relative values or monetized estimates could be generated. It would be important to use the risk reduction presentation captured in Figure 3.1 to guide assessment of the amount of risk reduction gained as exposure levels were reduced progressively.