The results of economic analyses, like the results of risk assessments, are often expressed as single numbers unaccompanied by any information on the precision or uncertainty that might be associated with them. The inconsistency among agencies and programs in estimating, for example, the cost per life saved in association with a regulatory decision reflects, in part, the uncertainty associated with valuing such a quantity.

FINDING 4.2.1: Like health risk assessment, economic analysis involves multiple assumptions and produces uncertain results. Estimates of the costs and benefits associated with alternative regulatory and nonregulatory options rely on data to the extent that data are available, relevant, and reasonably precise, but they also rely on judgments, values, assumptions, and extrapolations.

RECOMMENDATION: The primary sources of uncertainty associated with the results of economic analyses should be identified, characterized, stated explicitly, communicated clearly, and quantified where appropriate. The results of economic analyses should not be expressed as though they are precise measures of actual economic costs and benefits.

RATIONALE

As inputs to economic analysis, the results of health risk assessments contribute a large degree of uncertainty. The uncertainty associated with an upper-bound point estimate of individual risk can range over several orders of magnitude. Economic analysis relies not on point estimates of individual risk, but on the entire probability distribution of potential costs or benefits for an entire affected population, which cannot be accurately extrapolated from an upper-bound point estimate of individual risk. Economic analysis relies on information about the central tendencies (mean or median) of costs and benefits for a population as a whole as well as measures of dispersion, so that aggregate expected net benefits can be evaluated. Determining central tendencies and measures of dispersion requires information on the probability distributions underlying the important components of costs and benefits. If a scientific assessment of risk provides information only on the upper bounds of hazards the economic analysis will either overstate the net benefits to the general population or be relevant only to the tail of the risk distribution. However, relying only on central tendencies might misrepresent net costs or benefits to particular subpopulations. Avoiding these inconsistencies requires changes in approaches to both health risk assessment and economic analysis, as discussed later in section 4.3.

Other sources of uncertainty in economic analyses used in an environmental context are associated with valuing the benefits of environmental assets. Environmental assets are features of the natural environment whose degradation people would be willing to pay to avoid. They include recreation areas, endangered species, visual range, open space, and wetlands. People might value preventing degradation of those assets because they use the services that the assets provide ("use value") and because "they are there" ("non use value"); quantitative estimates of value in both cases can be highly variable and often controversial.

Cost estimates are also highly variable and imprecise, and they can vary according to the bias of the organizations affected. Regulatory agencies often must base their cost estimates on incomplete and possibly biased information, which might tend to overestimate or underestimate costs. The Office of Technology Assessment (1995) evaluated a variety of examples that illustrated how agency estimates of the costs of new regulations before enactment differed from the actual costs incurred. For example, industry comments suggested that implementing the workplace standard for vinyl chloride would cost industries $1 billion; actual costs were about $250 million. OSHA predicted that implementing the workplace standard for cotton dust would cost industries about $280 million a year; actual annual costs were about $80 million. Neither of those estimates anticipated process and technology changes that substantially decreased costs, increased efficiency, and reduced exposures.

In general, costs are initially overestimated, not underestimated, according to MIT Professor Nicholas Ashford's testimony to the Commission, for several reasons: costs are often provided by the regulated industries, the ability of regulated industries to learn more cost-effective means of compliance is neglected, economies of scale are ignored, and preregulatory cost estimates neglect the impressive effect that regulations can have on stimulating new technologies. Of course, estimating the economic impact of a new regulation before it occurs is inherently very difficult, relying of necessity on assumptions, judgments, and speculation.

Examples of documented cost underestimation are more difficult to identify, because of a dearth of retrospective analysis. Nevertheless, a number of analysts believe that it occurs with some frequency. For example, recent Clean Air Act rule-makings associated with operating permits did not adequately allow for affected emitters' opportunity cost that resulted from delays in receiving new permits. The Resource Conservation and Recovery Act's rule-making on assessing the toxicity of waste materials included large volumes of lower-risk materials inadvertently; as compared to EPA's estimate, the regulation of those materials under the rule substantially increased the actual costs of the rule.

The assumptions upon which economic analysis is based are associated with many sources of uncertainty, so it is misleading to express the results of economic analyses as single quantitative estimates of costs or benefits. Results of analyses should often include more than single estimates of costs and benefits, expressed in a manner that reflects their inherent uncertainty. In some cases, probabilistic techniques can provide a sense of the distribution of possible outcomes. In other cases, it might be possible to assess only a few alternative scenarios with some qualitative information about their relative plausibility. In all cases, however, it is essential to identify the primary sources of uncertainty.

FINDING 4.2.2: Monetized valuation of benefits for regulatory purposes is inconsistent across regulatory agencies and programs.

RECOMMENDATION: To achieve more nearly consistent benefit valuation among regulatory agencies, the value of mortality risks should be stated explicitly and valued with best estimates or ranges of estimates and with consistent use of procedures and basic assumptions. The development of federal guidelines for benefit valuation involving stakeholder input should be considered.

RATIONALE

Although a succession of administrations have issued executive orders that require consideration of costs and benefits in rulemaking, those administrations have explicitly refused to establish a consistent basis for valuing a death risk reduction (or "statistical life" saved) or to establish a basis for evaluating estimates of the cost statistical life saved associated with a policy option. As a result, under current guidance, agencies may choose not to value death risks (or "lives") explicitly or choose not to subject their regulations to comparison with a benchmark for cost effectiveness.

That kind of valuation inconsistency takes several forms, including whether an analysis even includes explicit values for death risk reductions, how such values are incorporated, and what values are chosen. For agencies that explicitly value death risk reductions, the implied value of a statistical life ranges from $1 million to $10 million. For agencies that do not explicitly value death risk reductions, but instead base decisions on an "acceptable" cost per life-saved, the implicit value of a statistical life can be far higher. One study of EPA regulatory decisions that affected cancer risks found regulations promulgated that cost over $50 million per life saved. The Office of Management and Budget study of such behavior, involving a broader range of causes of death, found even higher costs per life saved, as did a recent Congressional Budget Office study of drinking-water standards. Another way of valuing lives or social costs is by the ratio of false-negatives (failing to identify a chemical as a carcinogen) to false-positives (inappropriately identifying a chemical as a carcinogen, thereby leading to regulation and loss of its beneficial uses), as illustrated by the Lave-Omenn value-of-information model for carcinogenic test strategies (Lave et al. 1988, Omenn and Lave 1986, Omenn et al. 1995).

Encouraging agencies and programs to value death risks with consistent procedures that lead to the best estimates or ranges of estimates of such values under specified conditions could reduce interagency and intra-agency inconsistency. "Best estimates"⁽³⁾ can be devised within an interagency process that takes into account consensus and the range of uncertainty around published values, including the comparability of various types of risks. Government and private resources are less likely to be wasted when agency rule-making consistently reduces death risks at costs that reflect the value of the risk reduction. Explicit valuation of reductions in death risks also makes it easier to compare regulatory alternatives when expected benefits are nonquantifiable.

3 The term "best estimate" is ill-defined and controversial when used to describe the results of risk assessments (see abstract of paper prepared for the Commission by Cambridge Environmental, Inc., in appendix A.5). However, to economists, best estimate is a well-defined and accepted concept, referring to central tendency or expected value.