The primary message of Science and Judgment in Risk Assessment, the 1994 National Research Council (NRC) report to the Environmental Protection Agency (EPA) was that although EPA's health-risk assessment methods were fundamentally sound, it needed to establish more clearly the scientific and policy basis for those risk assessments and describe the uncertainties and variabilities associated with health risk estimates. This appendix reviews the NRC report's primary conclusions in science, policy, and uncertainty and comments on them in the context of the Commission's mandate.

1. Uses and Limitations of Risk Assessment

The NRC report emphasized that risk assessment is a set of tools and that it should be an adjunct to the primary regulatory goal of safeguarding public health, not an end in itself. Health risk assessment is but one element of environmental decision-making--a component of decisions about whether, how, and to what degree the assessed risk requires reduction. The factors that might be considered by decision-makers depend on the requirements of applicable statutes, precedents established within the responsible government agencies, and good public policy. The limited resources available for environmental protection should be spent to generate information that helps risk managers to choose the best possible course of action among the available options.

The Commission agrees that risk assessment is but one of a number of risk-management decision-making tools. The results of a risk assessment are not scientific estimates of actual risk; they are conditional estimates of the risk that could exist under specified sets of assumptions and--with political, engineering, social, and economic information--are useful for guiding decisions about risk reduction. The risk-management decision-making framework that is discussed in section 2 of the Commission's report provides guidance for including those kinds of information in risk-management decisions.

2. Maximal Use of Scientific Information versus Plausible Conservatism

The NRC report stated that EPA operates in a decision-making context that imposes pressures on the conduct of risk assessments and that these contextual pressures have led to recurrent problems of scientific credibility. Criticisms of EPA's risk assessments focus on three basic decision-making structural and functional problems:

The NRC report pointed out that whereas EPA's risk-assessment practices rely heavily on default options, EPA has never articulated the scientific or policy basis of those options. Because of limitations on time, resources, scientific knowledge, and available data, however, the report concluded that EPA should generally retain its conservative, default-based approach to risk assessment for screening analysis in standard-setting. The authors offered several recommendations to make this approach more effective:

The Commission concurs that default assumptions are a necessary part of the conduct of risk assessments. Risk assessments make predictions about the unknowable by using inferences that have not been or cannot be adequately tested with the scientific method. In the absence of adequate scientific information, science- and policy-based assumptions are appropriate. The Commission also supports the goal of transparency and believes that assumptions used in risk assessments and the uncertainty associated with their results should be clearly identified and justified.

An iterative approach to risk assessment also seems reasonable. An iterative approach would start with relatively inexpensive screening techniques and move to more resource-intensive data-gathering, model construction, and model application as the particular situation warranted. To guard against the possibility of underestimating risk, screening techniques must be constructed to err on the side of caution when there is uncertainty. In many situations, for example, gathering site-specific exposure information or investigating the human relevance of a particular toxicologic end point observed in rodents can reduce the extent to which default assumptions are required. Screening risk assessments that use assumptions instead of site-specific information might be used to set priorities by identifying the sites that are likely to pose the greatest risks to health or the environment. More refined risk assessments that use more sophisticated information could then be performed on the riskier sites to obtain better risk estimates. Such an iterative approach is intellectually satisfying.

However, the Commission is concerned about the possible public reaction to iterative determinations of risk. Suppose that a first-tier, screening risk assessment of a contaminated site concludes that an upper-bound incremental lifetime cancer risk greater than 10^-6 is possible. Later refined risk assessments of the same site conclude that the risk is likely to be less than 10 ^-6. The residents of the surrounding community have been told first that the site poses a risk to their health and now that it does not. It is unlikely that such apparently conflicting conclusions will establish any credibility for the regulatory agency or other organization that has announced them. Citizens will remain suspicious and will probably believe that the site constitutes a health hazard, despite messages to the contrary.

Nonetheless, the NRC report concluded that neither the resources nor the necessary scientific data exist to perform a full-scale risk assessment on every potentially hazardous chemical. Nor, in many cases, is such an assessment needed. There might be a vast difference between having "the truth" and having enough information to enable a risk manager to choose the best course of action from the options available. The latter criterion is more applicable in a world with resource and time constraints. Determining whether "enough information" exists to support a decision implies the need to evaluate a full range of decisions. Further improvement of a risk-assessment estimate might or might not be the most desirable course in a given situation, especially if the refinement is not likely to change the decision or if disproportionate resources have been directed to studying the risk at the expense of creating a full set of decision options from which to choose.

Using an iterative approach thus could yield the risk-management decisions required under regulatory mandates in a resource-sensitive manner and at the same time provide incentives for further research without the need for costly case-by-case evaluations. But communicating iterative estimates of risk to the public without loss of credibility will require serious consideration.

3. Inter-agency and Intra-agency Consistency

The NRC report observes that it often seems safest for a regulatory agency to take refuge in established procedures even if they have begun to appear scientifically outdated. External pressures, such as the demands of state agencies for precise guidance, strengthen this tendency. These managerial problems are faced by any regulatory body that is responsible for rendering consistent decisions based on changing scientific knowledge. To remain accountable to the public, regulatory agencies must assess uncertain science in accordance with principles that are fully and openly articulated and applied in a predictable and consistent manner from case to case. Science-policy rules might ensure a valuable degree of consistency from one case to another, but they do so in part by sometimes failing to stay abreast of changing consensus in the scientific community. Bureaucratic considerations of consistency can sometimes override good scientific judgment.

The NRC report concluded that there is a need for a tradeoff between flexibility on the one hand and predictability and consistency on the other regarding departure from default options. Agencies should seek a middle path between inflexibility and ad hoc judgments, but steering this course is difficult. Consistency and predictability are served if an agency sets out criteria for departing from its guidelines. If such criteria are themselves too rigidly applied, the guidelines could ossify into inflexible rules; but without such criteria, the guidelines could be subverted at will with the potential for political manipulation of risk assessment.

Appendix A.6 of the Commission's report surveys risk-related consistency issues both within EPA and among several regulatory agencies. The survey notes that differences in how risks are calculated and how risk-assessment results are used in regulatory decision-making have evolved in different agencies and programs for a variety of reasons. Some of those differences are necessary because of the differing mandates or goals of the various programs, but risk-assessment and risk-management practices are in general poorly coordinated. Better coordination is needed to resolve inappropriate inconsistencies in situations in which two or more agencies regulate similar health or ecologic hazards. Some inconsistencies might be appropriate, however, in light of each agency's or program's own goals and mandates.

4. Bright Lines

In its discussion of bright lines, the NRC report concluded that judicial review has not established any particular method for EPA to use in determining what level of risk should be considered negligible. EPA in turn has decided that it cannot use any single metric as a measure of whether a risk should be considered negligible. Instead, it has adopted a general presumption that a lifetime excess risk of cancer of about one in 10,000 (10^-4) for the most exposed person constitutes negligible risk and that the margin of safety should reduce the risk for the greatest possible number of persons to an individual lifetime excess risk no higher than one in 1 million (10^-6). Such factors as incidence, the distribution of risks, and uncertainties are taken into account in applying those benchmarks.

The 1990 amendments to the Clean Air Act require that standards be set for emission sources if maximum achievable control technology allows a residual risk of greater than 10^-6 to the person most exposed to emissions (the "maximally exposed individual", or MEI). Although that requirement appears to be an example of legislating risk-management decisions on the basis of the MEI, the 10^-6 criterion in fact need be interpreted only as an upper-limit screening device. In addition, those standards need not be expressed in terms of quantitative risk. EPA may use the 10^-6-10^-4 approach described above, but it is not required to do so. Any method that is consistent with the requirement that the standards provide an "ample margin of safety" and reduce risk to a level judged acceptable by EPA may be used.

As discussed in section 5.3 of the Commission's report, the Commission does not support legislating reliance on specific bright lines for environmental regulatory decision-making, except as guideposts or goals for decision-making. If numerical targets are to be included in agency rules, the Commission prefers the use of ranges between bright lines as goals, which would permit flexibility in decision-making that reflects uncertain risk estimates, uncertain cost estimates, and local stakeholder preferences. Decision-makers should be expected to apply bright line ranges flexibly, such as using 10^-6 as a benchmark for screening risk assessments, but not as a yes-or-no criterion for site cleanup decisions. Specific bright lines should not be mandated by Congress--they should be established, when appropriate, by regulatory agencies. Congress should continue to use qualitative language in legislation, such as "reasonable certainty of no harm".

5. Peer Review

The NRC report recommended that peer review, workshops, and other devices be used to ensure broad peer and scientific participation and guarantee, as much as possible, that EPA's risk-assessment decisions are made with access to the best science available. It also recommended that EPA continue to rely on its Science Advisory Board and other expert bodies to determine when departing from a default option is warranted.

The Commission goes further in its recommendations about peer review, noting that peer review has not been used to evaluate the use of scientific or other technical information in regulatory policy and that there is no process for evaluating the effectiveness of peer review. The economic information used in regulatory policy is seldom peer-reviewed, and most agencies do not have official guidelines or policies for peer review. The Commission recommends several remedies for those problems while cautioning that the level of peer review should be commensurate with the importance or impact of the decision to be made. Peer review should not be used to stall the decision-making process.

6. Comparative Risk

The NRC report concluded that EPA should pay more attention than it now does to the appropriateness of various procedures for risk comparison. A scientifically sound way to do that would be to modify risk-assessment procedures to characterize more specifically the uncertainties in each comparison of risks--some larger, some smaller than the uncertainties in individual risk assessments. Because of the substantial and varied degrees of model and parameter uncertainties in risk estimates, it is almost impossible to rank relative risks accurately unless the uncertainty in each risk is quantified or otherwise accounted for in the comparison. If comparison of risks is imperative for regulatory purposes, the report suggested attempting to compute the uncertainty distribution of the ratio of two risks and choosing from it one or more appropriate summary statistics.

The Commission has addressed comparative risks from the perspectives of both risk communication and of conducting comparative risk projects for priority-setting. The Commission recommends that risk comparisons for risk communication help to convey the nature and magnitude of a particular risk estimate and be restricted to comparisons of risks associated with chemically related agents, different sources of exposure to the same agent, different kinds of agents with the same exposure pathway, and different agents that produce similar effects. The Commission also agrees that the appropriateness of procedures used to compare risks for priority-setting requires attention and evaluation and suggests that comparative risk-ranking paradigms are appropriate for guiding resource-allocation decisions.

7. Exposure Assessment

The NRC report noted that EPA has traditionally characterized exposure according to two criteria: exposure of the total population and exposure of a specified highly or maximally exposed individual (MEI). The MEI's exposure is estimated as the plausible upper bound of the distribution of individual exposures. The reason for finding the MEI, as well as population, exposure is to assess whether any individual exposure might occur above a particular threshold that, as a policy matter, is considered important. In its most recent exposure-assessment guidelines, EPA no longer uses the term MEI, noting the difficulty in estimating it and the variety of its uses. The MEI has been replaced with two other estimators of the upper end of the individual-exposure distribution, a "high-end exposure estimate" (HEEE) and the theoretical upper-bounding estimate (TUBE). The HEEE is not specifically defined ("the Agency has not set policy on this matter"), but it is a value in the upper tail of the individual-exposure distribution. The HEEE is based on the estimation of the distribution of exposures that people might actually encounter; from the individual exposures, it is possible to develop population exposure (and risk) distributions and include uncertainty estimation and personal-activity patterns. The exact percentile that should be picked for the HEEE is not specified, but it should be chosen to be consistent with the population size in a particular application. The TUBE is a calculated value that is expected to exceed the exposures experienced by all individuals in the actual distribution. Neither the HEEE nor the TUBE is explicitly related to the MEI.

The NRC report recommended that the underlying assumption that calculated exposure estimates are conservative be reaffirmed; if it is not, alternative exposure models whose performance has been clearly demonstrated to be superior should be used in exposure assessment. Those alternative models should be chosen to provide more accurate, scientifically founded, and robust estimates of pollutant-exposure distributions (including variability, uncertainty, and demographic information).

The Commission believes that the results of an exposure assessment can be a source of greatest uncertainty in a risk assessment and agrees that there is a need for more accurate, scientific, and validated models for exposure assessment. EPA should move away from estimates of exposure that are based on a mythical overexposed individual, which are likely to overestimate the exposures of most of the population and underestimate the exposures of special populations, such as subsistence fishermen. Point estimates of exposure convey no information about the extent to which they overestimate or underestimate exposures, and they should be used only for screening risk assessments. The entire distribution of a population's exposure concentrations should be used for more refined risk assessments, rather than just the exposures of a highly exposed subpopulation (although highly exposed populations, if they exist, should be identified and evaluated separately).

8. Differences in Susceptibility

The NRC report points out that EPA and the research community have thought almost exclusively in terms of the bimodal type of variation, with a normal majority and a hypersusceptible minority. That model might be appropriate for noncarcinogenic effects, but it ignores a major class of variability with regard to cancer (the continuous, "silent" variety), and it fails to capture some bimodal cases in which hypersusceptibility might be the rule, rather than the exception. EPA's 1986 cancer risk-assessment guidelines, however, are silent regarding person-to-person variations in susceptibility and thereby treat all humans as identical, despite substantial evidence and theory to the contrary. That is an important "missing default" in the guidelines. The NRC report recommended that EPA adopt an explicit default assumption for susceptibility and that the magnitude and extent of human variability due to particular acquired or inherited cancer-susceptibility factors be determined through molecular epidemiologic and other studies. Results of the research should be used to adjust and refine estimates of risks to individuals and estimates of expected incidence in the general population. In addition, EPA should continue and increase its efforts to validate or improve the default assumption that, on average, humans to be protected at the risk-management stage have susceptibility similar to that of humans included in relevant epidemiologic studies, the most sensitive rodents tested, or both. EPA's 1996 Proposed Guidelines for Carcinogen Risk Assessment mention the importance of including information on susceptibility differences when available, but do not go so far as recommending an explicit default assumption.

The Commission agrees with the NRC report's conclusions regarding susceptibility. Risk assessments should be conducted so that populations with a special susceptibility or risk--whether because of greater exposures than the general population, because of other concurrent exposures, or because of some physiologic characteristic that increases sensitivity--are identified and the extent to which they are at greater risk determined.

9. Multipathway, Multisource, and Mixture Exposures

EPA currently adds the risks related to each chemical in a mixture to develop a risk estimate for that mixture. That approach is based on an assumption that doses of different agents can be treated as roughly additive with regard to inducing the end point; this assumption is reasonably consistent with much of the experimental evidence on the joint actions of chemicals in mixtures. The NRC report concluded that this additivity procedure is generally appropriate when the only risk characterization needed is a point estimate for use in screening. The Commission agrees that dose additivity of mixture components is an appropriate assumption for most cases, but it believes that the issue of dose additivity versus response additivity has not been adequately addressed.

The NRC report also concluded that any comprehensive assessment of health risk associated with environmental exposure to any particular compound must consider all possible routes by which people might be exposed to that compound, even if expected applications in risk management are limited to some particular medium or source. The report recommended that EPA consider using appropriate statistical procedures to aggregate cancer risks associated with exposure to multiple compounds. Aggregating risks associated with different exposures might not be possible, however, because the analyses for each exposure will produce risk estimates of differing accuracy and conservatism. The Commission agrees that procedures for aggregating risks must be explored. The issue of which end points or exposures can be aggregated appropriately is complex--for example, should different tumor types within the same organ or tumors in different organs be aggregated, or do these constitute different, independent responses? Considering multiple sources of contaminant exposure is particularly important in the context of environmental justice and identifying sensitive populations requiring special consideration, and methods to do so are needed.

10. Uncertainty

The NRC report concluded that it might be undesirable to reduce a risk characterization to a single number, or even to a range of numbers intended to portray uncertainty. Instead, the report recommended that EPA consider giving risk managers risk characterizations that are both qualitative and quantitative and both verbal and mathematical. The Commission concurs that better communication about risk-related uncertainty is needed, and it encourages regulatory agencies to explain the uncertainty associated with any numerical estimates of risk and to eliminate risk estimates with phony accuracy (e.g., 4.237 x 10^-5), which communicate a misleading confidence in accuracy. The Commission also believes that risk characterizations for routine risk assessments should emphasize qualitative information about risks more than quantitative information. Qualitative information is likely to be more understandable and useful than quantitative estimates or models to risk managers and the public. Qualitative information includes a careful description of the nature of the potential health effects of concern, of the strength and consistency of the evidence that supports an agency's classification of a chemical or other exposure as a health hazard, and of any means to prevent or reverse the effects of exposure.

The NRC report also concluded that any expression of probability regarding model uncertainties (i.e., inability to determine which scientific theory is correct or what assumptions should be used to derive risk estimates), whether qualitative or quantitative, is likely to be subjective. Subjective quantitative probabilities could be useful in conveying the judgments of individual scientists to risk managers and to the public, but the process of assessing subjective probabilities is difficult and essentially untried in a regulatory context. Substantial disagreement and misunderstanding about the reliability of quantitative probabilities could occur, especially if their basis is not set forth clearly and in detail.

As discussed in section 3.3 of the Commission's report, the Commission believes that, although there is general agreement as to the value of qualitative statements describing critical uncertainties in a risk assessment, there is opposition to the use of a more routine and formal mathematical approach to characterizing uncertainties. The opposition is based on the belief that a formal, quantitative approach is unnecessary, is difficult to perform, and will not improve risk communication. Uncertainty is inherent in any estimation procedure. Some sources of uncertainty, such as those related to estimating exposures, are likely to be relatively easily addressed through the use of statistical methods. Other types of uncertainty, such as those associated with species-to-species or high-to-low dose extrapolation, are less straightforward or quantifiable. Characterizing the uncertainty and variability that underlie a potential risks can generate a distribution of risks, instead of a point estimate, but it should be kept in mind that when data are scarce, assumptions about the underlying shape of a distribution will be needed--that is, when uncertainty is greatest, a range of probabilities based on assumptions would replace point estimates based on assumptions.

Providing a numerical range of risk estimates reflecting uncertainty and variability might allow decisions to be made in a more informed and more transparent manner than is possible when only a single point estimate is generated. However, communicating a range of risk estimates might be misconstrued by those unfamiliar with quantitative methods as implying that all the numbers in the range are equally likely or plausible and are therefore equally valid for regulation. Many risk assessments are crude yardsticks for decision-making. In this context, the routine provision of a range of risk estimates might only confuse and delay the regulatory process.