Summary of Meeting Paper

The 1996 Annual Meeting of the Society for Risk Analysis-Europe

Monitoring Perceptions of Food Hazards Over Time: Some Measurement Issues. Chris Fife-Schaw, Ph.D. and Gene Rowe, Ph.D. SPERI, Dept. of Psychology, University of Surrey, Guildford, LJK, GU2 5XH

Background

The psychometric approach to the study of hazard perceptions has yielded a number of useful insights into the differences between lay and expert assessments of risks (cf. Fischhoff et al, 1978; Slovic, 1993 etc.). Across a number of different hazard domains it has been demonstrated that lay samples make a variety of qualitatively different assessments about hazards when compared to expert risk assessors. Formal risk assessors are primarily concerned with making probabilistic estimates of the likelihood of harm or death from exposure to a hazard. Lay assessors consider additional characteristics of the hazard such as the degree to which the consequences of exposure are serious or 'dreadful' (the jargon term used in the field), the degree to which exposure is voluntary, the ease with which it can be avoided, the degree to which the hazard is known, understood and readily detected etc. These differences are thought to be responsible for the often major discrepancies between expert advice about 'rational' avoidance behaviour and public responses to the same hazards.

Studies using the psychometric approach traditionally ask respondents to make ratings about the degree to which a range of hazards possess each of a number of qualitatively different characteristics. These ratings are usually subjected to a principal components analysis to reduce the ratings to composite scores on a smaller number of components. It is common to see hazards plotted in component space to look at the relative positioning of the hazards and such analyses provide easy to understand summaries of hazard perceptions.

What is rare in this field is the use of this approach to study change over time and/or differences between groups. For example, were it feasible to map changes over time it would be possible to assess the impact of food 'scares' on public responses and the effectiveness of risk communications. Group comparisons might open up the possibility of better targeted risk and health communications and also, potentially, a better understanding of the origin of these perceptions. Whilst there is scope here to address important theoretical issues the necessary work on perfecting the measurement tools has yet to be done. In this paper we report data from a number of studies using, general public samples which have looked at the performance characteristics of a fairly standard psychometric 'risk perception' questionnaire.

In the domain of risk perception there is relatively little work that has concentrated specifically on food hazards. Using the psychometric approach we have been able to replicate the findings of Sparks and Shepherd (1994) which suggest that food hazards are judged in terms of the dreadfulness of the harm that might follow from exposure and the degree to which the hazards are known and understood. In addition, our data from focus groups indicate that a third judgmental dimension concerned with the 'naturalness' of the origin of the hazard is important in this domain. The following figure illustrates the relative positioning of 22 food hazards on the first two components from a principle components analysis. The data come from a random sample (n = 293) of the UK general public. The subsequent sections of this abstract deal with a number of important methodological issues.

Item Order Effects

In an experimental design random samples of the general public were mailed one of four versions of a questionnaire which differed only in the order in which the hazards appeared on the forms and the order in which the hazard characteristic judgments were to be made (2 hazard orders x 2 characteristic orders). While there are a range of analyses that could be reported here perhaps one of the most interesting concerns the impact of the presentation orders on worry/concern ratings. Worry is especially interesting since it taps into an affective response to each hazard unlike the majority of the hazard characteristics traditionally used in this kind of research which primarily tap into cognitions about the hazards.

We created a simple index of the total of the ratings of worry (Alpha = 0.88) and examined the effects of presentation order on this aggregate score. The data suggest that global worry ratings are influenced by both types of presentation order and that 9% of the variation in scores may be attributable to order effects. The larger of these two types of order effect can be interpreted as showing that filling in the questionnaire served to increase worry if this was rated at the end of the questionnaire. Further analyses are being conducted to determine the likely underlying mechanisms for these effects (e.g. possible anchoring and adjustment effects).

Reliability and Sensitivity

If the psychometric technique is to prove useful either as a tool for cross-sectional research or for tracking perceptions over time we need to be able to demonstrate both the reliability and sensitivity of the instrument. We have conducted a number of studies to address these issues. For example, we conducted a longitudinal study over a period of 5 months (total n = 1394) in which a random subsample of respondents received an identical questionnaire on both testing occasions. The responses suggest that at an aggregate level, responses on the two occasions are similar suggesting good reliability. Closer examination of the data also suggests that while the structure of perceptions appears constant over time there is considerable variability in ratings for some respondents. There remains, however, the possibility that apparent reliability is attributable to sample conditioning, a factor that we are currently pursuing in this research.

In a second study a random subsample received a second questionnaire along with a digest of the real-world food-related news stories that had occurred in the period between the questionnaire waves with the intention of increasing the likelihood of exposure to these news stories. Exposure to the digest seemed to have the effect of changing, ratings for those hazards mentioned in the digest but not others. While this is not a perfect test it does suggest that the instrument is sensitive to the impact of communication materials, a topic that we are currently pursuing, in this research.

Group Differences

To illustrate the potential utility of the psychometric approach for detecting differences between groups we will refer to comparisons between key groups who might, on a priori grounds, be thought to hold differing, views of food hazards. As an example we took hazard ratings of females under 40 with young children and compared their responses with unmarried males under 40 who had no children. The figure below plots both groups' ratings in the same component space (only the first two components are shown accounting for 66% and 21% of the total variance). In the figure the solid lines linking points indicate group differences that are significantly greater than zero (after Bonferroni correction). Females with children appear to regard food with colourings, foods potentially containing growth hormone residues, genetically altered foods and foods potentially containing the agent that causes BSE as more serious and 'dreadful' hazards than do males without children. We do not want to focus on why these perceptions might be different in this case, but we use this example to illustrate the potential utility of this technique for detecting differences between groups. Implications for validity tests will be discussed.

The paper will conclude that while there are serious methodological issues that must be addressed before attempting to apply the psychometric paradigm to the monitoring of food hazard perceptions it is possible and potentially very informative.

The research report here was funded by the Ministry of Agriculture, Fisheries and Food, Project 1A026.

References

Fife-Schaw, C.R. and Rowe, G. (in press). Public perceptions of everyday food hazards: A psychometric study. Risk Analysis.