Bench-Marking Safety? A Document for Discussion. Robert Miles, OSD HSE, Merton House, Staanley Road, Bootle, Merseysid, L20 3DL
Safety in the UK Sector of the North
Sea is in the process of changing from a Prescriptive Environment
to a Goal Setting one. This change is occurring because of the
widely held perception that the prescriptive approach had failed
and that a goal setting approach would avoid the failings of the
earlier approach.
The failings of the prescriptive
regime were those common to all forms of prescriptive regulation:
A goal setting regime was seen as
a cooperative venture in which Industry and Regulator would work
together to produce a climate which would motivate Operators to
drive safety standards beyond those previously achieved by regulation.
This was to be achieved in part by innovation and attention to
issues beyond those previously covered by safety analyses.
As such the goal setting regime should
be seen in the context of other organisational management trends.
The criteria by which a series of key business activities were
managed throughout industry had been subjected to analysis and
change. In each case the change had been from a system of defined
standards to one of aiming towards perfection. Total Quality
Management (TQM) changed production quality aims from a specified
failure rate (say 5%) to a strive for zero defects, Just in Time
(JIT) stock control systems substituted the guidelines of economic
reorder quantities with the goal of zero stock.
The essence of these and other such
initiatives was to substitute absolute goals for threshold criteria.
Experience had shown that defined criteria could only be improved
by small amounts, for example an organisation's target would be
a reduction in defects from 5% to 4% next year, this tended to
constrain thinking to a "more of the same" form of solution.
However absolute goals had the effect of promoting fundamental
reassessment of activities so that these previously unimaginable
goals could become within reach. The goal setting elements of
the Safety Case regime should be seen in this light.
The effects of this change of attention
to absolute values can be seen in the acute change in the trend
in offshore maiming figures for new installation size. A drive
towards ever larger and self sustaining platforms has been replaced
by one towards demanning and not normally manned installations.
This new trend allows zero accidents to be a viable goal in a
way that larger installations could never have done.
There are however two critical areas
in which a safety regime differs front those of quality or stock
control, one is the availability of information regarding the
wider industry performance, the other is in the perception of
the costs of failure. There has historically been a perception
in many sections of industry that the costs and other demands
of safety conflict with the aims of the organisation to maximise
profit. This perceived opponent process situation is unique to
safety; improvement in quality, reductions in stock or increases
in reliability are always seen as contributing directly to the
organisations profitability. That is why safety has to be subject
to external assessment and regulation and why quality, stocks
and reliability do not.
Returning to the first difference,
that of the information available in the market place; (with the
possible exception of surgical mortality rates), true safety performance
is remark-ably difficult to establish. In some cases reportable
incidents are few and far between so that it can take years to
achieve a significant sample, in many cases whole classes of incident
are reported to different organisations, in most cases near misses
are ignored and in all cases incidents are under reported,. Most
reporting is tinged with the suspicion of some form of retribution,
either from within the organisation or front tile regulatory authority.
However in the case of commercial variables the converse is often
true, quality can be inspected by any potential customer, the
competitive reliability of products is the subject of many popular
publications and the efficiency of stock control can be assessed
from the annual reports.
These two features of the safety
environment have an important effect; they act to weaken any innate
drive towards excellence which is the core feature of the progress
towards excellence in the other domains. The drive towards safety
has traditionally been motivated by completely different values,
those of human conscience and compassion. The valuable work now
being undertaken by HSE's Accident Prevention Advisory Unit (APAU)
in determining the true costs of accidents seeks to redress the
balance, never the less the basic principle holds.
The openness of these other "markets"
allows the establishment of historical criteria for the performance
of an industry on a relevant indicator against which current performance
can be judged. That is a functional industry performance standard.
The ultimate realisation of this approach must be the financial
ratios by which investors and financial market regulators judge
the health of companies.
A Safety Case regime such as that
now being put into place in the UKCS is dependent on the availability
of such industry performance standards in tile assessment of the
risks, and extent of their mitigation proposed in the Safety Cases.
In some cases the standard may be more implied than actual, in
so far as any individual in a Regulatory Authority who had read
a substantial number of Safety Cases would have an internalised
(possibly sub-conscious) estimate of the prevailing standard.
Such an individual may not be able to define that standard but
could probably be able to assert that a given submission was below
an acceptable level. The problem with this approach is that it
can not operate within the time scale available or within the
distributed "matrix" management structure necessary
to complete such a large task.
A prerequisite for such a goal setting
regime must therefore be the open provision of such data necessary
to allow an assessment of the relative position of an organisation.
That assessment could be by a Regulator, the organisation itself
possibly either its safety manager or a safety representative,
a Trade Union, a potential employee, a potential trading partner
in a joint venture, an investor, and insurance assessor or simply
an interested member of the public. This list it not exhaustive
but it seeks to establish an important point; once the ability
exists to establish the safety performance of an organisation
against the rest of the industry there will be a powerful set
of "market" forces acting to drive the activities of
the organisation into pushing its position up the rankings. There
is no equivalent to this positive force under a prescriptive regime.
The existence of the scale and a
measure does not in itself allow a decision to be made as to the
acceptability of a given score or rank position. Faced with such
a situation a range of possible acceptance protocols are possible:
All of these alternatives have advantages
and disadvantages, the use of a fixed criteria "score"
provides a clear target and is easily understood, however in practice
it functions simply as prescriptive regulation and suffers the
same failings. The use of a criteria based on standard deviations
has the advantage that the mean will tend to rise as improvements
are made so presenting a continual drive for improved performance.
Some assessment of the form of any possible distribution would
have to be made as would the effects of distribution profile (kurtosis
and skew) on the operation of the criteria
A rank order approach has the benefit
of being self evident in its operation, however all experience
of human behaviour would suggest that such overt grading will
provoke disputes or appeals regarding positions at or near the
cut off score. The use of quartiles may solve some of these difficulties, the measure has the advantages of being based
on a parametric analysis while using sufficiently wide groupings
so preventing detailed individual comparisons. Under such a regime
all organisations scoring within the lowest quartile would be
requested to demonstrate that they were instituting measures to
move from this bottom group. Organisations in the second lowest
quartile would of course also have to institute improvement measures
as they ran the risk of being overtaken and finding themselves
some time hence in the unacceptable region. There may be a case
of "damping" the system by applying a sliding scale
of times to correct deficiencies around the criterion point.
Alternatively it may be possible to apply a coefficient based
on an organisation's aggregated score across a range of assessed
functions.
How could such a system be operationalised?
First and foremost is tile collection and publication
of safety performance data. This can be coordinated by the industry,
independent commercial organisations, the Certifying Authorities
or a Regulatory Authority. Provided the data collection and collation
is transparent then it may not matter which of the above undertake
the task, there may indeed be a role for a range of types of organisation
to be involved. Any system would, for validity both in the public
eye and that of tile organisations, have to be independently audited.
Such a task would probably best be by the relevant Regulatory
Authority as concerns regarding commercial or other conflicts
of interest could rapidly undermine or discredit the system.
What types of data would be involved
in these processes? Three workable categories can be defined;
historical data, current performance and projected performance.
Historical data will consist of accident reports, safety records,
near miss reports and reliability or failure rates. Current
performance is concerned more with the methods by which these
risks are managed and so would encompass procedures and working
methods and their assessed risks. Future, or projective, data
will be focused on new or proposed designs and working practices
and their estimated risks, it is in this area that the
main push will occur for innovative solutions to the safety
questions raised by the current and historical data. However
any tendency to use generic predictive data in cases where historical
data exist should be resisted, only historical data can be verified
or provide the critical comparisons between organisations engaged
in the same activities. Most of this data will be perceived by
the organisations to have commercial implications, some of it
may have. Attention must be focused on the fact that a safer
industry will become a more profitable industry, this is the trend
that APAU is now quantifying and as their analyses show it is
driven by such factors as consequential loss, loss of subsequent
business and litigation.
How should the data be collected?
There have been a range of initiatives in this field from a variety
of sources, in the case of the offshore industry there already
exists the statutory OIR9/a reporting of accidents involving injury.
This system is being supplemented by the recent introduction
of the OIR12 (voluntary) system for the notification of hydrocarbon
releases. This second initiative is particularly interesting
as it forms part of a larger process to collect data relating
to the population of relevant equipment in the UK Sector, thus
allowing proper estimates of risk to be made. Independent industry
reliability and incident databases such as WOAD, E&P Forum
and OREDA have been in operation for some time. Most operators
will addition hold their own safety data as do the insurance
underwriters. Currently reporting rates vary and so an accurate
assessment of a particular hazard may require reference
to multiple sources. The model of no fault reporting common to
the civil aviation industry may well be the best method. This
issue of fault and possible attribution of blame is critical in
any reporting system. The current trend to include "human factors" in accident analyses has exacerbated
this problem by making the issues of human involvement
and by implication, agency, explicit. Ultimately this difficulty
may only be overcome by the establishment of a protocol or guidance
as to the balance between the collection of data and the apportion
of blame.
How should the information be disseminated?
A short answer to that might well be "in the simplest and most accessible manner." In practice the process
is likely to involve collation and processing of the data into
annual figures and moving averages thus exposing any significant
trends. The data would need to accompanied by a glossary and
published within a reasonable time of collection, in this case
probably a short while after the year end. This is effectively
the case with the accident statistics published in the appendices
of the HSE annual report and the data for the Offshore Oil Industry
contained in what has become known as the "Brown Book"
(published by the Department of Energy). An initiative is currently
under way to provide fir more detailed information on the risks
of diving operations using the data contained in the diving
incidents database.
To recap, the essence of the system
described is;
There are factors which can distort
or bias the operation of this type of "free market"
system. They are the same as those which occur in commercial
markets; monopolies and cartels. A monopoly situation will occur
when the bulk of the data on an issue originates from a
single source, be that all organisation, industry or procedure.
In such cases, just as with trade, efforts have to be
made to seek and encourage alternatives. This may not always
be a realistic option, never the less in these instances
it may be possible to generate alternative data from mathematical
models, or feasibility studies using synthetic or derived data.
The case of cartels may prove more
complex. The covert operation of a bias in reporting by a group
of organisations or individuals will be difficult to detect.
This situation is particularly damaging as its effect is to depress
the average and hence inflate performance for the whole of all
industry. Under such conditions a means of independent assessment
will be required to periodically test tile validity of data collected
by an industry. A particularly difficult case is that of only
presenting some possible technical improvements in a given year
and so carrying over technical innovation for use in a future
year. This regulation of the rate of progress is insidious and
difficult to detect for there is always some progress to be called
as evidence, tile point is that it is not the best available.
This type of abuse can only be detected by independent research
aimed at establishing the true current and future states of the
relevant technology. HSE maintains an independent research base
and its use in part to fulfill this function could be envisaged.
The use of research in this way is
a tangible manifestation of the "As Low As Is Reasonably
Practical" (ALARP) principle for it establishes an independent
measure of what constitutes "Reasonably Practical."
It is not necessary that independent research is brought to bear
in every instance of the ALARP principle, simply that the
ability to apply it exists for any instance; that is it functions
as a deterrent.
The ALARP principle is usually represented
by all inverted triangle divided into horizontal regions of respectively;
unacceptable risk, ALARP region, and acceptable risk. While this
presentation of the principle properly displays tile fact that
ALARP operates over a range and also that some risk will always
be present, it tends to push interpretation towards thresholds
and numerical criteria. It is an unfortunate truism that it is
almost impossible to present the ALARP "triangle" without
someone placing numerical values on the boundaries of the ALARP
region. This is a weakness of the presentation, not the principle.
ALARP should be represented as a decreasing exponential which
will asymptote at the underlying societal risk. This view of
the process focuses more attention oil the process on continual
improvement implicit in the ALARP principal and which remains
its greatest strength.
It may have been noted that no mention
has been made of the role of inspections and prosecutions in this
process. Inspection and prosecution would not be required if:
Unfortunately as we all recognise
neither is true of humans although as we would also recognise,
some individuals or groups approximate to the ideal more closely
than others. In the first instance the threat of prosecution
serves as a reminder to all that an intention to make safety improvements
in a process or operation should be followed within a reasonably
short period by action to implement those improvements. In the
second case prosecution demonstrates in the clearest way possible
that all actions relating to safety should be in the same direction;
towards safety, and that society does not accept and will punish
those who, particularly for gain, seek to drive the process in
the opposite direction. Without wishing to be pessimistic, experience
of human nature suggests that some form of inspection and prosecution
role is likely to be necessary for the foreseeable future, no
matter how effective the goal setting regime. What can realistically
be hoped is that in future the need for prosecutions is principally
as a result of individual criminality, taking place within essentially
safe organisations and that this level of deviant behaviour is
at or below that prevailing in the population at large. From
that point on safety improvements would mirror the progress in
society towards the social and educational control of behaviour
alone.
This paper is a discussion document
which reflects the views of the author, the issues raised are
the subject of wider discussion and any final position reached
may adopt a different view.