Why is our profession so obsessed with the recordable injury
rate? Think about it – almost every safety professional agrees that these sorts
of measures are, at best, not great, and, at worst, awful. Yet we still all use
these rates to measure safety success. We compare ourselves to others to see
how we’re doing using these rates. If the rate goes down we get excited. If the
rate goes up we get upset. Many of us even have our personal performance rated
based on these types of injury rates.
Why?
No really, think about it for a second, why? Why do we
consistently measure safety success using these rates?
If you’re like most, the reason that likely came to mind is
something along the lines of – it’s an easy thing to measure, everyone else is
doing it, and we need some way to measure performance, right? So essentially
there’s real thought put into this. We do it because (a) it’s easy, and (b)
everyone else is doing it. Effectively it’s a version of professional peer
pressure that we’re giving in to.
Now, some would point out, rightly so, that we do need some
sort of validation measure for the work we do. And, after all, if we’re doing
things right then we should see less accidents happen, right? Here’s where it
gets tricky. That’s sort of true, but not entirely. Yes, it’s true that if what
we’re doing doesn’t prevent accidents, particularly the serious ones, then we
need to question why we’re doing it. The problem is that the measures we’re
using are not sensitive enough to really give us an accurate measure. This is
really an issue with the statistics of it, and we know how many of you hate
math, so we won’t bore you with the numbers. But the short explanation is that
we simply do not have enough accidents. We need a large enough number of
accidents to be able to reliably and validly use the incident rate as a measure
of the effectiveness of any given intervention. Most organizations simply don’t
even get close to that number, particularly of the serious accidents.
Of course math is scary, so we don’t want to think about
that too much. That’s ok though, because even if we did not have the
mathematical problems with the incident rate, we still would have at least two
significant problems with using incident rates as measures of safety
performance. Lets look at each.
To understand the first problem, lets look at a different
kind of “incident”, the “near-miss” (or whatever you call it). There’s lots of
definitions of near-misses out there, but one we like is “an event that could
have caused an injury, illness, or other kind of accident, but didn’t because
of luck”. (Some people are uncomfortable with the word “luck”. If that’s you,
go ahead and substitute “luck” with “stochastic events within and outside the
system over which we have little or no control”.) If we accept that definition
then that means that an accident is an event that happened because of bad luck.
If we were lucky it wouldn’t have happened, but it did happen, so we were
unlucky. And if this is true, that means that a certain portion of an incident
rate is controlled not by us or the organization, but by luck. So your incident
rate can go up or down and the only reason is that you’re unlucky. Furthermore,
if you are managing your program using this rate that means that you are
attempting to manage luck. That doesn’t seem like a very effective business
strategy to us.
Even if we assume that the effect of luck is nonexistent or
negligible on your incident rate, we still have at least one more reason why
incident rates are bad measures of safety performance. Essentially, you don’t
really know what you’re measuring with an incident rate.
To illustrate, let’s use an example. Let’s say there’s an
organization that initiates a safety program that is just terrible. It’s poorly
planned and poorly executed, but the only measure the organization has in place
to measure this is the incident rates. The employees at the shop floor still
have to find a way to work without getting hurt, right? So they adapt to the
new, bad program. However, the law of fluency predicts that this adaptation will naturally hide what is being adapted to. So
employees will find a way to “make it work” and, unless the organization goes
out and looks, they will be none the wiser. As a result the number of incidents
may go down, causing the incident rate to go down. The organization will then
believe that the program is a success, when in reality it’s the workers
adapting to the poor work environment. The organization could have saved all
the resources used in implementing the program and probably have gotten the
same result.
This happens a lot in organizations. They implement a
program and then they look at the incidents rates to identify if it was
successful or not. If the rate goes down then they call it a success. But that
can have nothing to do with the effort the organization put into the program.
What this means is that if all you do is use incident rates to measure the
effectiveness of your safety management system, it is entirely possible that you
are wasting resources on ineffective programs.
Now we’re going to ask you to do something crazy – imagine
what would happen if you just stopped paying attention to your incident rate.
What if we just decided to stop using it to compare our performance year over
year or as compared to others? Sure there will be reasons to still keep track
of the rate, but if we stop making the rate so central to measuring safety
performance would it really be so bad?
I am fresh out of school and over the past 2 years of the safety program, we talked repeatedly about accident and incident rates.
ReplyDeleteMy question is, what would be the most effective way to measure the success of a company's safety program?
Thanks,
Melissa
I am fresh out of school and over the past 2 years of the safety program, we talked repeatedly about accident and incident rates.
ReplyDeleteMy question is, what would be the most effective way to measure the success of a company's safety program?
Great question Melissa! The short answer is to look at what you are doing and would like to do more of and find ways to measure that. Those actions that you are doing as an organization and/or want to do more of are likely better predictors of safety performance than accidents are. So this could include get more management walk arounds, audits completed, tracking of corrective actions, etc. Whatever you're working on, find ways to measure it and you're on your way to creating leading indicators. These change the conversation away from what you are trying to avoid, to where you are going towards.
ReplyDeleteFor a longer answer keep an eye out for this week's blog which should come out in a couple days. We'll discuss leading indicators a bit more.
Thanks!
Unfortunately,the truth is that management has been programmed to believe that the ORIR is the one thing that will bring OSHA to the front door quicker than anything else. To change this culture will require years of hard work on the part of a new generation of safety leaders,to which I pray are up to the task.
ReplyDelete