What If We Stopped Using Incident Rates So Much?

Why is our profession so obsessed with the recordable injury rate? Think about it – almost every safety professional agrees that these sorts of measures are, at best, not great, and, at worst, awful. Yet we still all use these rates to measure safety success. We compare ourselves to others to see how we’re doing using these rates. If the rate goes down we get excited. If the rate goes up we get upset. Many of us even have our personal performance rated based on these types of injury rates.

Why?

No really, think about it for a second, why? Why do we consistently measure safety success using these rates?

If you’re like most, the reason that likely came to mind is something along the lines of – it’s an easy thing to measure, everyone else is doing it, and we need some way to measure performance, right? So essentially there’s real thought put into this. We do it because (a) it’s easy, and (b) everyone else is doing it. Effectively it’s a version of professional peer pressure that we’re giving in to.

Now, some would point out, rightly so, that we do need some sort of validation measure for the work we do. And, after all, if we’re doing things right then we should see less accidents happen, right? Here’s where it gets tricky. That’s sort of true, but not entirely. Yes, it’s true that if what we’re doing doesn’t prevent accidents, particularly the serious ones, then we need to question why we’re doing it. The problem is that the measures we’re using are not sensitive enough to really give us an accurate measure. This is really an issue with the statistics of it, and we know how many of you hate math, so we won’t bore you with the numbers. But the short explanation is that we simply do not have enough accidents. We need a large enough number of accidents to be able to reliably and validly use the incident rate as a measure of the effectiveness of any given intervention. Most organizations simply don’t even get close to that number, particularly of the serious accidents.

Of course math is scary, so we don’t want to think about that too much. That’s ok though, because even if we did not have the mathematical problems with the incident rate, we still would have at least two significant problems with using incident rates as measures of safety performance. Lets look at each.

To understand the first problem, lets look at a different kind of “incident”, the “near-miss” (or whatever you call it). There’s lots of definitions of near-misses out there, but one we like is “an event that could have caused an injury, illness, or other kind of accident, but didn’t because of luck”. (Some people are uncomfortable with the word “luck”. If that’s you, go ahead and substitute “luck” with “stochastic events within and outside the system over which we have little or no control”.) If we accept that definition then that means that an accident is an event that happened because of bad luck. If we were lucky it wouldn’t have happened, but it did happen, so we were unlucky. And if this is true, that means that a certain portion of an incident rate is controlled not by us or the organization, but by luck. So your incident rate can go up or down and the only reason is that you’re unlucky. Furthermore, if you are managing your program using this rate that means that you are attempting to manage luck. That doesn’t seem like a very effective business strategy to us.

Even if we assume that the effect of luck is nonexistent or negligible on your incident rate, we still have at least one more reason why incident rates are bad measures of safety performance. Essentially, you don’t really know what you’re measuring with an incident rate.

To illustrate, let’s use an example. Let’s say there’s an organization that initiates a safety program that is just terrible. It’s poorly planned and poorly executed, but the only measure the organization has in place to measure this is the incident rates. The employees at the shop floor still have to find a way to work without getting hurt, right? So they adapt to the new, bad program. However, the law of fluency predicts that this adaptation will naturally hide what is being adapted to. So employees will find a way to “make it work” and, unless the organization goes out and looks, they will be none the wiser. As a result the number of incidents may go down, causing the incident rate to go down. The organization will then believe that the program is a success, when in reality it’s the workers adapting to the poor work environment. The organization could have saved all the resources used in implementing the program and probably have gotten the same result.

This happens a lot in organizations. They implement a program and then they look at the incidents rates to identify if it was successful or not. If the rate goes down then they call it a success. But that can have nothing to do with the effort the organization put into the program. What this means is that if all you do is use incident rates to measure the effectiveness of your safety management system, it is entirely possible that you are wasting resources on ineffective programs.

Now we’re going to ask you to do something crazy – imagine what would happen if you just stopped paying attention to your incident rate. What if we just decided to stop using it to compare our performance year over year or as compared to others? Sure there will be reasons to still keep track of the rate, but if we stop making the rate so central to measuring safety performance would it really be so bad?