Work health and safety data: business intelligence, distraction or fake news

Speaker 1 (00:05):

Hello and welcome to this MSD podcast. My guest today is Sharron O'Neill, Associate Professor at the School of Business at the University of New South Wales, Canberra. Sharron, welcome.

Sharron O'Neill (00:19):

Great to be here and thank you for inviting me.

Speaker 1 (00:21):

What are the problems with how safety data is being done today?

Sharron O'Neill (00:28):

Well, I think there's a number of problems. There's some companies doing it well, but there is a number of problems. The most important one would be poor quality data. So measuring the wrong things, measuring things in the wrong way, or interpreting the measures in all sorts of weird and wonderful ways. And so if we have poor quality data, it's not going to inform the decisions well. And the other problem that we have is OPMs, measuring other people's measures. So we often get this concern about benchmarking and wanting to be able to benchmark ourselves against other organizations. And so we measure what they measure rather than measuring what we really need to measure to help inform our decisions.

Speaker 1 (01:12):

So it is true that you manage what you measure?

Sharron O'Neill (01:17):

To an extent. We hear it both ways. You've got to measure in order to manage and you manage what you measure. And to some extent, that's true, especially when there is a good link between the measures and what you're managing, but that's not necessarily a straight line. So it depends on how useful and reliable and valid those measures are for what you're trying to achieve, how well they link to your strategic and operational decisions. And if you're measuring the right things and you're measuring them in the right way, then it's going to help you manage.

Speaker 1 (01:51):

Given that then, what sort of data should a business collect?

Sharron O'Neill (01:56):

The ones that matter. The things that actually do inform their decision. So I think the important thing is to not start with what measures do I collect. We have to start with what am I trying to achieve? What does good look like for my business? How do I know when things are working well? And once I know what good looks like, and I know what I'm aiming for, and I know what my goals are, then I can start choosing the measures that are going to let me know if I'm achieving that or not, and how I'm progressing towards that.

Sharron O'Neill (02:27):

So we get a lot of organizations that are very focused on lost time injuries and those kinds of outcome measures and they're useful, but not for managing. They're useful for telling us about how our people are harmed, how many of our people are harmed, but they're not necessarily going to tell us whether we've implemented an intervention really well because sometimes there's a lot of other confounding factors between that intervention and the injury outcomes that occur way up the track.

Sharron O'Neill (02:59):

So we need to get lead and lag indicators that are much closer to the kinds of interventions that we're putting in place. So I think when we're looking at what measures we do, we need to think about the different types of measures. What are you trying to measure? Are you trying to measure what you've done? So the implementation of policies and processes and interventions? Are you trying to measure how effective those activities have been? Or are you trying to measure what are the outcomes at the end of the day in terms of human damage? And I think if we can get a clear line of sight as to what we're actually trying to achieve with our data, what we need to know, then we can start choosing the measures that are going to help us actually inform those decisions best.

Speaker 1 (03:43):

And just on that, is there evidence or research that tells us what measures are the most effective to positively impact safety?

Sharron O'Neill (03:54):

I think there's a range of research that tells us about the measurement process and what we need to be doing in terms of selecting measures and how we need to develop them. So looking at all of those criteria, like making sure they're relevant to the decision, making sure they're reliable which means that they're accurate and that they're not biased, and making sure, we measure things and interpret those measures in a valid way is really important. So once we start thinking about those characteristics, we can start to say, "What are the measures that are most useful?" And the research on injury measures, for example, tells us that looking at lost time injury rates is not going to give us a good indication about the impact, the human impact or the business impact of injury outcomes because it's very closely correlated with very high-frequency, low-consequence injuries.

Sharron O'Neill (04:51):

So if you're really wanting to understand how many people are being seriously harmed, we need to be looking at high-potential injuries or high-consequence injuries. Aligning those measures that you're choosing to what you really want to know. I mean, are you interested in how many people get minor injuries and paper cuts? Probably yes, to a point, but more important, how many people are really seriously being damaged? And so we need to be able to separate that out in the data. And then, when it comes to our leading indicators, a lot of companies will measure things around audits and training. Is that telling you what you really need to know? So research. It's interesting the whole idea around leading indicators. There've been calls in the research for the better development of leading indicators since the 1980s, late 1980s, that I'm aware of. And we're still not really getting to grips with leading indicators.

Sharron O'Neill (05:51):

There's certainly been a lot of work around safety climate, and safety culture as some would call it, but climate indicators. And that can give you a good indication of leadership. But at the end of the day, these are perception measures, which will give you some information. They are a leading indicator, but they are not the leading indicator. So we need to also look at complimentary measures that can help provide closer, sort of detail and information about the things that really matter, what our burning issues are, and allow us to make some informed decisions.

Speaker 1 (06:26):

Sharron, what advice do you have for people in safety roles who are trying to put a business case forward for safety interventions, such as design controls for MSD risks?

Sharron O'Neill (06:39):

Well, I think there's a few things there. And one is when we talk about a business case, a business case can be a legal case, it can be a moral case, it can be a financial case. So it's not necessarily saying here's the cost benefit analysis. The other thing is cost is, of course, very important. And so when we are talking about business case, we need to look at the cost impact of the things, the suggestions and the interventions, and the infrastructure that we're putting forward. Using a mix of quantitative, qualitative, and financial information is usually good to sort of build that rounded case. But when it comes to the financial information, I think one thing to be aware of is, cost benefit is very difficult to show; cost effectiveness is easier. So rather than trying to say, "Here's the cost of doing it. Here's the cost of not doing it." We know that the costs of putting in place different programs or purchasing assets or taking on new things.

Sharron O'Neill (07:43):

Okay. So we know that when it comes to investing in infrastructure or programs and initiatives, the costs are quite easy to determine because this is what we're paying for those investments. The costs that we are mitigating in terms of the benefits are much harder to quantify because you're trying to cost things that won't happen or haven't happened. So they tend to be undercosted, so it makes it harder to actually convey a business case in the traditional cost benefit sense. I think if we can link it to operational effectiveness, if we think about costs in terms of that operating safe operation, safe business, the whole idea of not funding health and safety, but funding safe and healthy work is I think really, really important. And this is what we put forward in our measuring and reporting on safety document that we publish through Safe Work Australia's website a few years ago.

Sharron O'Neill (08:43):

It really has to come back to trying to see safety as part of the integral business operating activity. It's not about safety as a separate silo. And if we're trying to fund it as a separate silo, it's going to make it much more difficult to get that through. So if we can show that we're funding safe, healthy, and productive work, if we can make sure that we look at the effectiveness of our spend, for example, if I want to invest in some infrastructure, I might be looking at two alternatives and which one's going to give me the most safety benefit for my dollar, is much more useful than trying to say, "Here's an investment, and here's the cost that I'll avoid if I bring this on."

Sharron O'Neill (09:29):

So there's a number of reports on Safe Work Australia's website that we've written around these issues. We've got one there on the business case for safety; another one on performance measures, incentives, and organizational climate; and another one on measuring and reporting on health and safety. So we'll have to talk about these and much more on the upcoming presentation. And I'll also have a talk about how to actually look at designing some of those lead and lag indicators and identifying the ones that are going to be most relevant to you. So I look forward to seeing you there.

Speaker 1 (10:03):

Sharron O'Neill, thanks so much for your time today.

Sharron O'Neill (10:06):

Thank you.