When we have a business process that takes a very long time from start to finish, how can we measure and monitor its results to stay focused on frequent improvement?
One of our PuMP Community members, Mark, works in the pharmaceutical sector, where it’s not uncommon to have business processes that take a long time from start to finish. One of these is an investigation process, and Mark’s organisation has a desired performance result:
“Investigations are undertaken and closed without delay.”
Naturally, they want to track the duration from the time investigations begin to the time they are closed. Some cases can take over 1000 days, and the average is roughly 400 days. And there are usually a large number of active cases, that start and close on any day of the year. And Mark feels some pressure to find a way to measure this quarterly, because their Board meets quarterly and all their metrics are reported on a quarterly basis.
The duration of investigations is counted in days, but they have a few variations of how to compile this data into a measure. However, they’re not entirely convinced of the best one.
The first measure is:
Measure #1: Investigation Average Total Days = the average of the total days it takes to close an investigation, per quarter.
Mark see this as is a limited measure since the full range of investigation times (that is, the maximum and minimum) are ignored by averages, and he’s right. Therefore, one never knows just how long it can take to close some investigations. However, if the number of cases closing is high enough in each quarter, this measure, based on an average or based on a median, still provides a good ongoing assessment of the changes over time, even though there is a very large lag. I’d still include this as one of the measures for their result: “Investigations are undertaken and closed without delay.”
The second measure is designed by setting a maximum time target for investigation time and monitoring the % of cases completed within that time:
Measure #2: Investigations Meeting Completion Standard: the % of investigation cases completed in 365 days or less, from start to closed, per quarter.
This second alternative helps isolate the proportion of investigations that are taking too long, but it’s harder to track improvement as it happens, since the longest investigation times might be reducing closer to the 365 days, but the measure won’t pick this up until more are under 365 days. That makes this measure a “dull” instrument for detecting the size and speed of change, and not really capable of informing decisions about improvement. Rarely is a measure of this format accepted with high strength in a Measure Design.
A third measure is to express the total duration of days as a percentile. Mark is considering using the 90th percentile, as follows:
Measure #3: Investigation Time 90th Percentile: the 90th percentile of cases closed over a quarter, expressed as 12-month rolling average.
Mark is trying to find a way to track investigations regularly, even though they are a very slow-moving beast by their very nature. But the first problem with rolling averages is we cannot use XmR charts to monitor them, because XmR charts only work when the measure values are based on data from each values time period only. The second problem with rolling averages is that we cannot track the timing and size of changes. The size of change is averaged out over the rolling period, and the timing of the change is completely unknowable. We need both these pieces of information to make good decisions to improve performance.
But there is another option to measure the result of “Investigations are undertaken and closed without delay” frequently enough to make decisions that will improve it. Whenever we have a very lag result like this, we can look for opportunities to observe more frequent components of it. And the clue is in the way the result is articulated: “…without delay.”
Rather than monitoring only the total duration of investigations, where the data isn’t available until hundreds of days later, one option is to monitor the amount of delays during the investigations. There might be several steps within the investigations process where delays happen most. And that could suggest a measure like the following, to accompany the first measure above, “Investigation Average Total Days“:
Measure #4: Investigation Delay Days: the total days of delay that investigations experienced, divided by the total number of active investigations, per quarter (or per month if delays happen often enough).
Putting attention on delays means new data would likely need to be collected. But if reducing the time investigations take is the goal, then delays is exactly where attention needs to be placed anyway, don’t you think?