Read more: |
Matti Grönroos
Are Your Metrics Valid?
The next step is to turn the metrics to SLA targets. This is a good way to steer vendors, we believe.
But are your metrics relevant? Do they measure something which the businesses are interested in? Or do they exist just because it is nice to know?
Are your metrics trustworthy? Can they be manipulated by the vendor? Or would you like to manipulate them?
Especially when sanctions and bonuses are based on the metrics, the metrics and their targets shall be designed carefully. There are various criteria for the validity of the metrics. The Top 3 might be
- Relevant
- Indisputable
- Immune to manipulation
The relevance has a wide spectrum of aspects. The metrics must have some correlation to quality and preferably also a correlation to business needs.
Relevance in also about avoiding overlaps: Do not create several metrics to measure the same thing. A case: Because the ticketing systems make it technically possible, the Incident Management related SLAs often have targets for both the time-to-react and time-to-resolve. These are highly overlapping. From the customer's perspective, the time-to-react is usually irrelevant information. It should not be of any interest, because the time-to-resolve metrics reflects the quality perceived. The time-to-react is needed by the service provider as an internal metric for workload planning purposes.
Indisputable metrics can be trusted by both the vendor and the customer. Of course, the metrics can be interpreted in a different way at the different sides of the table, but neither party can repudiate the numerical value of the metric.
Most metrics can be manipulated. That risk shall be recognized, and the trends of the metrics shall be followed and analyzed.
Example: An outsourced Service Desk has tight time-to-resolve targets, and the billing is done by the number of solved tickets.
- Less tickets, less money
- Many tickets, more money
- Less tickets, much effort per ticket, loss
- Many tickets, less effort per ticket, profit
It is easy to be tempted to close the ticket as quickly as possible, whether the solution makes sense or not. If the user is not happy, the original ticket is not re-opened but a new one is created and closed as soon as possible. This is the way to nice SLA scores (and leading to the watermelon problems), and double money is earned.
This dishonesty can be easily improved: The user can be given a short time only to approve or disapprove the resolution. If the user is busy and misses this deadline, the ticket will be automatically closed as approved.
Also, the customer can be the one to manipulate metrics. Example: In many service agreements, the service provider is entitled to receive a bonus if the user satisfaction exceeds the threshold. If there is a lack of money on the customer side, the management might disallow giving too high satisfaction scores.