To manage something you need to measure it, and to be able to measure this you need to monitor things, always remember MMM.
Without KPIs you are just guessing, you have a feeling but you really don’t know how you are doing, this is a very dangerous position today, you really need to know how you are doing, this is pure survival.
It’s not that we in in IT do not like to measure things, most people working in IT I know smiles when talking about performance, utilization, availability and other types of technical measurements. We have been doing this for a long time, there are so much measurement information out there you would not believe it. Therefore, the challenge is not lack of data, rather the opposite.
A Key Performance Indicator (KPI) shows how well something are doing related to your business objectives and balanced using a set of metrics.
So, let us start by killing all the different “The number of X” indicators because they are not indicators, merely metrics that may be used together with other metrics to calculate a KPI. I have seen far too many reports containing 12 pages of very handsome graphs, showing the number of incidents, changes and other important events in the IT operations. When handed to the CIO you will soon discover that all your hard work putting this report together is for nothing. Please let me illustrate this with an example.
Some years ago, I was a consultant in a large multi-national corporation, I was managing the Service Desk reporting, or rather temporarily observing the report routines, as part of an interim management task. One of the weekly reports was hugely popular, in terms of recipients, there were many important business managers getting this report. I reached out to them all, asked if they were happy with the information, and based on a few answers, so it seemed. Therefore, I tried a little experiment. One week I did not send the report as usual, it only took about 45 minutes until the complaints starting to grow, so about 1 day late I sent it. The forthcoming weeks I started to falsify numbers, adding strange values, I even quoted the full version of Sweet Home Alabama. Absolutely no reaction. I provoked the managers again, saying I would not provide the report again until I got feedback on section 2.4.3 in the latest report (the Lynyrd Skynyrd lyrics) and got some big smilies back, “ok, point taken”.
After this we had a meeting and the report was drastically changed to a 3-page slide set of KPIs, not done overnight but extremely useful for the business managers. As it turned out none of them actually read this report for the last year or so, because it was full of technical jibber-jabber they did not understand or could use.
A very smart man, Steven Covey, once wrote, in a book I hold very close to my heart (The 7 habits of highly effective people), “begin with the end in mind”. This certainly goes for KPIs.
The KPIs need to be related to the business objectives, this is always easy to say and often a little bit more complex to actually achieve. Normally most business objectives becomes very “macro” for IT, like being top 5 in the market, expand market shares, reach a high % customer retention rate to mention some examples. This is why the CIO needs to be part of the company management team, to bring these business objectives with him and boil them down to objectives that makes sense to the IT service provider.
For the IT service provider there should be a set of Critical Success Factors (CSFs) that in turn defines the need for KPIs, in turn the KPIs will require metrics and the metrics demands ongoing measurements. I am guessing many of you have seen the chain that starts in Vision/Mission and ends up in measurements, thus passing through CSFs and KPIs.
What is really a CSF, what are you Critical Success Factors within IT? One of the most common one are of course maintaining our customer happy with our service. Normally we do not even try to attach some sort of quantification of a CSF; these are qualitative and therefore need a set of KPIs to give us some visual feedback about how we are doing. One obvious KPI in this case would for example be Customer Satisfaction, normally defined at a level, let’s say “87% more than average satisfied”, for example related to the service they received from our Service Desk since it’s the main customer interface for most IT service providers. To get this KPI you would need a few metrics, such as number of calls/emails/contacts, customer satisfaction levels for the same period. The last part in our chain is then the individual measurements.
Try in your organization to take this top-down approach to your KPIs, what are the most important CSFs for your organization, and how can you illustrate their well-being with KPIs, that in turn tells you what metrics you need, and finally what you need to measure.
This is the way to go, turning it around, measuring and reporting what you can based on your technology, IT operations console, log files and such, is definitely not the way to go, there is really no way for anyone outside the IT operations teams to understand how these things matters to the business other than the most obvious things. This being said, these things are definitely not unimportant and might very well be related to other KPIs that are more internally focused.
What I really recommend for everyone to do is to find a good set of CSFs and use these in IT-version of the balanced scorecard, showing the overall performance in all aspects that are directly connected to direct business value. This can of course be developed to real-time reporting to the management team (management dashboards etc) and much more. What is most important, start from the top and work your way down.