What makes for a compelling metrics story?

reading1In my first article “Do your metrics tell a story?” I discussed the “traditional” approach to reporting metrics, and why that approach is ineffective at driving action or decisions.

Personal observations are far more effective. Personal observations appearing to conflict with the data presented can actually strengthen opposition to whatever decision or action the data suggests. Presenting data as part of a story reboots the way we receive data. Done well, it creates an experience very similar to personal observation.

So how can we do this well? What makes a compelling metrics story?

Every element must lead to a singular goal

This cannot be stressed enough. Any metrics story we tell must have a singular purpose, and every element of the package must exist only to achieve that purpose. Look at any report package you produce or consume. Is there a single purpose for the report? Does every piece of information support that single purpose? Does the audience for the report know the singular purpose? If the answer to any of these questions is no, then there is no good reason to invest time in reading it.

ITSM legend Malcolm Fry provides an excellent example of the singular goal approach with his “Power of Metrics” workshops. If you haven’t been able to attend one of his metrics workshops, you are truly missing out. I had the honor when Fry’s metrics tour came through Minneapolis in August 2012. The most powerful takeaway (of many) was the importance of having a singular focus in metrics reporting.

In the workshop, Fry uses a “Good day / Bad day” determination as the singular focus of metrics reporting. ThoughtRock recorded an interview with him that provides a good background of his perspective and the “Good day / Bad day” concept for metrics. The metrics he proposed all roll up into the determination of whether IT had a good day, or a bad day. You can’t get clearer and more singular than that. The theme is understood by everyone: IT staff, business leaders … all the stakeholders.

There are mountains of CSF/KPI information on the Internet and organizations become easily overwhelmed by all the data, trying to decide which CSFs and KPIs to use. Fry takes the existing CSF and KPI concepts and adds a layer on top of CSFs. He calls the new layer “Service Focal Point”.

The Service Focal Point (SFP) provides a single measurement, based on data collected through KPIs. Good day, bad day is just one example of using SFPs. We only need to capture the KPIs relevant to determining the SFP.

(Fry also recently recorded a webinar: Service Desk Metrics — Are We Having a Good Day or a Bad Day? Sign up, or review the recording if you are reading this after the live date).

Create a shared experience

A good metrics story creates a new experience. Earlier I wrote about how personal histories – personal experiences – are stronger than statistics, logic, and objective data in forming opinions and perspectives. Stories act as proxies for personal experiences. Where personal experiences don’t exist, stories can affect opinions and perspectives. Where personal experience does exist, stories can create additional “experiences” to help others see things in a new way.

If the CIO walks by the service desk, and sometimes observes them chatting socially, her experience may lead to a conclusion that the service desk isn’t working hard enough (overstaffed, poorly engaged, etc.) Giving her data demonstrating high first contact resolution and short caller hold times won’t do much to change the negative perception. Instead, make the metrics a story about reduced costs and improved customer engagement.

A great story creates a shared experience by allowing us to experience similarities between ourselves and others. One of the most powerful ways to create a shared experience is by being consistent in what we report and how we report it. At one point in my practitioner career I changed metrics constantly. My logic was that I just needed to find the right measurement to connect with my stakeholders. It created the exact opposite outcome: My reports became less and less relevant.

The singular goal must remain consistent from reporting period to reporting period. For example, you may tweak the calculations that lead to a Good day / Bad day outcome, but the “storyline” (was it a good day or a bad day?) remains the same. We now have a shared experience and storyline. Everyone knows what to look for each day.

Use whatever storyline(s) works for your organization. Fry’s Good day / Bad day example is just one way to look at it. The point is making a consistent story.

Make the stakeholders care

A story contains an implied promise that the story will lead me somewhere worth my time. To put it simply, the punch line – the outcome – must be compelling to the stakeholders. There are few experiences worse than listening to a rambling story that ends up going nowhere. How quickly does the storyteller lose credibility as a storyteller? Immediately! The same thing happens with metrics. If I have to wade through a report only to find that there is ultimately nothing compelling to me, I’ll never pay attention to it again. You’ll need to work pretty hard to get my attention in the future.

This goes back to the dreaded Intro to Public Speaking class most US college students are required to take. When I taught that class, the two things I stressed more than anything was:

  • Know your audience
  • Make your topic relevant to them

If the CIO is your primary audience, she’s not going to care about average call wait times unless someone from the C-suite complained. Chances are good, however, that she will care about how much money is spent per incident, or the savings due to risk mitigation.

Know your ending before figuring out the middle of the story

This doesn’t mean you need to pre-determine your desired outcome and make the metrics fit. It means you need to know what decisions should be made as a result of the metrics presentation before diving into the measurement.

Here are just a few examples of “knowing the ending” in the ITSM context:

  • Do we need more service desk staff?
  • How should we utilize any new headcount?
  • Will the proposed process changes enable greater margins?
  • Are we on track to meet annual goals?
  • Did something happen yesterday that we need to address?
  • How will we know whether initiative XYZ is successful?

A practical example

Where should we focus Continual Service Improvement (CSI) efforts? The problem with many CSI efforts is that they end up being about process improvement, not service improvement. We spend far too much time on siloed process improvement, calling it service improvement.

For example, how often do you see measurement efforts around incident resolution time? How does that indicate service improvement by itself? Does the business care about the timeliness of incident resolution? Yes, but only in the context of productivity, and thereby cost, loss or savings.

A better approach is to look at the kind of incidents that cause the greatest productivity loss. This can tell us where to spend our service improvement time.

The story we want to tell is, “Are we providing business value?”

The metric could be a rating of each service, based on multiple factors, including: productivity lost due to incidents; the cost of incidents escalated to level 2 & 3 support; number of change requests opened for the service; and the overall business value of the service.

Don’t get hung up on the actual formula. The point is how we move the focus of ITSM metrics away from siloed numbers that mean nothing on their own, to information that tells a compelling story.

If you would like guidance on coming up with valid calculations for your stories, I highly recommend “How to Measure Anything: Finding the Value of Intangibles in Business” by Douglas Hubbard.

… and a few more excellent resources:

Image credit

Do Your Metrics Tell a Story?

Do your service management metrics tell a story? No? No wonder nobody reads them.

That was a tweet I sent a few weeks ago, and it’s had some resonance. I know that during my practitioner days, I missed many opportunities to tell a compelling story. I wanted everyone else to get the message I was trying to communicate, and couldn’t figure out why my metrics weren’t being acted upon. I had a communications background before getting into IT, so I should have known better.

Facts are not the only type of data

I’ve blogged about metrics a few times before. In “Lies, Damned Lies, and Statistics: 7 Ways to Improve Reception of Your Data” I shared a story about how my metrics had gone astray. I was trying to make a point to reinforce my perspective on an important management decision. In what became a fairly heated meeting, I found myself saying at least three different times, “the data shows…” Why wasn’t it resonating? Why was I repeating the same message and expecting a different result?

readingGo back and read that article to see how it resolved. The short answer: I lost.

I’d love to live in a world where only objective, factual data is considered when making decisions or influencing others; but we have to recognize two important realities:

  1. Other types of data, especially personal historical observations that often create biases, are more powerful than objective data ever could be.
  2. Your “objective” factual data can actually reduce your credibility, if it is inconsistent with the listener’s personal observations. As the information age moves from infancy into adolescence, we are becoming less trusting of numbers, not more.

So, giving reasons to change someone’s mind is not only ineffective, it can also make things worse. Psychological research indicates that providing facts to change opinion can cement opposing opinion more deeply than before.

Information, whether accurate or not, can be found that backs up almost any perspective. Why should I trust your data any more than the data I already have? Read the comments section from almost any news story about a controversial subject. How many minds get changed?

We need a reason to care

Why should I pay attention to, act on, or react to, your metrics if there is no compelling reason for me to do so? We have to give our audience a reason to care. We want the audience of ITSM metrics to do something as a result of the metrics. The metrics should tell a story that is compelling to your intended audience.

Let’s look at a fairly common metric – changes resulting in incidents. Frequently we look at the percentage of changes that generated major incidents (or any incidents at all). Standing alone, what does this metric say? Maybe it shows a trend of the percentage going up or down over time. Even so, what action or decision should be made as a result of that data? Without context we can look for several responses:

Service Desk Manager: “Changes are going in without proper vetting and testing.”

Application Development Manager: “We need to figure out why the service desk is creating so many incidents.”

IT Operations Director: “Who is responsible for this?”

CIO: “zzzzzzzz”

Who has the appropriate response? The CIO of course (and not just because she’s the boss)! The reality is that the metric means nothing at all. Which is kind of sad really, since there may actually be something to address.

Maybe the CIO will initiate some sort of action, but not until she hears a compelling story to accompany the metric. If the metric itself doesn’t tell the story, decisions will be made based on the most compelling anecdote, whether or not it is supported by the metric.

Metrics need to tell a story

At a new job around 15 years ago, I inherited a report that had both weekly (internal IT) and monthly (business leadership) versions. Since the report was already being run, I assumed it must be useful and used. The report consisted of the standard ITSM metrics:

  • number of calls opened last month vs. historical
  • incident response rate by team and priority
  • incident resolution rate by team and priority
  • highest volume of incidents by service
  • etc.

However after a few months I realized that nobody paid attention to these reports, which surprised me. According to ITIL these are all good metrics to pull. I saw useful things in the data, and even made some adjustments to support operations as a result. However, my adjustments were limited in scope, and the improvements I saw initially didn’t hold and so everyone simply went back to the “old ways”. The Help Desk team that reported to me did experience a sustained significant improvement in their first contact resolution rate, but all other areas of support saw nothing but modest improvements over time.

The fact is that the reports didn’t tell a compelling story. There were other factors as well, but looking back now I can see that the lack of a consistently compelling metrics story held us back from achieving the transformation for which we were looking.

So your metrics need to tell a story, but how?

The traditional ITSM approach to presenting data does a poor job at changing minds or driving action, and it can actually strengthen opposing perspectives. Can you think of an example where presenting numbers drove a significant decision? Most likely, the numbers had a narrative that was compelling to the decision maker. It could be something like, “our licensing spend will decrease by 25% over the next three years, and 10% every year after.” That would be a pretty compelling story for a CFO decision maker.

In my next article, we’ll look at how metrics can tell a compelling story.

Image credit

Why the CIO won't go the same way as the VP of Electricity

Dead as a...
CIO, Dead as a… ?

Commoditisation is, without doubt, a massive and revolutionary trend in IT. In just a handful of years, a huge range of industrialised, cost-effective solutions have created rapid change, so much so that some commentators now predict the end of the corporate IT department altogether.

Info-Tech Research Group’s June 2013 article highlights a comparison made by some, between today’s CIO, and the “VP of Electricity” role apparently ubiquitous in large organisations at the turn of the last century.

As electricity supply standardised and industrialised, the argument goes, the unsuspecting VP of Electricity (and presumably their whole department) found themselves anachronistic and redundant.

It’s a very flawed comparison. There can be no doubt that consumerisation is a reality, but business IT and electricity provision are very different things.

IT Service is not a light switch

Utility electricity is what it is: we may have a choice of billing suppliers, and perhaps several different options at the point of connection (such as smart metering), but the end user sees very little difference: it’s the same product from the same wires. I flip the light switch, and the light comes on. There is very little scope to vary the service, at the point of consumption, to gain significant competitive edge.

Hence, from the end-user’s point of view, electricity is an absolute service: it’s either there, or it isn’t (and usually, it’s there: Britain’s national grid, for instance, works to – and achieves – a 99.9999% reliability target. By comparison, the granddaddy of commoditised IT infrastructure, Amazon AWS, offers 99.95% – lower by a factor of 500).

By contrast, our customers experience IT as much more than a simple on/off service. It is far more complex, multi-faceted, and variable. We could frequently change our electricity supplier, and our customers and end-users would never be aware. However, change one of the many elements of IT with which they interact, though, and we can make a significant difference to their working day.

For many end-users, in fact, the “light switch” experience seems a long way off. A Forrester study of 900 end users and 900 IT professionals, in January 2013, found, for example, that 84% of business users experienced a severe or moderate impact on their ability to be productive on a monthly basis, as a direct result of IT issues. 14% experience difficulties at least once per day. The study also revealed that there’s a large gap between how the business thinks about IT and how IT thinks about itself. The difference varies regionally, between 13 to 16 percentage points.

If it’s done well, however, IT is an immensely powerful and proven asset to the business. MITSloan’s seminal 2007 study, “Avoiding the Alignment Trap in Information Technology“, identified a best-of-breed group of organisations delivering true “IT-enabled growth”. This group, representing the top 7% of their large sample, was found to be achieving an average of 35% compound annual growth rate, while spending 15% less than the study average on IT.

Commoditised tools

In our IT Service Management functions, commoditisation is just as much a reality as in any other facet of IT. As Gartner Research Director Jeff Brooks put it, “In the ITSM space, and more specifically the use of ITSM tools for IT Service Desk, we continue to see vendors provide commoditised tools”.  Does that mean, then, all of those tools, and the functions which use them, are as good as they will ever be? Clearly not: in January 2012, Brooks’s organisation observed that the average maturity level in IT Infrastructure and Operations was 2.35 on their 1-to-5 scale: a “disheartening conclusion”. As Brooks added: “Opportunities for differentiation exist, but the vendors have yet to capitalise on those prospects”.

He’s right. Even as the service desk, or any other ITSM function, becomes ubiquitous enough to be considered by many to have become commoditised, our customers expectations evolve and change. The age of the smartphone has increased mobility, and put leading-edge, location-aware, always-on technologies into homes and pockets. New-look frontline services like Apple’s Genius Bar have created demand for new ways to get support. On-demand commodity services, though convenient, create new management challenges around costs, control, and alignment. We should never consider ourselves finished.

Growth is driven by effective alignment of technology and processes. Henry Ford did not change the car industry simply by switching to standardised services. He created differentiation by aligning great processes with great technology. Some of those technologies, such as his electricity supply, would likely have been the same commodity service consumed by all of Ford’s rivals, but he made the difference with the things he created around them.  Fundamentally, as long as technology and innovation can give one business an edge over another, the role of the technologist – including the CIO and their business unit – will be relevant.

After all, if all companies were to standardise on a single set of commoditised IT offerings today, by tomorrow some of them would have found significant advantages in breaking out of the pattern.

Image Credit