Experience and Outcome Metrics with Barclay Rae

Pasi Nikkanen:

Welcome to HappyToday podcast. This is a podcast for those who want to improve service experience of internal services. If you use ServiceNow or other enterprise service management system, then this is for you. In this episode...

Pasi Nikkanen:

A culture moving away from SLA is not just that we change that to something, but it's totally something to think about the outcome of what IT is providing.

Pasi Nikkanen:

Hey, welcome to this episode of HappyToday podcast. My name is Pasi Nikkanen, and joined with me today is Sami Kallio our CEO and a special guest, Barclay Rae.

Pasi Nikkanen:

If you ever been to an ITSM event in UK, or basically elsewhere, you have most likely seen Barclay's presentation or name mentioned. He's been working in the ITSM for over 30 years with organizations like SDI, itSMF UK and recently redeveloping ITIL with AXELOS. Today, he uses mix of proven approaches, experiences, and common sense on guiding clients quickly to practical success. He's always saying that the focus is to help you deliver the best possible service to your customer.

Pasi Nikkanen:

Recently, Barclay wrote an article called Watertight Not Watermelon SLAs. Today, we are going to talk about that. You can find the link in the show notes. So please go and check that out.

Pasi Nikkanen:

But welcome Barclay, finally. Good to finally have you on our podcast.

Barclay Rae:

Great. Thank you. It's really great to be here and nice to speak to you both. We'll keep the discussion going around the watertight, not watermelon SLAs.

Pasi Nikkanen:

So watermelon, something that we use and maybe you see it somewhere behind me in the video as well. What's your definition of watermelon SLA, and how did you come up with it?

Barclay Rae:

It's something that just evolved a number of years ago that I started using, where it was a good way of just getting the point across, that most people that I came across, IT organizations, they had something that they called an SLA. It probably was more like an SLD, a service level disagreement or non-agreement. It wasn't based on any real discussion, but it was something that they had as their targets, and generally, misguidedly thought that if they achieved what was in that SLA, everything was great.

Barclay Rae:

So their reporting, their RAG reporting was all green. So the customers were happy, of course. But in fact, they weren't most of the time, because it wasn't actually reflecting the experience of what it was like to actually deal with that organization. So the watermelon, it looks like a green fruit, then you cut it open and it's actually red.

Barclay Rae:

That analogy has worked very well for a number of years. I know that many people have picked up on it, including you guys. I think it's very successful at just pointing out to people that their SLAs and the reports and the metrics that they produce just maybe aren't as fully developed as they should be.

Barclay Rae:

The simple problem tends to be that they are focused on IT components and IT practices and IT processes and what IT does in lots of little things, rather than the whole bundle, the end-to-end value stream, as we now call it in ITIL 4. But the whole... so whilst individual components might be working, an application might be running, a platform might be available, but there's other things impacting on that, that mean that the service and the experience is not good.

Barclay Rae:

We must then move on to thinking about how we measure that whole end-to-end process and experience, rather than just what they think, what IT thinks that they should be doing.

Sami Kallio:

Yeah. I fully agree from our side on that one, because SLAs are measuring the process rather than what it's like to use the services itself. But I think we could be talking about the traditional SLAs for this whole 15 minutes, for sure. But let's maybe move on to talk about how we and how you especially see the future. That's really interesting to see how you see the maturity of the industry and so on, even it comes to really measuring the outcomes of service management.

Pasi Nikkanen:

Yeah. I guess one of the trends in measuring the outcomes has now been that experience level agreement. So what's the first time you have heard about that or how you've seen that kind of topic maturing?

Barclay Rae:

Yeah. So, that's something that's come up in the last few years. I think it's actually referenced in ITIL 4 as a term. It's not gone into in great detail, but it's there.

Barclay Rae:

The idea being that you capture more around, than the performance metrics and you try and pull in experience. That could be customer experience, user experience, employee... the people who are actually working there as well. So I think these are all good things.

Barclay Rae:

The extent to which you can have an agreement on that is always going to be challenging. And so what I put in the article, I think whilst I concur with the idea of the experience level agreement, I think it's very useful. I think we also just need to make sure we go further to include the outcomes.

Barclay Rae:

I think that probably is the natural evolution of this, whatever we call it. We've talked about SLAs, we've talked about experience management. I'm talking about experience and outcome metrics. And that's the distinction for me is that I'm not too bothered about how much of this is an agreement. I think it's important that people engage with their clients and users. I think it's important that they go through that process.

Barclay Rae:

What they come out with at the end of it will vary significantly. I think what really matters is capturing a good, broad cross-section of feedback. Feedback being metrics, performance metrics, customer feedback, user feedback, employee feedback.

Barclay Rae:

The key one for me is the outcome one. Were we able to do the thing in our business that we need to do, was the technology able to help? Rather than fussing about saying that we've got 99.6 availability on a service or a platform, let's make sure that we've got 100% availability when it's needed. That's the outcome that needs to be clear.

Barclay Rae:

What is the work that we're supporting? Do we provide healthcare? Do we provide insurance or pensions? Do we provide retail? All those things are the things that the technology is doing, and it's building that element into it as well, that I was really starting to reference in the article.

Pasi Nikkanen:

Yeah. I think that the whole outcome topic is something that we have been talking with Sami, a long time, is that it really should be the culture moving away from SLAs, not just that we change that to something, but it's totally starting to think about the outcome of what IT is providing.

Pasi Nikkanen:

One of our customers said that he wants to start using this experience thing, because he wants to understand the output of IT, not just the input of IT. Also, communicate that to the business side of the... not just how much money are we spending on IT, which is the input of IT, but actually, what is IT providing for the business?

Barclay Rae:

Just to go back to ITIL 4, a lot of the stuff that's in ITIL 4, the lower level stuff hasn't changed that much. Incident management is what it is and it works. We've tried elevate the place, I suppose, of service management. So that it is seen as more strategic and it is based more around... and this thing of demand to value. That's the whole system, the whole service value system is based around going from demand, however that's captured, through some kind of washing machine, some kind of engine.

Barclay Rae:

Then at the end of that, we've got something that we've identified as being value. And value can be a whole number of things, and not necessarily just financial or cost, or cost reduction. But it could be, we've delivered what our business needs in a way that it needs it, quickly, or certainly with the quality or whatever. We've achieved certain goals. We've done that in a way that is equitable for all our people, and in a way that has built a good relationship with our customers. Value is more than just cost.

Barclay Rae:

So that whole thing about outcome has got to be clear, and that's the bit that we do need to still have the... or the process that leads to an agreement, put it that way. We get too bogged down in trying to write a document that looks like a contract, that doesn't really mean anything.

Barclay Rae:

Whereas actually, there's only maybe a couple of key things that we need to focus on, or that our customers want. When we've got that, if we've achieved that on a weekly basis, that's green, that's the kiwifruit, rather than the watermelon.

Pasi Nikkanen:

Yeah. You were also using this metaphor of the ship. So that's why the watertight, not just the watermelon SLA. So can you elaborate a little bit on that story of the metaphor?

Barclay Rae:

What metaphor, for what? Sorry, I didn't catch what you said.

Pasi Nikkanen:

Kind of having the ship, so it's nice if the food is fine, but...

Barclay Rae:

Yeah. So, the ship is sailing in a particular direction. There's lots of things involved in that. Not just people running the ship and the sailing and the safety, but food, and you have a doctor. There's lots of things that are required and maybe that are not completely direct. The cook and the doctor don't steer the ship, so they don't get you to the destination, but they help you along the way.

Barclay Rae:

All those things are elements. Making sure that we understand that the delivery of that product and that service, whatever it is, requires a number of different things to work together. As long as we understand where we're going and that we're doing it together, this whole idea of a value stream and being part of an end-to-end process, I think is absolutely key.

Barclay Rae:

It's something that's held us back, I think, for a long time, that we've had the kind of culture that said, we've done our bit, so everything's good. But actually, no, we all need to work together.

Barclay Rae:

You see it from a service desk. If you go into an organization, and they might be well set up, but they're not terribly well supported by other teams. And the other teams are going, well, that service desk stuff is... No, actually you have to contribute to that as well. We're all in the same boat. So you need to contribute to part of that as well. It's not just the frontline.

Pasi Nikkanen:

I guess Sami you have something you're going to say.

Sami Kallio:

Yeah, I was thinking that when I read that article about the ship, I was thinking the trip here from Helsinki to Tallinn, for example, and thinking about those ships going there. There are fast ships, bigger ships, helicopters.

Sami Kallio:

That kind of route, I was thinking that one ship for all those outcome needs that I have, I will have to get to Tallin in 30 minutes, or I would love to spend two days, have good food, company, whatever. That is my need of outcome.

Sami Kallio:

I was thinking that maybe it's cattle that would be even better in this metaphor. Do you want to travel from place A to B? You select the way or whatever to get there. And understand the business need would be that, you have to have a bit different service levels and ways of working with different needs.

Sami Kallio:

So if it's a critical thing, the challenge we have with our customers is that if we don't talk it very, very deeply with them, it's quite easy for them to start thinking about, okay, this is our Net Promoter Score, happiness score target, 75. There is no point to think about that. I make everything average because you don't need that.

Sami Kallio:

You need 100 for those which are business critical, from this perspective and those which are not, you have to do compromises. And that's starting to really understand the business outcomes. What is critical for them is something where we see experience being the easiest way to get that understanding from the business. Of course, you have to discuss. It's not the only metrics that you do, you discuss with the business to understand what they need.

Sami Kallio:

That was kind of my thinking about that.

Pasi Nikkanen:

How do you guys see that now? Since IT is also, role is to be more like innovator and help the business develop things. What about now, for example, during COVID and questioning the whole, do we actually need the ship? Why don't we just use this online thing and actually get that outcome done? We do have an old way of that. This is how we've always done it. We fly planes and we meet people face-to-face.

Pasi Nikkanen:

Do you think you could use this outcome and experience based metrics, to understand where should we actually innovate?

Barclay Rae:

I think as soon as we put up examples of metrics, I've heard this problem for some time. If I put up an example of what a agreement looks like, or what a metric looks like, people tend to look at that and go, well, that's what it should look like. So we'll make ours like that.

Barclay Rae:

And the whole point of this process, and I go back to whether, to what extent we need an agreement, well, we need something. We need something that clarifies what the expectations or the shared expectations are. It could just be a couple of lines. It could be a number. It could be whatever it is.

Barclay Rae:

But the process of getting there is probably more important, of engagement, of sharing information and knowledge, sitting down and listening. That's the other thing I've always said about IT. We tend to go into these kind of meetings and discussions in shouting mode or solution mode, and go, we can fix this.

Barclay Rae:

And actually, all you need to do is listen and let people talk. Listening is a great relationship building feature.

Sami Kallio:

Sorry, I have to tell a story. When we started this company, we did these group interviews with our customers, and we always took their lead in this area, in the room for three hours, without possibility to speak. So they had to listen to real stories from their end users, three hours without teaching how they really should work.

Sami Kallio:

They were really threat necking in the meeting, but they said after that, that was one of the best meetings they have ever been, because they were forced to listen. That was really interesting to do that, because I knew that what we heard from the input was something that was not allowed, how they worked, how they wanted to work.

Sami Kallio:

But then some of those practices that they used, started to be real practices in the company, when they learned that, okay, this is how they see, this is the real critical things. They do whatever they need to do to get them solved, even without IT's help. That started a different kind of process of understanding the end users.

Sami Kallio:

And I really recommend that kind of work. We are not doing that anymore. Sorry if somebody wants to use us for that, but we I don't have time for that, but it's a really great way of forcing people to learn.

Barclay Rae:

It is. From a consulting point of view, the way that I've always approached, if an organization engages me to do a piece of work and they won't let me talk to some customers and get some feedback, and then be able to present it back to them, I'm thinking, well, they're not really interested in doing this.

Barclay Rae:

I remember doing that years ago with a company where actually the CIO was part of the discussion, and I was briefing him beforehand. I said please don't talk, just listen. Don't go into solution mode, and it was going pretty well. And then at the end, it was the CIO that cracked and dived in about five minutes at the end of the interview. And I was like, no, just be quiet.

Barclay Rae:

But just going back to the point, the thing is that these things are not fixed. You need different levels of them. You need to recognize the diversity of expectation and without creating a monster of lots of... that's why it becomes difficult, because you ended up having lots of bits of paper and documentation.

Barclay Rae:

But actually, recognizing the difference and being fluid and being flexible about how you apply these will add so much value to whatever agreement or metric is produced. It's all about... listening is just, we've not been very good at that in IT, I don't think.

Pasi Nikkanen:

Getting back to the article, and what you just said. So you talk about this outcome and experience based metrics. I'd like an example, because now probably most of the listeners have never heard it before. What does that include? How does it differ maybe from an XLA type of idea?

Barclay Rae:

Well, from an SLA or an XLA, it's usually focusing on... and then you may have a number of different versions of this, depending on different departments, different customers, and so on, or on different stages even, different time period. But it's taking customer feedback, it's taking employee feedback, it's taking the performance metrics that you currently produce, all your incident management, telephony and stuff.

Barclay Rae:

And then most importantly, it's taking the business outcome, the moment of truth as well. So part of that listening session is to say, right, well, we understand what's really important to you, in your job, to get done. If it's a university, it might be to get the examinations produced or to have classrooms available on time. Or, if it's a bank, then it's availability and working hours or it's transactions that are processed correctly or whatever.

Barclay Rae:

But you only hear that by talking to the business or listening to the business, hearing what they say is important to them. And actually then you just track has that happened or not? Has that outcome been supported? Did you manage to process the exams? Were the classrooms available? Did these things...

Barclay Rae:

Rather than it being a abstract ability that nobody cares about, what we care about is this particular event happening and the IT guys understand that, they know that that's the important stuff, they make sure it happens for that. So combining those and usually using weighting, that's the bit that you would agree with the customer, which is the most important element.

Barclay Rae:

So it may be that the key metric, the moment of truth is 40% of the score and your existing performance metrics are 10, or it may not. It may be the other way around, it may be that customer satisfaction is the most important or employee satisfaction.

Barclay Rae:

That's the discussion that you can... I think it's quite useful to use that simple balanced score card approach to say, here's four things. And again, thinking of the ITIL 4 dimensions, we're not just thinking about it from one perspective, we're thinking about it from several. And then you say, so is your feedback more important than we achieve the business outcome? Or is the business outcome more important? That's the reality of it.

Pasi Nikkanen:

I like the name because it just says in planning list, outcome and experience. It's also the same things that we discussed and changed the culture towards that one. Oh, yeah, please.

Barclay Rae:

Sorry. And it's metrics, it's not just an agreement. You can build an agreement around it. Absolutely. But you can start using it to build up your metrics and your information, your data about the service. Sorry.

Pasi Nikkanen:

I totally agree, because it'd be different services than you might have, if it creates just one agreement that tries to cover all things, but for each service, there's a certain outcome that they are supposed to achieve. And that's more concrete for the IT people also to work towards.

Pasi Nikkanen:

Sami, anything you still want to ask?

Sami Kallio:

I think we've used quite a lot of time already. So it's time to start to wrap up. But I really as well love the term experience measurement, and this really I would say basis of the experience management. And for me, it's going back a bit to the agreements. I would think that in the future it would be much, much better if also in the agreement, it would be more about how the provider commits to use the experience data, rather than that this is exactly the target level. The target level set, only six months and then renews again. So it's the principle of really having the practices around how you use these metrics.

Pasi Nikkanen:

Yeah. Because it's like things happen like COVID, and then suddenly stuff happens. And if you try to just follow that, we agree on this level, but it's actually trying to meet at that moment, the needs of the people with the new experience that they are having.

Sami Kallio:

I'd have to say to that one, now doing this, I think you also wrote about that in the article, during the COVID time, the expectations has changed. On average, our customers or their end users are much more happier to the IT support that they were before COVID. I believe mainly that is because the expectation level has changed. So it has been really interesting to follow up what happens in this industry during this COVID time, and would have to say that the whole industry have done quite a great job, compared to what you would maybe expect in this kind of a change.

Pasi Nikkanen:

Maybe a last question, how do you say the maturity on this, currently? So how many of the companies that you consult with are really interested in this or how many are still like stuck with the old way of thinking?

Barclay Rae:

I've been doing stuff like this for really quite some time, in one form or other for over 10 years, really. I've still been doing it under the banner of trying to do SLAs well. I think a lot of the language of that has changed, and the focus was more around...

Barclay Rae:

If you look at things like the SDI standard, where we've had lots of metrics and we've had employee experience and customers. And so people are doing it, they're doing it a lot more. I think obviously, the maturity of their SLAs varies massively. I was working last week with an organization where they had SLAs that were effectively 15 years out of date, but they had found ways of managing different parts of that. They were doing customer experience measurement and they were doing some employee...

Barclay Rae:

They weren't bringing it all together. And I think part of this is just helping people to move further up the maturity field. Some of them are great at it. Some of them are still stuck on, no, we have to follow this response and fix times and availability. I must admit, I don't find so many of them now that are doing that and don't realize that they need to change, or they've just gone so far past it, they just go, well, we can't do SLAs.

Barclay Rae:

That's where it's changed as well, because the relationship has thrown up the fact that they... if you look at things like promise theory, which says what you deliver is what you are effectively committing to, that has changed the game for a lot of organizations as well. But I think on the other side of that, the COVID experience has said, well, they really got us out of a mess very effectively and very quickly.

Barclay Rae:

Since COVID, the organizations I have worked for have all been in that position, where the service desk and frontline operation has suddenly become much more appreciated, because people realize how much they actually do.

Pasi Nikkanen:

True, true. I think that's maybe a good way to end this thing. And if you want to hear Barclay talk, you could join the itSMF UK CX/SLA forum on September 15th. It's going to be a 10:00 a.m. UK time and naturally an online event. But also Sami is going to be there. So if you want to catch both Sami and Barclay in the same show, that's a good day to book in your calendar. I'll put the lin on the event, on the show notes.

Pasi Nikkanen:

If you're listening this after the 15th of September, then I'm sure that it's okay to reach out to you Barclay and have a discussion. So if people get interested, what's the best way of getting in touch with you?

Barclay Rae:

Well, you can find me on LinkedIn or Twitter under BarclayRae, or you can just visit my website, which is barclayrae.com dot.

Pasi Nikkanen:

Okay. Do you have some other interesting resources that people can use on the website?

Barclay Rae:

There's lots of free resources. I created this model a few years ago, Goodness, which has got steps. And the first few steps of that are very much related to this about engaging, listening to customers and building some definition of services.

Pasi Nikkanen:

We were watching it with Sami before we started the recording. I think that's maybe something that people should go and check out, that steps to the goodness.

Barclay Rae:

The Goodness step. It's still talked about, ITSM, but it's still there. It's still all open source. So people can go and people do go onto it every week and pick up some of the documents and stuff.

Pasi Nikkanen:

Hey, thank you so much. And as we usually end this, we'd like to say stay happy.

Barclay Rae:

Definitely stay happy. Thank you. Thanks very much, guys.

Pasi Nikkanen:

Thank you. Thank you. Bye-bye.