Welcome everyone on this frigid, frigid Wednesday in November. I'm William and I work as director of tech evangelism at iTentional. With me is Holly Holcomb, director of customer success. How are you doing on this fine Wednesday, Holly? Also frigid. We actually got snow this week in Atlanta, which, like, is very unusual for Atlanta this time of year. I'm used to seventy degree days, so it's way too cold. That is unusual. Yeah. We have, like, ice crystals floating down from the sky. It's cold. I had to let the car warm up this morning, all the things that I, yeah, didn't miss doing during spring and summer. So Yeah. So in kinda, I guess, let's get down to business. So in the first webinar in this series, we covered that the invisible issues that kind of kill automation programs before they kind of get off the ground. Then we did a follow-up. We did a second one where we dove into configuration consistency. And then the last one we did, we talked about actually building a program that enabled just diverse contributions across teams, you know, in these multi vendor environments that we see every day. And today we're really, we're bringing it all together by talking about how you actually measure and demonstrate the value of all this this work. And, you know, because at the end of the day, if you can't show the business value, if you can't demonstrate ROI, you're going to have a very hard time getting continued support, continued funding and, you know, just any sort of momentum for your program. So I think in the original blog that you you authored that kind of kicked off this whole series, you know, that inspired the whole the whole thing. You kind of argued that network automation ROI goes far beyond just cost savings. And I think that is such an important point. That's kind of what I wanted to kick off with today is, you know, why do you think so many organizations just kinda stop it? You know, hey. We saved Dex dollars, and they're never pushing anything any further. What does making ROI kind of a core practice really mean in in this automation program? Yeah. I think you nailed it on the head. I I think a lot of a lot of teams begin with cost savings as their initial metric when it comes to ROI for an orchestration program, but it is that's just one. Right? We've got a white paper on our website that's got a ton of different ideas for how you can measure the success of your team. And I think that when it comes to justifying budget and allocating a team's time or resources for infrastructure, sometimes that cost savings metric or productivity centric metric, which is like, it took x amount of time for two people to do it previously, and now it takes five minutes of somebody's time to do it with orchestration. That comes up because it's a it's a really common thought process when it comes to justifying automation. And if it makes sense for your organization, like, it's a it's a great one to try and, like, bake into your process, especially when it comes to prioritizing the work that your team is going to choose to do. In other organizations, there's, you know, metrics that make more sense, especially when the delivery of network services and delivery time to give maybe an application developer the infrastructure they need to build their application. Maybe that delivery time is a better metric that should be used to help prioritize and make sure that what you're building aligns with the business' needs. I think whatever the definition is that you choose, the bottom line is that it needs to be written down, and it has to be a mechanism, that you use to, I think, understand what it is that you're going to target first and how you're going to measure the value of that, long term for that body of work. So just as an example, maybe there's some degree of innovation as, like, a core value of your program or your organization, and your metric is all about how you're going to embed maybe, like, AI in a meaningful way within your automation orchestra orchestration program. Having some degree of metric that aligns with that driver within your program is so critical. Love that. Yeah. It's also important too, like, when you think about it. Sometimes sometimes funding is just there for tools and for platforms, and then you end up with a lot of them, and that funding is there until it's not. And then they start looking at that list of things that they're gonna cut funding for. And if you don't have all your t's crossed and your i's dotted, hey. Guess what? You're you're funding for network automation or some platform that you've bought or some tools that you bought, that might be the first thing to go. So it makes a lot of sense to, you know, not put the cart before the horse and think about these things, you know, as you start trying to offer meaningful value to the business. And especially when it comes to, like, having a, a metric that is, like, very specifically a data point, that aligns with what the organization is trying to accomplish. Like, it's, to your point, like, it it is very easy to decide you're going to decommission a tool or remove it from your ecosystem for cost cutting purposes, but it's much more compelling to point to data driven metrics that align with the business, to help justify the existence of a team or a program. I love that. That's great. So, William, I guess the next question that I've got for you is what have you seen that has worked in the industry? Like, what metrics are effective? What do you see as, like, you know, working with the people that you talk to on a day to day basis? This is a this is a hard question. And I think the question, like, this question in particular kinda gets lost in, like, smoke and mirrors a lot of the time. So one of the things I've seen just looking at the the culmination of like a lot of years is first, like this is finally starting to evolve in a meaningful way, which is really good. But also is that metrics, like, have to change over time, like, as your organization changes or your your organization matures. And I know saying that just seems like really obvious, like obviously, you know, but let me let me just expand a little. So like very early on in an automation journey, you're yeah. You're focused on, I would say, efficiency metrics. So things like how much, how much time did we save here or how many, fewer tickets did we open for this Or how many VLANs did we configure? Whatever. Insert whatever efficiency metric you want to. And those they're totally valid. They're they're great metrics, especially when you're starting out because you need to show the quick win, Like some sort of immediate value is the why you need, you know, dollars for a thing. But as you mature, that doesn't cut it forever. So what I've seen is like organizations really shift their their mentality, their mental model more towards enablement metrics. And this is really, really powerful because it's not just about doing, you know, the same old story. Okay. I did, you know, x, y, and z faster. It's about doing things that you couldn't do before, which is super awesome. So as an example, there's a lot of examples, I guess, you could insert here, but maybe before you had any sort of orchestration in place, like spinning up a new branch office location maybe took three weeks and, you know, maybe required pretty extensive coordination across maybe six different teams. I don't know. Just hypothetical. And now with automation and orchestration, you can do it in, two days with, you know, minimal, like, manual intervention. Now that's not that's not an efficiency metric. That's that's enablement right there. Like you've fundamentally changed what is possible for the business. And I think that's a really important evolution because, you know, efficiency metrics, they have a ceiling, right? So you can only save so much time or so much money, but enablement metrics like those can fundamentally change the trajectory of your organization and what your org can actually accomplish. So I'd say to folks out there, start with efficiency, of course, you know, because you have to. But always be thinking about how you evolve towards enablement as you mature because that's where the real transformation value, etcetera, etcetera, is kinda tucked away at. Oh, man. I have got a couple of a couple of customers that I that come to mind with that have, like, really leaned into the enablement side of things. Like, on the service provider side of the house, like, value that you get out of formalizing orchestration around, like, really important business process, like service activation, just, like, as an example, the ability to do it consistently according to an SLA that, you know, the data is captured and stored in a really consistent way, all of the doors that that opens up for that organization to understand better optimizations, whether they can hook into the process for additional business process, consistency of the data and your sources of truth every single time that can help, you know, boost an AI strategy. All of those things, are very much like enablement centric. You know? It's all the things that you can do now that you've got, a core orchestration in place that help evolve and transform the way that the business, functions. Yeah. I love that. William, you mentioned something a moment ago around customers like finding their north star and understanding what it is that's really going to drive the direction of their road map and what they target in terms of the work that they do. And we've got this visual that we oftentimes will use that kind of helps customers identify where their core value driver is in terms of, like, a north star and then some tactical metrics that align with those. So this is kind of a fun tool that we'll use sometimes to spitball ideas with our customers on what the value drivers are that are going to help decide where their road map lands in terms of measuring and prioritizing work and outcomes. I love it. Yeah. You have to have an overlay. You have to have some sort of best practice framework or something that you can start trying to fit yourself into, and I think this is a great representation of that. And something I don't know. It's kinda like when I think about just you don't do things just to do things. You know? If you if you're doing things just to do things, then, like, what's what's the point? How do you show the value? How do you show progress? How do you measure anything? Like, no one like, I don't do household chores just for the sake of doing household chores. Like, there's a specific outcome, there's a purpose. Kinda like if you think of, I'd say, you know, say like, okay, let me just paint a picture, a picture of chaos, if you will, Ollie. So, talking about the outlaws, I mean the in laws, you know, they're visiting for the weekend and in this hypothetical scenario, the the in laws are pretty much auditors. So they're coming to inspect. They're they're gonna be checking, like, behind the fridge, you know, looking in the kids' closets, like, what's going on here? You know? And they're they're gonna look at things you didn't even know existed or could be inspected. You have no clue. So suddenly, speed to completion prior to their arrival becomes the critical metric because you have an external forcing function that kinda has the ability to bring, you know, morale as we know it crashing to the ground around the house. So I guess what I'm trying to say is you don't measure things just to measure things. You don't measure chores the same way every time. You measure them in different ways, both based on, like, where you are in, like, this maturity cycle and what the circumstances are because there are very short term goals that you have to accomplish, but there's also the really, like, the long term strategy. And, you know, these metrics and these frameworks can change a little bit based on what it is that you're actually trying to accomplish. So just something to something to keep in mind there. Oh, I love that analogy. I feel like forevermore when I talk to customers about, like, the, percentage of, like, vulnerabilities on across their network, that I'm gonna, like, imagine an in law asking them, like, what is the percentage of compliance that your network has for known vulnerabilities? Yeah. I mean, the last thing you want is your mother-in-law, like, going down the stairs and stepping on a stack of Legos. Like, I did that not too long ago. Well, I do that frequently. So it's Yeah. Yeah. Hot wheels. Routine. That's what we have in our house. Legs. Yeah. These hard animals too. They have, like, these big hard animals now, and my daughter has, a thousand of them. And I stepped on those things so much. They're they're, like, the worst. They they don't budge. It's it's brutal. So I guess transitioning to another topic that is not always, top of mind when it comes to impact and measurable impact is culture. Tell me, William, do you have anything that comes to mind when you think about orchestration's impact or measurable outcomes of orchestration when it comes to an organization's culture? That's huge. Culture eats technology for breakfast, lunch, and dinner, so it's, you know, obviously really important. So just top of mind, I think what I've seen with organizations that really get this right, And I'm actually thinking about one in particular just popped into my head, who has hit this out of the park in the real world. And what they did is they built a culture basically around, you know, a bias for action. And let's unpack that. So what that means is, you know, they they set they set these guiding principles and, you know, they define their their big rock initiatives, and then they stuck with them. So they they didn't and they continue to not deviate from the strategy. Every time there's like a new shiny object or a new change in leadership, a new VP gets hired or the CIO changes or something, like they they stick with their strategy. And this is very important to facilitate measuring impact on culture and really measuring impact on a lot of things in any capacity, but especially culture. And, you know, that's really hard to do, by the way, you know, as everybody listening knows because there's always gonna be pressure to pivot. There's there's always gonna be pressure to chase the latest trend to, you know, to respond to the the loudest voice in the room, I guess. But the organizations that really succeed are the ones that say, hey, no. We we defined our strategy for a reason, and and we're gonna see it through. And this you know, the particular company I was just referring to, and I actually just talked to him not too long ago, and, you know, they handle a few things in very interesting ways that are just kinda novel when you think about how most big companies work. But they one of the great things they do is they treat innovation, kinda like the NFL draft. So they bring in interns or new team members who are completely disconnected from the the day to day processes. So these folks aren't they're not bound by, okay, this is how we've always done these things, you know, and they're really enabled to go and build. They're given a problem. And they're given limited guardrails and and a safe environment to work in, and they're empowered to go figure out the art of the possible for new and novel ways to solve industry problems. And what happens then is, like, leadership then comes in and extracts the the meaningful and and and reasonable changes from their ideas and their work and implements something, you know, somewhere probably in the middle, you know, not too radical to break everything, but not the status quo either. And I and I think that's such a really cool balance. And, you know, remember when you when when you create, you know, we always talk about tech debt, but cultural debt is huge, and that rent comes due. So you you have all the, you can have all the technical success on the planet. But if the culture isn't there, you're it's not gonna be sustainable. So great cultures build great teams, and you need teamwork to make the dream work. So I'd say find a way to measure engagement, contribution levels, and and sentiment. And, you know, if you see red flags, try and address them as early as possible, you know, because, again, culture compounds in both positive and and negative ways. That's such a good example of how, you embed kind of, like, an innovation metric inside of your program. I love that. I think the million dollar question at this stage is how I can embed a culture of cleanliness in my kids, to avoid those hot wheel landlines that we were talking about. Yeah. Or get them to consistently the funny thing with that is, like, just they'll hide things. Like, will find things. Like, we we miss my my kids have, like, no socks ever. Because when they put things away, they'll just stick them somewhere, so there's never there's never any socks. So let's talk about one of the most common challenges that I hear a lot from customers and other folks in the industry. I know you hear this all the time too, Holly, and it's around capturing data on the current baseline that you're actually gonna measure against. You know, I've been in so many conversations where a team, you know, wants to automate something And, you know, we talk and, you know, we're having a conversation. It's like, okay. So what's your baseline today? Like, how does the process like, how long does it currently take? How many people are involved? Just getting what's the error rate? Just any sort of information, any sort of data points. And a lot of times, it's like, oh, I don't have that data. We need to think through that and get a lot of people on a call. But, yeah, I'd love to get your thoughts on that, Holly. Yeah. I feel like we have this conversation a lot with our customers. And initially, it comes through the form of asking help on how to prioritize their backlog or their roadmap. And usually, the call progresses, and it turns into me asking lots of questions about what exists today, what data do you have access to today that can help you give that baseline information. For a lot of customers, it's going back and digging through their ITSM to figure out, you know, if especially in the in the case that we talked about at the top of our conversation where it's delivery time, how quickly they can deliver a network resource or service to their internal customers. A lot of times, it's going through the ITSM and trying to find a baseline. And and so it's really being specific about what you're trying to effect change on with your orchestration, you know, if it is delivery times, if it's just coverage of orchestration based on the types of requests that come through to your team. All of it is very much driven by, you know, understanding the type of data that's going to be most impactful to measure against. And actually taking the time to invest in that research of the baseline is so important because I think I feel I feel like we see this really frequently. Without that baseline, trying to make a case for how we're improving it or the impact of orchestration at the end of the life cycle, like the development life cycle, we release it, we see the results coming through. It's huge. You know? So you really wanna spend time to understand what your baseline is and how you're going to impact it with the orchestration. I love that. And if you don't know, like, hey. If you don't know where you started, how do you how do you measure anything? How are you going to demonstrate improvement in any capacity? I mean, you can't just say like, hey. Well, this used to be hard, and now it's easier. Like, that is not going to cut it when you're trying to justify getting budget or people or resources. So yeah. I mean, one thing I've learned is, you know, this is something that I think is really important is that you need to do some you know, like you were saying, you've gotta do the research. You have to put in the work up front. And I know we talked about it on the last webinar I think we recorded, but there's not really shortcuts to that. There is work that you have to put in upfront if you really wanna see this get off the ground. Very important. And one last thing is even and I think this is the most controversial sometimes, but a lot of times, you'll see some strategy, and it's like we have to automate every little thing no matter what it is. One hundred percent automation at all costs. And I mean, even that kind of decision, like, you know, one thing to start with is, hey, you know, you might have two different buckets of things. You might have, hey, these are things you should automate, and maybe these are things you probably shouldn't automate for various reasons. So even taking the tasks and putting them into the right buckets to categorize them because, I mean, here's the reality, not everything should be automated. You know, I work for, like, Tenshil, an automation company, but, you know, this is this should be standard knowledge. And, you know, when when the thing you're thinking about automating maybe isn't even measurable and you can't demonstrate meaningful change if you do automate it, you know, that's a great opportunity to pause and decide if this body of of work should even be prioritized because there's a lot of other stuff that probably has a, you know, higher priority. Oh, that's such a good point. I think a lot of folks, have great intuition. A lot of our customers have great intuition around the areas where they should focus their time and energy. And that intuition will sometimes yield, like, excellent results, but doing doing the baseline research and also, like, to your point, understanding what we should be targeting, what maybe we shouldn't be targeting, is, like, great a great aspect of that research. Yeah. One last thing. I don't wanna rant here, but this is something I learned the hard way is, like, I always underestimated how complicated infrastructure is and can be. So we get you know, a lot of times we're humans. We get tunnel vision. You're usually focused on a very specific piece of the overarching, like, ecosystem and and making sure you do your homework on, the larger effort, especially when you're going from like automation to true orchestration, that can yield incredible results that will help justify additional budget and resources in the future. So don't just look at this tiny little slice of the network that you manage. Look at your know, how that automation and orchestration fits into this larger picture because oftentimes, the ROI story gets way more compelling when you can connect, you know, connect a few dots across a few different technical domains. So we're starting to wrap up here, and I think it's worth taking a step back and asking, maybe what what does success even look like here? And I think the answer might be pretty straightforward. You want to build a sustainable, successful automation program, and you and you need to demonstrate the value with it, you have to take ROI seriously, not just once, not just like at the very beginning when you're getting funding or you get some time committed to it, but continuously through, the the whole entire process. And the the organizations that do this well, the ones that have data driven, you know, prioritization built into the process, they're gonna probably see continued investment and support. But, do you do you have anything for the audience, as we close, Holly? Yeah. I think, what we see most is that, most automation programs and orchestration programs stall when they can no longer effectively answer the question of why are we doing this. And the truth is it's not enough to say we just saved some time or cut some costs. Leaders want to know, did it move the business forward? When teams treat ROI as a discipline instead of a checkbox, they get, like, some really key benefits that you just mentioned. So, like, there's pieces of it, related to the ability to measure and justify expansion of your program. So easily mapping the outcomes of your program to business outcomes that all all stakeholders inside of the organization can understand, whether it's, a reduction in MTTR, whether it's faster onboarding or activation of customers, whether it's quicker application deployment, like we were talking about earlier, and getting access to those network resources faster, they all earn buy in beyond just the technical teams. So ensuring that the outcomes of your program align with business outcomes and those business outcomes are easily understandable to all of the peers and leaders within the organization is super, super important. Using the same types of metrics that help you prioritize things, like the things that are going to best align with those value drivers that we talked about previously, it's gonna help you filter out the noise. Like we talked about before, a lot of people have really great intuition, but there are times when we make mistakes. We try and focus on work that maybe shouldn't be automated, like you mentioned. Making sure that you've got some, like, hard data and metrics that will help you filter out the noise and really focus on high impact work that has measurable outcomes is super, super important. And we see that in our customers that have that very I think about hygiene when I think about this effort, but, like, they have the road maps that very clearly track business outcomes and have very specific metrics associated with why the work got prioritized. So those are some of the key attributes that I think we see that really make a big impact with customers when it comes to, like, how how this somewhat theoretical sounding activity comes into play with the actual tactical pieces of running a program. And the real answer is we're doing it because we're trying to justify expansion and growth and innovation within the program, and it's not necessarily just about tracking, you know, in the past. It's not about saying this is what we worked on, and this was, you know, why we prioritized it. It's it's really about making sure that moving forward, as you mentioned, like, when you're shifting what kinds of value drivers or metrics you're using within your program because they should transform with time, that they continually align with the direction of the overarching organization. Like, they should they should transform with time. They should represent the market, on your company, all of it. It it it's all it all interweaves, I guess, is what I'm trying to point out. So it's not a onetime activity like you mentioned. It's something that should define the overall leadership direction and road map of your of your, orchestration program. I love that. Or so if anybody wants to learn more, what can they go to? Do we have any resources at Ascendum? We do. Yeah. I mentioned it at the beginning of our conversation, but we've got a white paper that can get give you lots of examples of what, some great metric ideas are that are outside of the box of, you know, normal productivity or that type of thing that, oftentimes usually are, like, the seed that starts the automation program. But, there's lots of great ideas in that white paper, that is all about aligning automation to business value and identifying the metrics that are gonna get you there.