The Data Advantage: How AI Helps L&D Speak the Language of Business

[00:00:07] - [Speaker 0]
Welcome to the Learning and Development podcast sponsored by InfoPro Learning. As always, I'm your host, Nolan Hout. Joining me today, we have Matthew Ead, Senior Director of Learning and Development, where he's leading a team of over 20 learning professionals who are passionate about creating and delivering learning solutions that really drive business performance and employee engagement, kind of the two critical factors that I love to talk about. So happy to have him on. Today, we're going to be talking about how we can leverage AI and data to help L and D better understand the business, identify those performance outcomes, rapidly build content and measure outcomes.

[00:00:45] - [Speaker 0]
So as you can tell, we have a pretty full slate today. So with that, we'll go ahead and meet Matthew. Matthew, welcome to the podcast.

[00:00:52] - [Speaker 1]
Hi Nolan. Thanks so much for having me. I've been listening for a while and you've had some incredible thought leaders. So I feel very humbled for the invitation and I hope I can provide some value to your listeners today.

[00:01:04] - [Speaker 0]
Oh, I'm sure you were. I think anytime somebody is willing to come on and talk about their experience, especially in something like AI, people grasp onto that. I think there's so much information. Some people say misinformation, who knows? So any, any stories we have, successes, failures, whatever, happy to have you on.

[00:01:23] - [Speaker 0]
But before we get into that, Matthew, we want to learn a little bit more about you. I mean, you obviously have a pretty large team there. But you didn't start there. I doubt you graduated. They said, here's the keys to the kingdom.

[00:01:34] - [Speaker 0]
How did you get into this field of learning and development in the first place?

[00:01:39] - [Speaker 1]
Well, thanks for the question. I'm not much of a self reflector. I'm usually kind of forward thinking, but I know you've asked this question, so I had to give it some thought. I guess like many L and D professionals, I have a sort of a non traditional pathway into L and D. My career journey began when I was 14 back in Melbourne, Australia, when I was going through high school and I wanted a part time job.

[00:02:03] - [Speaker 1]
And I started working at Toys R Us, which unfortunately is no longer around, then but Joel Did asked

[00:02:10] - [Speaker 0]
you hear that they're trying to open up a couple stores for the holidays?

[00:02:14] - [Speaker 1]
I did not hear that.

[00:02:15] - [Speaker 0]
I didn't even read the article. This could be fake news, but I heard that they're talking about opening up a couple satellite stores just for Christmas shopping. So anyways, it might be something to check out if you live in a big city.

[00:02:30] - [Speaker 1]
Yeah, that can be surreal. It's been many decades since I've, you know, been in my Toys R Us store.

[00:02:36] - [Speaker 0]
Yeah. And my kids have never been in one. So if they open one, I'm I mean, the nostalgia of going into one and buying zebra gum, at least that's what I did, will make it hopefully pay off. So anyways, flashback, you're working at Toys R Us.

[00:02:50] - [Speaker 1]
Yeah. And, you know, I really I I came from a kind of dysfunctional family background and went to a really rough school. I know it was because I remember watching the news one day in in Melbourne, they and said, The best schools and the worst schools, and it was my school they had up there as the worst school. It's no longer around. But working part time through high school was my kind of escape and I really appreciated that.

[00:03:17] - [Speaker 1]
And then I worked really hard through high school to get into university, ended up at one of the most prestigious universities in the country. Being very independent at that time, financially and otherwise, That first year at university, kind of really struggled and realities kind of slapped me in the face. And I also had a bit of imposter syndrome. I kind of felt like I didn't belong because of my background and who I was rubbing shoulders with. So I was really appreciating learning and I felt like I was getting a real kind of broad education, a type of education I didn't feel I received in high school.

[00:04:00] - [Speaker 1]
And I really enjoyed it, you know, philosophy and humanities and developed this love of learning. But I had to make a decision and it was at the end of that first year in summer when I was then working at Target on the evenings and weekends, who offered me a management position. I thought, you know what, I'm going to take a step away from school, clear some debts, but what I didn't expect was that I would fall in love with it. I really enjoyed working in retail management, the fast paced nature of the business, being really connected to, you know, very clear performance outcomes. Either made the sale or you didn't either made budget that day or you didn't.

[00:04:40] - [Speaker 1]
And also the development that I received, you know, as a management employee at Target, you know, they had a pretty strong management and leadership development program. And I guess that's where I kind of really have a deep appreciation for those types of development programs today. And I do recall that was when I learned that I am an introvert. Know, I just always thought there was something wrong with me and actually it was through that program that I identified it's not a weakness, it's actually a strength. So I really kind of gained a deep appreciation.

[00:05:10] - [Speaker 1]
You know, nearly ten years at Target and I felt like something was missing, I wanted something more, I guess I fell out of love of the role. So, before I went back to school, I decided to travel the world, which is sort of a rite of passage for many Australians. Go live in another country and experience that. And so I moved to Canada and lived there for a couple of years and traveled around North America and South America, had some incredible experiences, met some wonderful people, learned a lot of skills that I still carry with me today, adaptability and resilience and cross cultural awareness, and really appreciate the lifelong lessons I learned by traveling the world. And then I decided to go back home, finish my studies.

[00:05:55] - [Speaker 1]
I was then majoring in media communications thinking I was going to get into journalism. I was interning with one of the biggest newspapers in the country and actually actively applying for a cadetship, the hiring manager said basically it's yours Matt, just give me a couple of weeks to figure it out. Then as you can imagine in the newspaper industry there was a decline and reductions in force they're like sorry, we can't go through that role, it doesn't exist. So it was at that moment in time when I had the opportunity to move to The US. I'd always wanted to live in The US and so this opportunity came up and I jumped at it.

[00:06:32] - [Speaker 1]
I moved to a new country, didn't know anyone, didn't have any connections. I was starting over again. And my first job was with this company called Avant, which was, you know, I start up in the FinTech industry of a couple of 100 employees and I joined them in the operations function. And within a few months, the operations manager offered me a position as a trainer. And I was like, well, I've never done that before.

[00:06:59] - [Speaker 1]
Why are you proactively offering me this job? And he saw how quickly I was learning things and applying it and I was coaching others on the job. And So I took the role and I kind of felt like for the first time I'd found my passion, which was training and development. And then was promoted pretty quickly and then helped the organization build out their training and development function from the ground up and supported their hyper growth from 200 to 2,000 employees within a couple of years. Know, built out a team, implemented programs and systems and LMS and developed people internally into roles like instructional designer and trainer.

[00:07:37] - [Speaker 1]
Then unfortunately there was some turmoil in the financing of the FinTech industry due to some bad actors and we had to cut some stuff and I took that as the hint to maybe find my next opportunity. And that's where I ended up at Empire and the hiring manager and executive at the time in the interview said, Look, Matt, this is basically like a 60 year old startup. And in many, many ways that is true. Was very fast paced, no bureaucracy. And again, I was able to build out the learning and development function from the ground up because they didn't have one.

[00:08:15] - [Speaker 1]
To where you called out a team of 20 today, training managers, OD managers, learning administration, instructional design. We're doing some phenomenal things. You know, when I look back, I think, you know, the career theme is operating in very fast paced, you know, lean environments where, you know, measurable performance really matters.

[00:08:36] - [Speaker 0]
Yeah. And I mean, so that it's interesting because you started, you know, you had such a, and what industry does Empire play in?

[00:08:45] - [Speaker 1]
Retail and home improvement, home services.

[00:08:48] - [Speaker 0]
Home improvement, right? Yeah. So you've kind of gone from a lot of, you know, or hardcore, you know, whatever they, forget what they call it, frontline retail, to then shifting to startup FinTech, which is, or finance, which is like, couldn't, I would think couldn't be more opposite from each other. And then back into kind of some, you know, retail position and a little bit along the way. So lots of different industry changes, but it was interesting that, that L and D kind of took a hold later in that.

[00:09:20] - [Speaker 0]
And then it seems like once you grasped that, it really just became, no, this is it. Like, this is where I want to be. This is what I want to do. What is it? Like, why do you think that happened?

[00:09:34] - [Speaker 0]
Why, why do you think it is what clicked? And you said, that's what, that's what I want to do. Is it like you, as a kid, like you mentioned, you know, you went to a bad school and maybe you thought, well, I don't want any other kid to go through this again. So I'm going to be the teacher, you know, of the, of the corporation. What do you think?

[00:09:53] - [Speaker 1]
Well, for me, I really like big challenges, right? So I love learning and I love big challenges. As an L D leader, as an L and D professional, you have to be a lifelong learner. You're thrown into different projects and different functional areas and different industries. I know many L and D professionals do cross industries and you have to learn very quickly on the job what matters and what's going make an impact.

[00:10:18] - [Speaker 1]
You have to be always learning. But I also love doing big things, whether it was pushing myself to be the exception in my high school to go to the best university in the country, or it was to sell everything I owned and travel and move to the other side of the planet twice, to building out L and D functions from the ground up, taking uncertainty and creating certainty. Recently people called me crazy because I underwent a gut rehab of a 120 year old home through the pandemic, and when we had labor shortages and inflation going through So, the you know, I don't typically take on, you know, small projects. So I think that's a through line. But also, there's a lot of my interest in there, interest in journalism, took up a lot of those skills of learning and educating and teaching others.

[00:11:12] - [Speaker 0]
Engaging people.

[00:11:13] - [Speaker 1]
Engaging, yep. Being really engaged in business problems and leading business issues, from frontline retail management to engaging with senior stakeholders, whether that be at my current organization or in the startup. So, you know, I think there's a through line there. But, you know, I think it's mostly being really attracted to taking on big challenges and building out functions and always learning.

[00:11:44] - [Speaker 0]
So one of the things that you mentioned was, you know, how you've built these teams in kind of agile and scalable ways. You know, one of the big ways to do that is AI. You know, while I want to get into the hows of it, I am curious to know, you know, if you were to start over again at your last or, you know, where you are today if you were to start over, talk to me a little bit about what you think that team might look like. Right. Cause obviously L and D on the org structure side is going through a massive change of what does AI mean for?

[00:12:22] - [Speaker 0]
What are the roles? What are the skills that we need? If you reflect back on how you would build that team up today, how, how similar do you think it would be? And I'm putting you on the spot. This is not what we talked about, but I just had this entry, this question, because I'm sure a lot of people are thinking, you budget cycles, it's timing right now.

[00:12:40] - [Speaker 0]
Trying to do their next level planning. How do you think that would change?

[00:12:45] - [Speaker 1]
Look, I think it would look very similar. You know, even pre AI there's been the conversation around L and D kind of being more focused on performance and less about learning attendance and learning completions and things like that. So the concept of a role of like a performance analyst or even kind of learning data analysts. And I have a data analyst on my team today who is overworked, let's say, to put it lightly. And that's because to make sense of what needs to be done and the impact that L and D teams can have, we really need to take a look at tonnes of data and make sense of it to make informed decisions.

[00:13:29] - [Speaker 1]
So, you know, I think there'd probably be a more heavy reliance on having talent on the team that is really great at that. But fortunately, Gen I is doing a lot of that work today for us. And, you know, I'm excited to talk a little bit more about that.

[00:13:45] - [Speaker 0]
Interesting. So, so let's get into it. I mean, the, the, the first thing you talked about was leveraging AI and data to help L and D better understand the business. I just want to kind of focus on that, that first component, because a lot of AI talk today, especially in L and D is more, I think, you know, what I've seen is more on like the creation of assets, the development of content, the leadership code, whatever it is. How are you, how can we leverage AI to better understand our business?

[00:14:15] - [Speaker 1]
Yeah. You know, that's really the foundational starting point, right? But before I jump into that, I just do want to do a quick disclaimer on responsible AI use, right? So I'm going talk about a lot of ways that I use it today and my team uses it. I just don't think it gets enough airtime talking about responsible AI use.

[00:14:33] - [Speaker 1]
It's still a bit of a Wild West out there with AI moving so rapidly. I think anyone who's not to stop people from using it, I just think it's really important on the front end for every individual to be really informed and knowledgeable on the risks of AI, you know, from ensuring, you know, data protection is considered, right? A lot of these, particularly, you know, freemium or freeware tools, nothing is free as they say, right? So when you put in your proprietary information into these tools, you need to question what are they doing with that? So, I strongly recommend anyone who's not considering those kind of responsible uses to consider that.

[00:15:17] - [Speaker 1]
There's also government regulation around, for example, in Illinois, you can't record someone without them knowing. So like AI note takers and things like that. But then also the accuracy of outputs. You know, I think this gets a bit more talk and focus, but it's really important as L and D professionals in particular that, you know, we are taking ownership of those outputs and we're putting time into ensuring the quality of the outputs as well. So, you know, I think it's really important, you know, for everyone to consider, you know, their role in ensuring the use of AI is protecting proprietary data.

[00:15:56] - [Speaker 1]
You know, if you work in a business that has customer information that you need to protect, you know, but also ensuring that the outputs are accurate. So I just wanted to put that quick disclaimer out there because I think it's really important. You know, we recently went through big review of what our AI policy should be, do's and don'ts, what you can use it for, what you cannot use it for. So I encourage everyone who has not taken that step with their organisations to engage with their legal teams, engage with their IT teams and make sure that everyone's on the same page about how AI should be used in their organization.

[00:16:30] - [Speaker 0]
Yeah. Good call out. And I mean, as a resource, you know, I know Matthew, you know, Matthew Eid would love to, you know, he'll, he'll have his links in the bio and stuff. I'm sure he'd be happy to share what he learned. I helped write our policy at InfoPro.

[00:16:45] - [Speaker 0]
I'd be happy to do that. I also have a good buddy who sits on the board at USA Today, who's in L and D who helped draft their policy. So like, there's a, if you kind of find yourself with, yeah, I agree, but I don't really know where to begin. Feel free to kind of reach out to some people who have maybe learned their lessons along the way. You know, the, the one thing that I, I touch on that, that was interesting is the accuracy component.

[00:17:11] - [Speaker 0]
And it's an interesting one because what I tell people is, you know, I don't know if you played, do you play golf?

[00:17:19] - [Speaker 1]
I don't. I'm sorry.

[00:17:20] - [Speaker 0]
Okay. So when you use your driver, right? That's the club that you want to hit the furthest thing, right? So you're hitting a tiny little ball and you're trying to hit it 300 yards, three football fields away. Obviously, if you make a little mistake at the beginning of that swing, it's going to end 300 yards one way or the other, right?

[00:17:40] - [Speaker 0]
It's the, the, the, the mistake you make is exasperated by the power of the club versus if I use a putter that's only designed to roll the ball. You know, maybe you're not a golfer, but you've probably played mini putt, you know, can't make that big of a mistake with a putter. I feel like AI is one of those things. It is your driver. It is your, one of your more powerful tools, which comes with this great responsibility of if you're inaccurate from the beginning, everything you do is going to expand that problem and expand that mistake even further.

[00:18:11] - [Speaker 0]
So, not again, obviously we're not, I'm not saying don't use it. I'm just saying it does become with it being so powerful accuracy. The importance of it can be magnified as you leverage that tool more and more.

[00:18:27] - [Speaker 1]
Yeah, absolutely. So, you know, I couldn't agree more. And so I just want to preface that. And that's something that we've done internally, but, you know, going back to your question, understanding the business, right, you know, we hear a lot in L and D about kind of moving from order taker to strategic business partner, right? And to do that effectively, you know, an L and D professional needs to speak the language of the business.

[00:18:53] - [Speaker 1]
You know, when you're in an L and D function who's supporting many different functions, that means being knowledgeable on many different domains needing to be familiar with nomenclature that is, you know, very broad, right? So, the way I think about this is threefold. Let's say it's about doing research, both internal and external research. So let's start with external research. Gen I, in the way I use it is when I'm engaging with different stakeholders, whether it's sales or it's warehousing and distributions, customer service, it's operations, it's HR, IT, marketing, right?

[00:19:32] - [Speaker 1]
I leverage that to kind of get some background on what's being discussed in thought leadership circles around this topic, what's important to these types of domains, just so I can get familiar with what's going on. So prepare myself for going into a conversation with that stakeholder to try to get a head start on what matters to them, right? And sure, we've had Google for a long time, but this speeds the process up incredibly. Absolutely. Even if you're on a pinch, you've got invited to a meeting last minute and you're like, oh, I'm not prepped for this.

[00:20:02] - [Speaker 1]
You can do some quick research and and kind of get the ball rolling so you can speak with some kind of credibility with different business leaders. You know, and then the the internal research, right? There's a lot of data in most organisations, particularly in the organisations, you know, I work at or have worked at. Pulling that in so you can try to understand it, to understand the business better is really critical. And, you know, an example of how this has come to light recently by using internal data when engaging with the business to understand the needs.

[00:20:37] - [Speaker 1]
Senior leader came to me and said, you know, we've noticed in some geographies that, you know, sales performance of a particular product category has weakened. We don't quite understand why we want to deliver some training. This probably sounds familiar, right? And I said, that seems strange because why is it only happening to these markets? Let's try to figure out what's going So what I did, I used AI and I was able to bring in performance data, training data, but also some HRIS data.

[00:21:09] - [Speaker 1]
Data like the individuals who are in the roles in those areas, how long have they been with the business? And what I was able to find in that example by bringing that internal data and doing some research, was that those markets were actually impacted by some higher turnover recently amongst the sales workforce, which meant those markets had a higher percentage of newer consultants overall compared to other markets. And we know, we've known for a long time, there is a ramp to productivity. So if you have an overly distributed kind of amount of consultants who are fairly new, of course the performance overall is going come down. What typically helps folks when they're new is coaching and reinforcement and support and ride alongs in the example of sales where I work at.

[00:21:53] - [Speaker 1]
So we were able to rather than just say everyone needs to be retrained, it was let's identify those who are struggling, give them some additional coaching and feedback, support the local managers in those areas because they had more new hires than a manager would typically have at one point in time. And we actually found that moved the needle. So, you know, that was about kind of understanding the business challenge, bringing in AI and various data points to help find a solution that was going to better move the needle than the typical let's go deliver trainings.

[00:22:27] - [Speaker 0]
Yeah. So to kind of, you know, bundle that it's like we went from, Oh, come in and deliver, you know, closing or negotiation skills, right? Like, Oh, let's pay consultant. Let's read, never split the difference. Let's whatever, bring that guy into talk versus saying, no, we just need like better onboarding or management training.

[00:22:49] - [Speaker 0]
Like it's such a stark contrast between what the business asks for versus what they needed. And your team was that, that cat, that conduit to say, yeah, this is actually where the performance metrics matter. And we talk, I did the last podcast I talked about, you know, how do you kind of have these conversations with the business when they come to you with one thing and your responses, that doesn't sound like a training problem. I really like how you have said, well, that doesn't sound like a training problem, but let me tell you what I think the problem is. Like it may be a training problem, but it seems like this might be the actual problem.

[00:23:25] - [Speaker 0]
So I think, and I think that's really where L and D can be the star of the show is, is kind of them digging and doing their own research. Because now that you can and pull that thread to get to the crux of the issue and provide solutions that that will work. A question on that though, you know, we're talking to, you know, starting with the data element, how do you ensure that the data you're using and you're plugging in is accurate, Greg? So you're pulling from sources that are, especially in L and D, a lot of times you're probably pulling from sources that aren't yours, right? They're not.

[00:24:01] - [Speaker 0]
So I guess, are you just like trusting the data is accurate? Do you rely on the business having cleaned it and cleansed it? Or what is your, what is your traditional approach when you're trying to leverage this data that isn't necessarily L and D owned?

[00:24:17] - [Speaker 1]
Yeah, you know, that's a really great question. And, you know, even just, let's say, for example, we have different KPIs and depending on who you ask in the business, whether it's marketing or sales or finance, their definition of that KPI or how they calculate that KPI can vary broadly. There's sort of a trust but verify approach, and before making any meaningful decisions or taking any considerable action, there is a review process to comb into the data, consult with different business leaders to make sure the findings and the insights we've gathered from AI kind of make sense to them based on the data we've pulled. You're right, sometimes AI does hallucinate, right? Was a recent example of, I was using it to summarize open ended comments in an engagement survey or training survey where we had hundreds of responses.

[00:25:10] - [Speaker 1]
And it started making up quotes, which didn't exist. I said, that's actually for our blue collar workforce. That actually sounds like a really well drafted comment. And so when I was searching for that comment, it didn't exist. So, you know, there is a word of caution there.

[00:25:24] - [Speaker 1]
And again, it can help steer you in the right direction, but you do need to follow-up, verify that we'll check that it's accurate.

[00:25:31] - [Speaker 0]
And that's, you know, I think that's a good, a really good mention is that especially when it's not your own data, right? I do think just the kind of lit, you know, gut check of checking it back to the business and, Hey, this is what I saw, you know, does this seem right? You know, does that, is this tracking with you? But then also I think you bring into the importance, like the understanding of, of the business. You have to have a baseline almost understanding of business to understand what AI has hallucinating.

[00:26:05] - [Speaker 0]
Right. Because you, and I was just talking to my, my wife about this and my brother, who's a teacher, and we were talking about how AI can or cannot detect cheating. Can it detect itself? And I was like, well, the nature of these AI tools is they don't want it to be able to detect itself. That's how they prove that they're worth, you know, they're like humans.

[00:26:30] - [Speaker 0]
And I said, so the interesting bit of it is about a teacher, if they, if they don't use an AI tool to determine if the tool is AI, they probably have a better understanding because they know the language of Nolan and they know the language of Matthew and they know the language of the people that they use. And so I throw that out because I think I've talked about this so many times, so I guess maybe I don't need to throw it out, but giving AI to people that don't know the business ahead of time is really tough. You're more likely to get these hallucinations because the AI tools present it with so much confidence. It'd be hard to tell it was not right.

[00:27:13] - [Speaker 1]
Yeah. And you know, there's all the tweaks that you can do, you know, behind the scenes and the tool that you use to try to get get it to provide you the answers in a format, you know, that you're seeking. You don't want it to be kind to you all the time and make up answers just so it can provide you an answer. So there's some obviously tweaking that you can do behind the scenes. But you're right, it's all contextual and you have to have a fairly solid understanding of your organisation and the context to identify if it tracks, you know, the outputs track.

[00:27:40] - [Speaker 1]
And, but again, it's a trust but verify approach for us at least.

[00:27:45] - [Speaker 0]
So we, you know, let's take this example that you had just mentioned, right? So you're using AI, you looked at the data, you put it together, You know, when you're using it to understand the business and then you use it to identify the performance outcomes, you know, what would the performance outcomes be? So we know what we're hoping to achieve. How are you then leveraging AI to do whatever it is that needs to be done?

[00:28:09] - [Speaker 1]
Yeah, you know, so this is kind of when I think about, you know, Kirkpatrick's levels of evaluation, right? We always want to get to level four, but unfortunately, all the time we're to get to level one with, you know, the completion and the smile sheets and whatnot. So, I think there's nothing new with what I'm about to say relative to AI, but it's about starting with the business outcomes in mind. Is it revenue and sales and closing? Is it productivity?

[00:28:35] - [Speaker 1]
Is it employee or HR metrics like engagement and turnover? So understand kind of what are you trying to influence and then work backwards from there. What AI has done for us internally is allows us to bring in all the different data sets that we have, whether it's job descriptions, it's learner feedback surveys, it's performance data, it's learning data to help us construct and what we've done is construct competency frameworks for all of the major job roles that we train and develop. While we've known for a long time, there are certain knowledge and skill sets that are required, by coming through that data, we've identified other skills that we hadn't before that became really important or have been really important, but we'd never identified them before. And AI has enabled us to do that.

[00:29:28] - [Speaker 1]
And it's enabled us to do it in a pretty incredibly fast way, but also a more thorough way. Because not only was it pulling in our internal data and there's some of the examples I shared, but it's also able to bring in external benchmarks. Right? Most organizations are not unique, right? If you're in a sales role or a customer service role or a warehouse role, there are some pretty common kind of knowledge, skills and abilities you require.

[00:29:54] - [Speaker 1]
AI can call on that and kind of blend together your organizational context with what it knows to be best practices around those types of roles to help determine kind of what is the roadmap for training? What is the knowledge that L and D should endeavor to kind of transfer? What are the skills that we should be focused on developing to influence, you know, the business outcomes that we're seeking to influence?

[00:30:17] - [Speaker 0]
And then you're taking that. So you have this competency framework. How have you been able, do you feel that AI is helping you get like to the bottom of it quicker? Like for you, is that the big takeaway, on the first half is like, I could probably get there. Like, do you think it's I can get there faster or is able to draw correlations that I would have never been able to determine, even if given an unlimited amount of time?

[00:30:50] - [Speaker 1]
You I think it's a bit of both. For us, working at a very fast paced operation, sometimes it takes time to do this type of research and analysis to kind of guide the direction. But sometimes it can just pull in insights that, you know, without kind of meaningful experience in that, you know, across industries that you wouldn't be able to generate. So I think it's a bit of both. But I think mostly it's about in some ways, mostly it's about speed, but in some ways it's also enabling a deeper and better quality of output as well, you know, with kind of more limited time.

[00:31:31] - [Speaker 0]
So, so talking about the, you know, the production end of it, how are you leveraging AI once you have all this defined? How are you using it to develop learning or in the flow of learning or as performance coaching? What are some of the things that you found to be, have the, you know, the most impact so far?

[00:31:47] - [Speaker 1]
Yeah, and I think this is one of the areas that gets the most talk and discussion. Obviously it's some of the most exciting things because it's taking, it's going beyond kind of text generation and doing some really exciting things. But, you know, for me working in startup environments or startup like environments that are very fast in their pace, you know, the concept of minimum viable product has always been pretty strong my And organisational it's about getting the content out there as quickly as possible. You know, not a lot of finesse and not a lot of kind of back and forth on the quality, right? It's just kind of eightytwenty, you know, minimum viable product, get it out.

[00:32:27] - [Speaker 1]
And, you know, for teams like mine, kind of lean organisations, it's sort of been a force multiplier. It's allowing us to continue to add the value we've been adding, but, know, give the work product finesse and a level of quality that we just don't typically have the time to do. Right. So, you know, we're using it in many ways from building on the competency frameworks to build out, you know, knowledge documentation, turning that into, so let's say e learning scripts, and even kind of now leveraging AI to build some of the e learning for us and get it to a certain kind of state or first draft that we kind of tweak. You know, we're using it to create voiceovers, human like voiceovers.

[00:33:12] - [Speaker 1]
And this is a really exciting part of AI, which just a couple of years ago, it wasn't quite there, but now it is. It's kind of wild when you listen to some of these voiceovers and we're that confident with it that we've started using that. You know, there are some technologies that are not quite there yet, at least I haven't looked at it in the last few months, so maybe it's gotten there that's going that quickly, but AI video avatars, I still have a bit of uncanny valley about them. So we are still keeping our eye on that, but we haven't explored that further at this point. But, you know, that's some of the ways that we've developed, you know, content for delivery.

[00:33:52] - [Speaker 1]
And, you know, there are a lot of other ways we're kind of building our content for assessments that I'll speak about as well. In terms of delivery, kind of that's what we're really focused on.

[00:34:02] - [Speaker 0]
Are you using like DeepL or Eleven Labs for your voiceover or a different tool?

[00:34:07] - [Speaker 1]
Eleven Labs. But Storyline and programs like that are now integrating kind of that type of technology, is pretty comparable to 11 labs and others.

[00:34:18] - [Speaker 0]
Yeah, I think it's going to follow that classic arc of tech, right? Where you don't have a, you know, where the best in classes are the worst in class for the first bit. You know, like Storyline would be a good example, right? Like 11 labs would be much better than what you would get, but then eventually those tools catch up either by buying them like Workday just bought Sana. You know, there's acquisition every other day from any other company of these AI tools.

[00:34:45] - [Speaker 0]
So I think as that space gets more, mature, you'll hopefully have these full stack providers have a lot better, you know, tools within their tools themselves. Tell people, and again, you know, I'm, by the time this is out, it's probably old data, but it's like, I I've yet to find that real, you know, magic bullet that people exist of, tell me what you want to have done and I'll create a fifteen minute training program for, right? Like end to end it's, it's some level of engaging. I just, I I've yet to find that exists. You know, I got to stitch together Claude to help me build a course outline or maybe NotebookLM.

[00:35:23] - [Speaker 0]
And then I got to use a visual tool, Adobe Firefly, whatever it is to create the visuals. Then I got to stitch together. Yeah. So it's not quite, I'm not quite there on the full surface one. And from video, you know, I think video images, I think it can do all those things, but from my experience, you have to be in a really advanced user to get something that isn't looking obviously AI, right?

[00:35:49] - [Speaker 0]
Like, so I think, you know, when you see these videos, they're always posted, right? The second that one gets created, it's all over LinkedIn of look, this is AI and it's so easy to use. Well, you go on there and you try it. And like, if you like, I went on there, I was like, okay, I'll buy the pro version of, I forgot what Google's video tools called Vio or something like that. Whatever, whatever it was, you know, it was the one where the avatars could sing and they could talk and they looked real.

[00:36:18] - [Speaker 0]
And they're like, actually wasn't that hard. It was like an hour of this. And I was like, no way, no way is anybody getting this done? So yeah, we're still probably far out on that. A question though about the content,

[00:36:29] - [Speaker 1]
because

[00:36:29] - [Speaker 0]
I think we're in this really, I think we're in like a weird spot now where we're still leveraging AI to create the same type of learning that we're doing. How far out or have you already started to see where you're leveraging AI to replace what you traditionally would have used like a storyline course for?

[00:36:51] - [Speaker 1]
You know, what I'm really excited about is organizations who are taking steps towards taking their, let's say, proprietary knowledge, right, and making it available as a performance support. Right? ChatGPT, you know, for your organization where it's got all your proprietary knowledge, it's got all your product knowledge, all of that in one place, and an employee who's on the spot and needs help, whether it's with knowledge or even skill development, you know, can tap into that chatbot. And, you know, as part of some of the ways that we're kind of measuring outcomes, you know, we've been doing standard role plays for a long time. And to create a really good solid role play, it does take time to think through, you know, the scenario and make sure it matches the performance outcomes and there's a good solid rubric.

[00:37:40] - [Speaker 1]
So, you know, AI is helping us do that much quicker. But what I'm really excited about is kind of AI role playing tools. Correct. You know, I've been using it personally with ChatGPT, you know, enabled in voice mode and if I need to have a difficult conversation with an employee, I'm on there, this is the situation, this is the context, I'm going to go, give me feedback, and it does that. You know, to take that concept and scale it across a larger organization, you know, we're in kind of late stage conversations right now with some vendors who we're kind of looking to kind of pilot this type of technology.

[00:38:15] - [Speaker 0]
Wonderful. You

[00:38:17] - [Speaker 1]
know, with almost a thousand sales consultants, know, and you know, the business model we have means there's fairly high turnover unfortunately, and so we're always bringing new consultants in and there are a lot of skills that need to develop pretty quickly, which means, you know, we have a ton of skills and a large audience that we need to scale that for. And it can be really challenging. A lot of the time they, you know, for many skills at least, they just need to figure it out on the job. But to kind of scale those types of role plays en masse is really exciting.

[00:38:47] - [Speaker 0]
Yeah. You know, we did this job for it was a SaaS company and they were trying to do role plays for like a demo and it was a pretty, pretty complex environment. And they're like, yeah, he's like, listen, you know, yeah, the role play is good, but it's not as good as a sales manager could do. And I said, you're a 100% right. Tell me how many times your sales managers do demos and role plays.

[00:39:20] - [Speaker 0]
And can you share the feedback from the last 100? And they're like, well, they just, that's just a matter of them not having time to do it. If we just gave them time and I'm like, is something going to change in the past hundred years of sales managers, managing salespeople? Are they now going to have time to manage salespeople? Like, are now they going to have time?

[00:39:40] - [Speaker 0]
And I'm so, I'm like, you know, they, and so this is a, you know, a progress over perfection for me. If you're using, if you're in a similar boat, you know, yeah, you want to find the best that you can, course. And there's a level where it's just, it's too subpar, it's not helping anybody. But I do think, you know, eventually, because I think some pushback will come from the business of this isn't great. Then you can say, well, listen, if your team's willing to commit to do it, then I'm willing to not invest.

[00:40:10] - [Speaker 0]
But if your team's not willing to commit to do it, well, this tool is better than that. This tool is better than inconsistent feedback. This tool is better than a bad manager giving a good employee feedback. So there's, you know, I like to say sometimes AI, the sword seems to cut one way, but it really can cut both. If you look at, well, like how often do your humans lie to you?

[00:40:33] - [Speaker 0]
Not, not intentionally, like for the most part, I don't think employees are out there trying to lie. I don't think Claude is trying to lie to me or JETGBT is trying to lie to me. They just are by the nature of trying to please me. You can't tell me that our employees aren't doing the same thing. Right.

[00:40:48] - [Speaker 0]
They're trying to please us. Maybe they stretch the truth. Maybe they don't. So interesting one. I promised that we'd get out on time, but maybe really the way we can end, if you have the time, Matthew is talking about measurement of the outcome.

[00:41:04] - [Speaker 0]
So we've done this, we've implemented the training and we're doing all this, you know, you mentioned the sales scenarios. And in short words, how are you leveraging AI to then measure did all of this stuff I just did have an outcome on the business?

[00:41:18] - [Speaker 1]
Yeah. You know, I think this is another example of not necessarily AI doing something we couldn't do. I think it's about AI doing something we just have never had the time to do and do very well. Right. So, you know, AI is providing at least us in L and D to provide the business true tangible evidence of learning outcomes influencing business performance.

[00:41:45] - [Speaker 1]
I mentioned a little earlier about the certification or competency frameworks that we've built out. We've used the concept of certification for a while internally. We're get them certified on that product or certified on that skill. And it's usually a binary thing. You're either certified or you're not, right?

[00:42:01] - [Speaker 1]
We're taking that concept and kind of exploring it further with proficiency levels, right? Kind of think about Kirkpatrick's levels of evaluation as well, right? So we can get someone to a level one proficiency by doing an e learn, doing some knowledge transfer and a knowledge check. So Gen AI is helping us with creating those knowledge checks really quickly. Here's a 10 page script from the course, identify a 10 question knowledge check really quickly.

[00:42:30] - [Speaker 0]
Good point. Really good point.

[00:42:31] - [Speaker 1]
You know, a level two evaluation, which is what we call kind of a, you can do it in a safe learning environment, a training room, right? So building out kind of really robust and quality role playing scenarios with kind of very unique customer scenarios tied to our products, right? With this really strong rubric connected to the competency framework, right? Then providing managers on the job assessment tools as well to say, all right, they've done it in a safe learning environment. It's your job manager to use this one pager to verify they can go do it on the job.

[00:43:04] - [Speaker 1]
Right? And then ultimately kind of scaling up to five levels of proficiency, you know, three being proficient, they're demonstrating they're doing it on the job using these tools that AI is helping us create really quickly. You know, the level four is the business metric we're attempting to influence, right? So the idea being if we did our job accurately, if you're doing that skill consistently, right, on the job, you should be influencing that KPI. And then, you know, providing the managers the direct correlation between that skill and the KPI to influence whether they are now skilled or not, because they're now hitting their targets on their KPI.

[00:43:40] - [Speaker 1]
And then we reserve a fifth layer for those kind of experts, those who are so strong at that skill that they're coaching and teaching others, and we reward them and recognise them with that kind of expert level status. So, you know, that's some of the ways we're doing that. But, you know, we've been using a video role playing tool for a long time. We're excited to pivot from that to an AI role playing tool. But we've been using video role playing for a long time and, you know, AI is now allowing us to take all the data for thousands of people who've been using the tool over the years, right?

[00:44:10] - [Speaker 1]
And then looking at their performance and we've clear tangible connection to learning and performance outcomes. Those who have engaged into the role playing activities more than those who have not had a 2% greater margin in their sales closing than those who didn't. 2% may not sound like a lot, but when you're talking about millions of dollars, it's huge. So Gen AI is now enabling us to connect the learning data with the performance data.

[00:44:42] - [Speaker 0]
Yeah. Mean, that's an amazing application from very simple, like you said, build like out my knowledge chat, like, do you know, get the level one and level two out of the way? Like, you know, don't, I don't want to waste a second on it. Just get that done. Then let's focus on the tools, building the tools for level three and four.

[00:44:59] - [Speaker 0]
Really good. Well, Nathi, thank you so much for giving us some time to talk about real world application of AI that is just so needed. So I appreciate you investing your time. For those listening, you know, obviously Matthew's on LinkedIn, feel free to connect with him. You know, he's, he's, he's a beacon of knowledge for us.

[00:45:19] - [Speaker 0]
So thanks Matthew for stopping by and giving us some time.

[00:45:23] - [Speaker 1]
Thank you Nolan, for having me on. This has been a great conversation.

[00:45:27] - [Speaker 0]
Yeah, likewise. Catch you soon.

[00:45:29] - [Speaker 1]
Okay. Thanks Nolan.