Steps to Implement AI in L&D

[00:00:08] - [Speaker 0]
Doctor. Ashwin Mata. Welcome to the podcast.

[00:00:12] - [Speaker 1]
Hi, Nolan. Thank you for having me.

[00:00:15] - [Speaker 0]
Of course. And it's a pleasure to see you again virtually. I know we've connected in person at Learn Tech Events in London, Brandon Hall. Always good to reconnect with you and have you on, and especially talking about a topic that I know you have definitely made a name for yourself in the L and D industry and the training and HR industry as well on the topic of AI. Really, what we wanna pick your brain on today is how do we get started with AI?

[00:00:46] - [Speaker 0]
You know, and a lot of the conversations that I'm having with other CLOs and learning leaders is there's a lot of excitement and a lot of buzz around what we can do, and a lot of people have seen the you know, they've seen the tools, they've seen the strategies, they've seen the methodologies, but actually getting from zero to one, you know, getting it started seems to be a big stumbling block for a lot. So that's what we're really talking about today. But let us just start maybe, Ashwin, with a a state of the state. Like, where where do you see l and d now with AI? Where are we on the adoption cycle of AI in l and d?

[00:01:25] - [Speaker 1]
So it's an interesting question, and we have to sort of set the context first. Right? So what do we actually mean by AI in l and d? Now most of the LMD departments that I've worked with in the past, most of the companies that I've worked with in the past have had some kind of AI already working for them. And if we think about the platform space specifically, a lot of companies have got LMSs or LXPs.

[00:01:53] - [Speaker 1]
They might have recommender engines. They might have some kind of search capability. Maybe they have pathway creation. You know? Some of these things are without anybody really calling them out.

[00:02:06] - [Speaker 1]
These are AI functionalities that are part of the bread and butter cycle of L and D, which is usually content, platforms, and then some kind of data. So it's easy to be scared off by the terminology, but it depends on what we mean. Right? Mhmm. So if we first of all just put a pin in that.

[00:02:25] - [Speaker 1]
Right? Traditional AI use cases in technology platforms, software as a service, SaaS products, have been around for many years, and L and D have been using them for making, content accessible to their learners. So it's probably not a case where we're saying from scratch, nobody's ever used AI, and now suddenly we have to go and use AI. So it's important to ground our conversation in that. Probably what we're talking about is generative AI, which people have been talking about for the last two years, the whole ChatGPT launch, you know, Microsoft Copilot, you know, Gemini, all of these kinds of things, and how do we start to use some of the tools around that.

[00:03:10] - [Speaker 1]
Now if if we take what we what I just said, so content infrastructure data, very simplistic way to look at Mhmm. L and d. So L and D departments, either use suppliers or do it themselves that make some content. They put that content on a platform, and that learners access that content via that platform. When learners access content on a platform, that produces some data.

[00:03:36] - [Speaker 1]
So this is kind of the simplest way of looking at it. Yeah. Now if we look at it in that very simplistic way, we need to be thinking about, first of all, purpose. You know, we want to use generative AI. Why do we want to do that?

[00:03:51] - [Speaker 1]
Is there a reason that we get a get a, you know, enrich the learning experience? Are we going to make more content? Are we gonna reduce the cost base of L and D? Do we wanna shift headcount into something else like coaching, for example, and get the content production digital con content production side done by, by AI? So there needs to, first of all, be some kind of tangible reason for doing it because that then guides you into what are you going to do.

[00:04:19] - [Speaker 1]
If you just jump in to the I'm going to do this thing

[00:04:22] - [Speaker 0]
Yes.

[00:04:23] - [Speaker 1]
Without a purpose, then inevitably you're gonna get it wrong.

[00:04:28] - [Speaker 0]
Yeah. The So the the the, like, the the foundational question of everything and the the whole real like, I think the first five episodes of the podcast were just all around, like, the performance led questioning in L and D, and, like, why are we doing this? Let us not just do this to say that we did it. What is the actual performance metric that we're trying to improve? Yeah.

[00:04:52] - [Speaker 0]
And let's see if AI can help us get there. Absolutely.

[00:04:56] - [Speaker 1]
And it it gives a couple of interesting nuances to the discussion because L and D, generally speaking, has been incentivized to look at L and D metrics.

[00:05:06] - [Speaker 0]
So Yep.

[00:05:07] - [Speaker 1]
Batch throughput or continuous throughput. What are we looking at as as a factory, as a content factory? I produced 10 items last year. Now I want to produce a 100 items. Amazing.

[00:05:17] - [Speaker 1]
That's that's one way of thinking about metrics. But we've long had this ambition in L and B. We must tie learning strategy to business strategy. So that starts to also, you know, beg the question, well, what's the business doing with AI? What's the business doing with activities and tasks and roles, and which of these can be partly automated or can be partly served better by AI?

[00:05:44] - [Speaker 1]
And where do we need learning interventions that allow us to make digital media with AI? So very two very different conversations there. Mhmm. So that's why this purpose thing is so vitally important, because we need to figure out where are we going to deploy the best of the technology that we have available to us.

[00:06:04] - [Speaker 0]
So when you when you start fraction when we start going down these two routes, right, but we have this one starting point. As we say, you know, whatever route we wanna go, we we need to have a purpose to do what anything that we do. Right? It's gonna seed our investment, seed our funding. And we've created two clear paths, and we've said, we've got these business metrics that we can entertain and help the business improve those.

[00:06:24] - [Speaker 0]
Right? And maybe I'm gonna implement a GenAI bot. Maybe I'm gonna implement a sales agent into my sales workforce where I can ask questions and get real time answers and you know, create an actual sales pitch for my team, whatever it had be. That would improve things like average order value, time to sale, pipeline velocity, pipeline value, everything. Then you have the complete other side, which is, okay, How am I, you know, building better ships?

[00:06:50] - [Speaker 0]
How am I building widgets faster, better, cheaper, whatever it is? There we have things like how quickly can we produce an hour of content or whatever it should be or an asset.

[00:06:59] - [Speaker 1]
Sure.

[00:07:00] - [Speaker 0]
Out of those two areas, Ashwin, which one have you seen to be an easier path?

[00:07:09] - [Speaker 1]
So it depends on with whom we are talking. If we're talking with l and d departments, the content production piece is way easier. Right. If we're talking with operations, then the other the other way is.

[00:07:21] - [Speaker 0]
Right. Course.

[00:07:21] - [Speaker 1]
Right? So the locus of discussion changes what the discussion's about. But the reason I mentioned that is we should be integrated with what the operational side of businesses are doing because ideally, we want to be supporting the skills and the throughput and all of the things that operations do. So it's important to be involved in that discussion or at least be aware of that discussion.

[00:07:49] - [Speaker 0]
Yeah. I think and I think, you know, one of the and this is a series. So for for those that are listening now, if, know, if you like this, this will be a a part of a series. I know one of the other things that we'll talk about is kind of the future of AI and how that sets your path. And I think that absolutely holds true is that before maybe even though you can't really boil the ocean and wanna look thirty years in advance, there is a lot of value before you even start.

[00:08:18] - [Speaker 0]
If we're talking about how do I go from zero to one in AI, just even spending a couple hours with some other executives, understanding where the business is headed and where the technology is headed. Although you may have no intention to get there in the next year, the next two years, it definitely is a good foundation of how you wanna structure which of these paths am I gonna go down, where should I seed investment today versus what are the things when I hear about them, it's kind of nice to know, but I'm not gonna tackle it right now. Because I I feel like in a lot of organizations, sometimes choosing what not to do becomes a harder conversation than what to do.

[00:08:56] - [Speaker 1]
Absolutely. And I think we'll touch a bit more on this when we talk about data in Mhmm. One of our other discussions. But yeah. So first of all, we have purpose.

[00:09:06] - [Speaker 1]
What are we actually trying to do? If we are trying to go down the content route, I know you guys have got a significant content studio capability. So we probably start to go down the road of which content or media development tools are we gonna start to use. And there can be quite a landscape of those tools. We talk I'll probably talk about that in two different parts.

[00:09:30] - [Speaker 1]
So part one is what are the tools we're gonna use? This tends to be the bit that everybody gets really excited about. You know?

[00:09:37] - [Speaker 0]
Is Right.

[00:09:38] - [Speaker 1]
Is my tool gonna get mentioned, or am I gonna be able to make video more effectively or whatever it is. Right? And the reality of the situation is there are lots of tools out there. There are benchmarks for different tools, and some of them are in the video production space. Some of them are in the image generation space.

[00:09:55] - [Speaker 1]
Some of them are in the avatar space. But if we take the very basic use case of an elearning developer, right, usually they use something like Articulate or

[00:10:08] - [Speaker 0]
Mhmm.

[00:10:10] - [Speaker 1]
Captivate or Lucidat or some authoring tool.

[00:10:13] - [Speaker 0]
Authoring tool.

[00:10:15] - [Speaker 1]
Now, I think it was last month maybe that Articulate has entered the AI race. So they've they've said, right. We've got an integration with OpenAI or whatever it is that they do. I haven't really looked into it in much detail, but they now have AI capability. Now a lot of the other tools that we generally tend to use in the video production or maybe the avatar space, so Theseus, Colossians, r one, all of these tools, they've had this kind of capability for quite a while where even even beyond in the animation space.

[00:10:49] - [Speaker 1]
Sure. They've had these capabilities where eve even though l and d departments were probably using these tools anyway, AI snuck in, and you can go in and create a course from basically not that much. Say, make me a course on x, and it will make the scene structure for you. And that's when you say, what's the easiest step to getting into using AI in your workflows? The tools that you're using probably already have some AI capability.

[00:11:19] - [Speaker 1]
Figure out which tools do and which tools don't, and make sure you're using them to the the most or the least of however you want to use them for your purpose, so relate it back to that purpose. Mhmm. So that's probably the easiest step. But the reason I said I'll split that into two parts of the discussion is that with great power comes great responsibility, etcetera. But with tools comes a little bit of governance.

[00:11:47] - [Speaker 1]
So we need to be thinking about in in an enterprise sense, most enterprises have got some kind of governance around how do we implement technology tools into our environment. And with that, it's helpful to understand the tools that you already have, now they have new functionality. Does this still meet your governance? Are there still things that, you know, are permissible within what was assessed in terms of risk? So are you releasing any of your data into the wild?

[00:12:17] - [Speaker 1]
Is everything still secure? You know, these kinds of considerations need to be made. And and that's that's the simplest step. So use what you're already using. It's probably got AI in it.

[00:12:29] - [Speaker 1]
Yeah. Now governance tools, if we start to think about now we're gonna we're gonna evolve. We're gonna look at a slightly different tool set. So moving outside of the elearning development world, you have a lot of tools that are, for example, the mid journeys of this world or things like Runway and Flux

[00:12:48] - [Speaker 0]
Right.

[00:12:49] - [Speaker 1]
Where you're starting to be able to manipulate images and manipulate create images and then create videos from images. So you you create a sense of movement. And Adobe made an announcement, I think, last week where they were looking at rotating two d images, and it does that. And Yeah. So they had they had, like, a a little guy that was fighting a dragon, and the sprite of the guy was facing the camera.

[00:13:18] - [Speaker 1]
And he said, we want to look at the dragon, so we'll just turn him around. And it just did it. Right? Oh, wow. So that's actually quite it's quite impressive.

[00:13:25] - [Speaker 1]
Right? Was just a cartoon figure, but doing that is quite impressive. So you start to get some of these other capabilities. And if we look at the the Photoshop side of things, so, you know, the Adobe Suite.

[00:13:37] - [Speaker 0]
Yeah.

[00:13:38] - [Speaker 1]
You know, editing, video editing, or image editing. There's a lot of generative capability in those things now. So we start to build the landscape, and I'm not particularly calling out tools for any reason other than just to say what we have now is we have a landscape of media generation tools, all of which have amazing capabilities compared to what they had two or three years ago, and they all have a little bit of additional governance that's needed around them. Now the reason for keep to keep mentioning governance is some enterprises that I've worked with have been fairly about. You know?

[00:14:17] - [Speaker 1]
You can just go and use tools, and we'll deal with it later. Use it use it for a pilot, and it's

[00:14:21] - [Speaker 0]
fine. Right.

[00:14:21] - [Speaker 1]
Some organizations and particularly regulated organizations are very strict around, you know, you cannot bring anything into the environment unless you've done lengthy and detailed governance around where data centers are and how data is processed and all of these kinds of things. So it depends on the risk appetite of your business as to, you know, which of these routes is gonna be applicable in your context.

[00:14:45] - [Speaker 0]
So, you know, and this is where I remember I was having a conversation with someone maybe a year ago, and I was showing them our tool just for generative AI, like a Copilot, but with a lot of learning elements added into it, to help with prompt engineering and to help create storyboards, video scripts, and things like that. And we were kind of showing the flow of, okay, first we use this generative AI content tool to actually tell us what to do. Right? We load in product manuals and everything like that. Essentially, we're trying to democratize this SME.

[00:15:23] - [Speaker 0]
Right? Everybody has this big problem of all this knowledge is locked in my SMEs. Well, with AI, you can take a 10,000 word document, find the answer within a second, and then go to your SME and say, hey. Just validate. Is this right or wrong?

[00:15:36] - [Speaker 0]
Versus tell me everything on this 10,000 page document, but I I I digress. So I was showing them our tool for that. And then I went to Colossian, and I said, now now you can take this and look at you can just create this avatar, then then then you can take that that that video and that avatar, you can convert it into any languages. And then you export it into SCORM, and and then you can create a different elearning on this tool. And and kinda the question was, what is your advice when it comes to using this tool for this and this tool for this and this tool for this versus a best in breed?

[00:16:09] - [Speaker 0]
And the question was, isn't Articulate just gonna come out with something that has all of it in house? And, you know, I think as as us that follow the tech space know that those are traditionally the laggards because they can't afford to get it wrong. Can't afford to launch AI and it blow up in their face or to give all you know, for a security breach, and now everything is gone. And so them them launching it this last month, I think a lot of people have it in their idea of, can I get away with just a tool? You know, have you and and and the tool space is so evolving.

[00:16:46] - [Speaker 0]
Are you are you a proponent of a best of breed where you get, you know, the best AI, the best video tool, the best content tool, the best Photoshop tool, the best avatar tool? Or are you more of the of, you know, let us do it all in one shop. Let's just let's let's be use Articulate, and let's do everything Articulate has. If it doesn't exist in Articulate, we are not gonna onboard it. Have you advised people in one way or the other or seen an approach that works better for one person or the next?

[00:17:18] - [Speaker 1]
There's not really a one size fits all, and the reason is that we have to go back to the purpose. Think about learning strategy. Right? Now some learning departments will have a very small group of learners, and they want very high quality experiences. Some learning departments will be serving hundreds of thousands of learners and want to focus on scale.

[00:17:42] - [Speaker 1]
So for for these kinds of reasons, we need to focus on the strategy of the business and what works for that business. Generally, I would say that I mean, we're still only talking about development. We've got a lot of other other things that, you know, consider as well. In in the development space, you know, I think all of the learning designers I know would be like, well, what about design? We've gotta think about design.

[00:18:06] - [Speaker 1]
Right? So we'll talk about that in a second. But in the development space, you could, and this is generally what I think, that you could say, well, what I need is one good image generator. I need one good video, creator. I need one shell that's capable of giving me, you know, first draft.

[00:18:28] - [Speaker 1]
Here's a bunch of scenes. And then, and maybe an avatar tool or maybe a couple of other things. But you say, well, these are my typical pedagogies. This is usually what I do in terms of design. I'm looking at scenario based or experiential learning or problem based learning or, you know, whatever it is that suits the appetite and suits the the mode of my business.

[00:18:49] - [Speaker 1]
Mhmm. These are the things that reach my learners because we're effectively in a competitive space. We're competing for attention.

[00:18:56] - [Speaker 0]
Correct.

[00:18:57] - [Speaker 1]
We're competing with work priorities, with other things that go on in people's lives. And generally, what we're trying to do is make a stimulus that reaches a person such that when they receive that stimulus, they somehow create knowledge in their minds, and they go off and do different things than than when they did before. And we can borrow a lot of things from the the field of marketing in this respect because marketing, they're reaching out to people, and they're trying to influence behavior through the use of media. That's basically what we're doing. Right?

[00:19:28] - [Speaker 0]
Mhmm.

[00:19:28] - [Speaker 1]
And mark in in the marketing space, you think about adverts, you know, they are competing with other products. We are competing for time. So if we're competing for time, then we need to be leveraging the best of our creative design capabilities with a a typical range of tools that those folks want to use. So depends on your business, but it also depends on your designers. Right?

[00:19:50] - [Speaker 1]
And that's why we then move into the design phase of things.

[00:19:56] - [Speaker 0]
Yeah. And and you know what I my advice to people, I mean, in general with tools, as well as somebody who's, you know, led the mark our marketing stack and tech stack and and and built an LMS and an LXP. I just, you know, I'm always of the advice that, know, find a software that solves a big pain point and does that really well. Mhmm. Don't don't worry about the edge cases.

[00:20:25] - [Speaker 0]
Don't worry about the things that you rarely use. Like, you go back to purpose. If you have a ton of avatars in your training mix in your library and you're using avatars and you've got a different languages, find a tool that allows you to shrink that production by 30%. That will gladly it'll easily pay for that tool. Don't worry about, well, does can Articulate also do it?

[00:20:50] - [Speaker 0]
Or can this tool also do it? I mean, if you're as you said, if your in house tools can, so be it. But find the tool that works for you today and has a little bit of ride in the future. I I think what happened to a lot of organizations that I've talked to in this past year, right, so as you said, we're kind of on a two year cycle now, and we're in the second year. As I started talking to people, you know, year one was a lot of, like, what is this?

[00:21:16] - [Speaker 0]
Skepticism, skepticism. Now a lot of this is more like, how do I get started skepticism versus what is this skepticism. And I think they, a lot of organizations punted because they were just afraid to choose the wrong tool or choose the wrong thing. When if they would have just started, even the inefficiencies of starting, even the inefficiencies of choosing the wrong tool or the wrong platform at this point would have been greatly outweighed by the results that they would have had if they would have just gone with one and said, even if we just use it for a year or six months, it pays for itself. So so we we you know covered that stuff.

[00:21:52] - [Speaker 0]
Now what's next? So we've we've talked about the development. You mentioned you know hey we have this whole other steps that come. What's the next one?

[00:21:59] - [Speaker 1]
Just one more point on the tool thing. Now if you do what you've said, and, you know, I remember, as as you can see, I do music. Right? So I remember back in the day when we used to have this Cubase versus Pro Tools debate. Right?

[00:22:19] - [Speaker 1]
So some people were using Pro Tools, and some people were using Cubase. And this was your digital audio workstation. And they worked quite differently. The interfaces were different. You know, where where are the buttons?

[00:22:32] - [Speaker 1]
How do I get it to do this or that or the other? How do I add channels? So all of the functionality was very different, and it was a different skill set to use one software versus another. Now, generally, this is something we would think about if we have a a sprawling software landscape. But nowadays, that's not really true because user interfaces and mechanisms have kind of converged, so most of the softwares look very, very similar.

[00:22:59] - [Speaker 1]
So the skill set required to pick up a new bit of software and then go and use it is much lower than it used to be. So that's one

[00:23:07] - [Speaker 0]
point to note. Absolutely. I softwares. Yeah. Very good point.

[00:23:11] - [Speaker 0]
I know that I use I I still get ridiculed for it. I use this very terrible graphic design tool called paint.net, and I do it because it's free, and I've been using it since college. It's not that my company won't pay for Adobe. I and I know the tools, but it was, you know, this software at this point is probably thirty, forty years old. Like you said, they divert.

[00:23:34] - [Speaker 0]
Like, it used to be that, yes, you either could do one or the other, and learning that had a huge adoption. We just became partners with Colossian. Zero training. I just went to the tool and started poking around, and I, you know, I was able to get it done within 30 minutes. So yeah.

[00:23:50] - [Speaker 0]
Yeah. Absolutely agree there.

[00:23:51] - [Speaker 1]
Very user friendly. Yeah. So let's pivot slightly to design. Right? Because we're still in the tech space at the moment.

[00:23:58] - [Speaker 1]
Now design is, again, something you guys do. So you do your analysis, design development, the usual kind of waterfall Adi. Now for folks who do that, the the thinking part of it, the, you mentioned SMEs earlier, so your experts, the people who will possibly want to work with you to provide expertise into some kind of learning intervention. Now those folks, usually, their time is, you know, precious. They have lots of other competing demands.

[00:24:31] - [Speaker 1]
So in a business, do we want them to be creating a first draft of something? Now what we said in the previous section was that a lot of the tools and technologies that are out there have some kind of capability of creating a shell, creating a first draft of a script, creating something that hopefully minimizes this button. But what if they don't? How do you get your designers to be working back and forth with some of the foundational models to be thinking about pedagogy, to be thinking about the first draft, to be thinking about objectives and satisfying those objectives, and all of the things that we, you know, hold near and dear as some of the the key points in that development journey, that starts with analysis and goes through design. These are content pieces.

[00:25:23] - [Speaker 1]
Right? And I'm by no means saying that, you know, content is the way forward, but I'm saying that for those who are still making content, then not only thinking about media development, but thinking about the entirety of the journey is, potentially a way forward to starting with AI. So has to the audience, has anybody tried using all of these tools? Have you used ChatGPT to produce yourself a design, an outline, a first draft of a script? Because this kind of thing is quite easy.

[00:25:56] - [Speaker 1]
You mentioned prompt engineering earlier. And with a little bit of experimentation and, dare I say it, maybe a little bit of, online discovery, look look on YouTube or Coursera or anything for prompt engineering courses. There are thousands of them out there. But you can figure out pretty quickly that you can start to streamline your process that goes from an expert or a business demand or something that you need as an intervention through to something that is, you know, almost the first draft. You can take that process and use a lot of AI to get there.

[00:26:34] - [Speaker 0]
Yeah. And I think so I I'm I'm implementing this internally at InfoPro right now in a couple of our kind of back office cycles. And I realized, you know, I kind of needed to take a little bit of my own advice is at first I was like, oh, well, you know, maybe I can just wait for, you know, the Copilot, you know, our corporate org to have Copilot and start loading our assets and make it our own. But then I realized, like, the ability for me to build out my own world in Copilot or my own world within whatever, you know, Claude, whatever Gen AI tool that you're allowed to use is not very complicated. And I realize that most of the options out there to get me from zero to 90% there or 80% there, what I mean, even if it gets you 50% there, that's a hell of a lot of time, is not a lot of effort.

[00:27:29] - [Speaker 0]
And it takes no code. Know, you don't know develop, you don't have to understand code, understand development at all. It takes some basic understanding of how the whole Gen AI process works. But but if you do, you can build yourself a working model that gets you really far really, really fast. What but for those that that might be scared to start, like, what are those areas that are scary for people that are thinking like, what is on the minds of people that aren't doing it you think that it might be roadblock for them to start down there?

[00:28:05] - [Speaker 1]
It's a whole variety of things, to be honest. So some some organizations early on when, you know, November 22, and we were all, able to play with chat GPT a little bit. Yeah. A lot of organizations, not a lot, but some organizations decided, no. Nobody can use this.

[00:28:26] - [Speaker 1]
We don't know enough about the risks. We don't know enough about the security, so no one can use it. And that was was for those organizations, I'm sure that was the right decision. But with that, then comes a little bit of fear in the worker population because messages don't always trickle through the organization in the same way. So we have this very strong mandate.

[00:28:49] - [Speaker 1]
Nobody touched this stuff. Right? And then we have maybe the trickle through of actually, we can use it for a little bit of this or a little bit of that. Those those messages have trickled through a lot less strongly than the mandate to say no, because mandates to say no tend to be Correct. Yeah.

[00:29:07] - [Speaker 1]
Very widely spread.

[00:29:08] - [Speaker 0]
Right? Yeah.

[00:29:09] - [Speaker 1]
So so you have a little bit of that, that kind of fear of, well, you you said no. You said don't touch the button, and now you're saying it's okay to touch the button. I don't know if the button's hot or not. Right? Yeah.

[00:29:20] - [Speaker 1]
Why would

[00:29:20] - [Speaker 0]
I touch this?

[00:29:22] - [Speaker 1]
So you got this natural human inclination to say, well, should I really do this? Yeah. Which is potentially healthy. Right? Because if we we get onto the we get onto the topic of AI skills a little bit there, because if your organization as a whole is AI literate, then you can have the robust and sometimes thorough discussion with people around you you can use AI in these use cases.

[00:29:48] - [Speaker 1]
We don't want you to use it in these other use cases. Maybe it's useful for x and maybe it's not useful for something else. Or maybe we want everybody to be experimenting. So whatever the organization is saying that it wants people to do, you that needs to be accompanied with a baseline of literacy such that people understand those messages and can then go and action them for competitive advantage in the business. So that's kind of one reason why some of the adoption has been a little bit low.

[00:30:20] - [Speaker 1]
Other businesses, of course, have been very quick to adopt and say, everybody go and play because Yeah. Yeah. It's going to it's going to, affect our position in the market, and we want to be the best or what whatever, you know, the position of the business is at the time. So we want everybody to play, go and invent and experiment. And that's, you know, also fine.

[00:30:42] - [Speaker 1]
But the skills points, I think, it it not only comes from this, this idea of, you know, let's say no and make sure that people don't play with it, but also, you know, media scaremongering and things. You know? So if people aren't, work they aren't working with AI, they're not using AI at work, they're not really using it at home either. You get a you get a lot of

[00:31:04] - [Speaker 0]
Yeah.

[00:31:05] - [Speaker 1]
A lot of noise, a lot of chatter, then maybe it becomes a thing that you don't wanna do. And I think the last point is around things like self confidence, self efficacy. Right? So AI seems this big scary thing. I don't know how to use it.

[00:31:21] - [Speaker 1]
Should I should I try to use it? Am I gonna need to learn coding to use it? Right? Yeah. So you've got all of these things as well.

[00:31:28] - [Speaker 1]
And the reality of the situation, as I said at the start of this conversation, is probably most of the things you're used to using have some AI in them anyway.

[00:31:37] - [Speaker 0]
Yeah. I mean, I you know, one of the things that caught to my mind was within our our LMS and our LXP. One of our our selling points, I mean, even four years, five years ago, we were we were using intelligent search to to give it better answers and implementing LLMs and things like that within the tool to to help create a better response. And that was a big, big selling point was and I think I don't think people really thought of it as, well, this is AI, but it really I mean, it was. It's not a script.

[00:32:14] - [Speaker 0]
It, you know, wasn't this you know, interacts. It outputs. Why? It was into a string of things and will gradually get better over time. So, yeah, I think a lot of that stigma is there and maybe speaking of that, maybe we're jumping too far back, but what have you seen as a good advice for maybe L and D organizations in particular that are kind of in that space, whether it's at the CLO level or maybe you're a CLO yourself and you sense that this is happening you know, in those layers below, what have you seen to as a as a good tool to break down those barriers a little bit?

[00:32:56] - [Speaker 1]
Well, the best the best tool is education. Right? So we're we're in the learning space. And Yeah. The the cobbler's children have the worst shoes.

[00:33:05] - [Speaker 1]
Right? Yeah. So Yeah. We need to be ensuring that the L and D organization is equipped with the skills and the awareness of what's happening. And the the luxury, let's face it, it is a luxury to be able to innovate and experiment with the different things that are out on the market.

[00:33:21] - [Speaker 1]
Mhmm. Because there's not really, there are plenty of courses out there. You can go and do a course, but if you go and experiment and you play and you figure out what is useful and what doesn't work for you, then you've learned probably quite a lot during just doing that research for yourself. Yeah. So I would gen generally encourage people to go and play.

[00:33:46] - [Speaker 1]
And that's not exclusive to the foundational models because a lot of the market tools have, you know, free trials and things like that which encourage you to play with them. So all of those things are

[00:33:57] - [Speaker 0]
That's how I built my first e learning course in AI is we had this I I don't know if you know of Slice Slice knowledge. I don't know if you've used those those gentlemen. But yeah, they had touted in a meeting of mine of, know, essentially tell it the course you want it to create, load some assets, and it will create an e learning course. And I thought, I don't know way it's gonna actually do that. I didn't believe it.

[00:34:23] - [Speaker 0]
So I went and did it, and I loaded it in. And what it did for me is one, it kind of showed me, as you mentioned, and I think it's a really important point, and I hope a lot of people take it is, well, Ashwin had said, you know, a lot of the technologies are similar now, so it it decreases that user adoption. But what it also does is even if you do play around and you go experiment with whatever, Clausius and Slice, whatever, The likelihood of even if you don't land on that tool, even if you go with other tool, it's gonna be similar. And you actually understand the fundamental layer of how these organizations are structuring the automation and the AI within this space. And so there is a ton of learning.

[00:35:06] - [Speaker 0]
And I remember that I gathered understanding, okay, what is the step by step process that they're gonna wanna take me down to build this course? And how much of it am I gonna build it all and then just start removing things? Am I gonna build it first and then add the images? Is it gonna add the images? So much of that you learn just by playing, and and and the act of playing actually is how you learn.

[00:35:31] - [Speaker 0]
So it's so connected. I think that's really good advice, you know, especially for those that that that are in kind of the development, production, delivery, design space. Just go get some free trial. Just about every SaaS product out there is going to have a free trial. And and and encourage them to to leverage one and come back and give you the pros and cons.

[00:35:54] - [Speaker 0]
That's actually that's what we did internally at InfoPro Learning when when this started back in November a couple years ago. We said, go play. Come back. And every week, somebody reported on a new tool and said, here's what I like. Here's what I don't like.

[00:36:06] - [Speaker 0]
Tremendous lessons there.

[00:36:09] - [Speaker 1]
Absolutely. And, obviously, if it's a free trial and you don't like the product, then don't forget to cancel it at

[00:36:14] - [Speaker 0]
the end. Right? Yeah.

[00:36:17] - [Speaker 1]
We're all guilty of that. So Yeah. That's that's effectively content development, content design, and working back and forth with not only market tools, but also working back and forth with foundational models, GPTs, copilots, and things like that. As we move into more complex space, we start to back to the purpose. Right?

[00:36:36] - [Speaker 1]
We start to think about slightly different purposes now. Because what we started this discussion with was how do we effectively how do we use AI to make content? We talked about platforms, platforms, content platforms, data. But if we move away from that paradigm for a second, right, so we make content, which is the stimulus for people to to go and learn something. And then, hopefully, they learn that and then go and apply it, and they practice and hone that skill.

[00:37:03] - [Speaker 1]
Yeah. If we move away from that paradigm for a second towards the probably more realistic paradigm now, that we've got Copilots. Right? We've got if you're a Microsoft house, you've got Copilot. If you're a Google house, probably you've got Gemini.

[00:37:18] - [Speaker 1]
And Chatuchiki and, you know, all of the others, Claude. Now in all of these models, you start to get a a tiering structure. Right? And that tiering structure is where you say, well, if I have Copilot, I can ask it questions. As an L and D person, you might wanna ask it to make content designs.

[00:37:40] - [Speaker 1]
But we start to jump to a state a state where, actually, your learners have access to Copilot. So if they want to know something, are they really going to stop what they're doing, go to an LMS, find an elearning course, sit through fifteen minutes of click next elearning. Right? I'm not saying it's gonna be that bad, but

[00:37:59] - [Speaker 0]
you Right.

[00:37:59] - [Speaker 1]
Yeah. You you get the idea. Yeah. Yeah. We're moving further and further away from the idea of accessing information in the flow of work.

[00:38:08] - [Speaker 1]
When we have the capability with Copilot and Gemini and all of these things to access that information. So there's probably a point at which l and d departments need to work with IT departments to figure out, well, how are we going to embed the skills, the basic literacy in terms of AI and data? Because you can't really, you can't really underplay the importance of data when it comes to AI. How are we gonna embed all of these skills in our organization such that people can start to be creative with whatever toolset we have embedded in the organization? Productivity tools.

[00:38:46] - [Speaker 1]
So that's a very different way to think about it, and it starts to move away from the idea of content. And if you're doing that, then you start to move into the territory of, we're going to use foundation models. We're going to use things like retrieval augmented generation for supplying a foundation model with our company data in a secure way and then allowing the responses to be way more contextual and way more accurate relating to our processes. Now everything I've just said. Now this starts to move from the idea that I can just use the tools that I have, and they've got a little bit of AI in them, first steps, to maybe second or third steps where you wanna be thinking about something that's a little bit more complicated.

[00:39:35] - [Speaker 1]
Now depending on the organization, you might have an implementation of something, OpenAI or Microsoft, that allows you to put documents in and get a response back so everybody can do that. You might want to be making some bespoke agents to support people with specific things, And this is where you start to get into this territory of, well, is this scary? Do I need support? Is this going to require some coding? Is it going to require x, y, and zed?

[00:40:07] - [Speaker 1]
Right? And you start to get into this. Now there's a barrier to entry. With that barrier to entry, of course, comes, you know, how do I do it? Do I need people in my organization?

[00:40:18] - [Speaker 1]
Do I need to liaise with IT? Do I need a supplier to do this? So this starts to be a little bit more difficult, and that's why I say this starts to be a step two discussion rather than a first steps discussion.

[00:40:30] - [Speaker 0]
Yeah. Absolutely. And I think that's you know, when when we started off and we said, let us understand the kind of the landscape of AI. I really do think that's part of even if you don't even if what we're talking about now and creating an an agent for people to to to answer questions, even if that seems five years out in advance, it's important to know that it exists today and know that that is the step two or the step three, what you know, wherever you might be, because that is the eventuality of it. I mean, there's a you know, I'm sure once Google was created and then subsequently Wikipedia, encyclopedia sales and library visits probably shrink.

[00:41:13] - [Speaker 0]
And now Mhmm. Know, now people go to libraries for completely different reasons. You know, we me and my family go probably once a week, but it's rarely because that's where the stuff is to get the answers that we're looking for. We go because we live in Northern Idaho. It's dark and cold at 4PM, and we gotta go somewhere with the kids to prevent them from tearing our house up.

[00:41:33] - [Speaker 0]
And it's a nice, peaceful place to go and and hang out and, you know, have them draw or whatever it is. So it's important to know that's where it is headed. And and as you said, I'm I'm so that's the one thing that I'm doing right now. I realized getting my agent to, like, a one dot o. Right?

[00:41:52] - [Speaker 0]
Load in my documents and get an answer. I can do that relatively easily. Anybody can honestly, you know, do that. But then how do I serve it to my people? How where does my agent live?

[00:42:06] - [Speaker 0]
Right? Do in my case, I'm you know, I I'd like it to live in Microsoft. I'd like the agent to be in Teams, which is something that a lot of organizations are implementing now within their their whatever their, you know, Slack, teams, whatever it is. And that becomes and I and I remember we had about a couple years back, we implemented a bot on top of our a chatbot on top of our LMS and our LXP. But then we realized that it was much more valuable if it wasn't just pulling in what was in the LMS but sources from everywhere else.

[00:42:44] - [Speaker 0]
But then that created this whole other problem of, well, if it's pulling in the information from everywhere, where is this bot best served? Where is it where should it actually sit? Because let's face it. It's very rare that when an employee has a question, they go to their LMS for an answer.

[00:43:03] - [Speaker 1]
Absolutely.

[00:43:04] - [Speaker 0]
They have 10,000 other places they go to.

[00:43:07] - [Speaker 1]
Well, this is this is going to be probably one of the the key considerations for the industry over the next, let's say, five years for arguments sake. That's what I said earlier, the existing paradigm, content infrastructure data, content platform data effectively, that paradigm actually goes away if we adopt a copilot model. Yeah. And I mean, copilot not in the Microsoft sense, but where humans work with AI as their supporting agent. So it's going to be something we will all need to consider.

[00:43:44] - [Speaker 1]
It's something the platform manufacturers are, of course, considering how do they, you know, augment their their offering to make it more more rounded. So, yes, it's a thing that we need to consider. Absolutely. Yeah. In the space of, where where does it live?

[00:44:01] - [Speaker 1]
Because that's something that you mentioned. Where does it live has got so many different answers. Right? Yeah. And it's got answers because it's the agents live.

[00:44:13] - [Speaker 1]
It could be where are you hosting it? How is it being accessed? So all of these things are slightly different ways to think about it. In the Microsoft sense, if you are using Microsoft, you're using Windows, you know, you can use Copilot through Edge. So, effectively, you have access within the tools that you're using.

[00:44:32] - [Speaker 1]
The wave two announcement from Microsoft, whenever that was two months ago or something, that had Copilot integration with most of the Office suites. So you had, you know, Word and PowerPoint and Excel and all of these things as, have Copilot functionality. So to some degree, they're they're ahead of the curve in solving some of those issues. If you're building your own, which is kind of what you were saying, if you're building your own, where does that live? That, again, depends on your organizational setup.

[00:45:06] - [Speaker 1]
Yeah. And it depends on how you're building it.

[00:45:08] - [Speaker 0]
And the interesting thing is, you know, it used to be you had to have it almost live or, like, you had to have it live in one place because the idea of it living in multiple places meant it was gonna be serving up, you know, it was gonna create this never ending, what would you call it? Like, you know, like string theory, right? It's gonna create so many different versions of itself, and you're never actually gonna know where it ends. But today with how connected everything is, you know, I guess the question more is like, well, where wouldn't you want it to live? And it should live everywhere.

[00:45:45] - [Speaker 0]
So is there one more component we wanna discuss, Ashut? Or should we kind of come back now and kind of re outline and summarize? Or is there another component you wanted to touch on?

[00:45:58] - [Speaker 1]
So the key components, strategy, purpose, make sure you're doing something purposeful. Right? Make sure you have the right governance in place. If you're using technology, make sure that you've surveyed the market and you know what's out there. Data is something we need to touch on, but we'll do that in another session.

[00:46:11] - [Speaker 1]
Yes. Because data is the fuel for the AI engine, and that's quite a big topic. And then we touched a little bit on the idea of AI and data literacy, and that that kind of culture of innovation that marries itself with skills that are required by the business. So I would say the the key factors to consider if you're going to be implementing AI are those. So strategy, governance, technology, data, culture, and skills.

[00:46:36] - [Speaker 1]
But how each of those manifests in an organization is ever so slightly different depending on, of course, things like business strategy, risk appetite, regulation, the underlying culture, and and the baseline of literacy in the organization. So it's difficult to say there's a one size fits all, but I would say everybody should be considering these things. And that's not only for AI implementation for L and D. It's also how the L and D department serve their organization and aligns a business strategy, and it's also for AI implementation company wide. So this is tech strategy now.

[00:47:12] - [Speaker 1]
So the factors will be the same. The considerations will be different.

[00:47:16] - [Speaker 0]
Yeah. And I think, you know, very early on in my marketing career, was fortunate enough to go out to dinner with the owner of our company. And he owns our company, which is attached to two or three others. So, you know, I'm essentially a speck in this organization at this point. And he's talking to me and he says, You know, Nolan, what is the value of a lead to you?

[00:47:45] - [Speaker 0]
What does it cost? You know, do do you know how much you spend on just a lead or the inverse of that actually and and how much money you generate from that said lead? He's like, if you don't know that, you can't diagnose any problems associated to that. It's impossible for you to actually determine what that that problem is or what the payoff is for solving that said problem. And I feel like so many companies, if they started with that phase zero of why am I doing this?

[00:48:22] - [Speaker 0]
If you put a dollar figure or whatever, financial figure or, you know, headcount, whatever it is attached to, what does this improvement in AI bring to my organization? You know, you you have the direction where the business is going and where they wanna go, but then if you take a financial analysis of what could this do to my organization, my department, whether that's learning, HR, marketing, sales, whatever it is, and you actually start putting dollars and cents behind the problem, I've always found that those problems become much more real and much more important for you to solve and for your organization to solve versus just, Oh yeah, if I got this done, I'd be able to create assets quicker. What does 30% faster mean? What is your dollar? Like, what is the dollar?

[00:49:13] - [Speaker 0]
What is the payoff at the end of the day? And I always I always feel like if you can really nail that, it actually helps facilitate and grease the wheels for everything that comes later.

[00:49:28] - [Speaker 1]
I I think I would make that slightly wider and say, if if we if we take this to the, you know, the L and D development side of things, which is what we've talked about, over the last forty, fifty minutes or so. If we take it there and we say, forget AI. Just let's let's not talk about AI for a second. Let's just talk about the principles of what we do. Now the principles of what we do, we are going to spend some money for an intervention, which means other people have to spend their time.

[00:49:58] - [Speaker 1]
Right? We spend money so people can spend their time. That's all cost. There's a cost associated with all of that. Now if we're going to spend company money, which all of this is Yep.

[00:50:10] - [Speaker 1]
Why are we doing it? What are we measuring? And do we routinely if let's say I'm making one elearning course, thirty minute course for argument's sake. A thousand people are gonna do a thirty minute course. Great.

[00:50:25] - [Speaker 1]
Right. We've spent five hundred hours now. Now what was the reason for doing that? The reason for doing that should be measured in some way. So are we doing AB testing or, you know, pre post testing or something that says, there was a business metric.

[00:50:39] - [Speaker 1]
There was something around the time it takes to do a particular operation, the number of sales calls we made in a particular month, what whatever it happens to be. Something that's operationally measurable should be different after we've spent this particular amount of time, and we should be able to measure it afterwards, pre and post. Or we have two groups, an intervention group and a non intervention group, and we compare them after the intervention. So we've got, like, an AB style test. Now this is I'm I'm describing this, and it's fairly basic stuff.

[00:51:13] - [Speaker 1]
And you're saying, yeah. Yeah. Yeah. Right? But how many of all of your clients you've got a lot of clients, right, in for pro learning.

[00:51:19] - [Speaker 1]
You go and make elearning for lots of clients. Every single bit of elearning that you've made should have a measurement behind it. How many of them do?

[00:51:29] - [Speaker 0]
Yeah. That that is the subject of, I think, I've done at least seven podcasts on just this one topic. When I started in in this industry fourteen years ago, I think, that was what actually I encourage our company to pin itself on. We have been able to do our tagline, learning for performance for a long time.

[00:51:51] - [Speaker 1]
Mhmm.

[00:51:53] - [Speaker 0]
You know what? Interestingly enough so we did a lot of things to try to step people there, right? So we created something called an outcome success plan, where we say, if we do x, we will do y, categorically, like at a program level, not at a thirty minute asset level, but at a program level, because we realized that was an easier starting point. And we still didn't quite I I don't know. It was still tough, I think, to go back and to get the data.

[00:52:23] - [Speaker 0]
You mentioned data. To get the data ahead of time, most of the conversations were were wanted to go like, well, we'll see what happens once it's done. And it's like, Well, we need to know where we are today, and we need to know where we want to go so that we can design the program to get it, versus design the program, and then afterwards take a look at a couple things and see what moved. You know, oddly enough, Ashwin, what moved the needle the most, which was a really interesting concept, is we started going to our clients and saying, listen. We think this program should win a Brandon Hall award.

[00:53:01] - [Speaker 0]
Because most companies that come to us are doing large thing. They're not like, hey. With thirty minute here, it's a large initiative they wanna do. And we said this should win a Brandon Hall award. But to do so, the number one component, or not the, it's one of the five, but it's the big they say, hey, this is the number.

[00:53:17] - [Speaker 0]
This is what we're gonna focus on is what is the impact? And so that little thing of, you know, we're gonna do this Brandon Hall award, but if we do it, we have to know the impact. So we gotta know what are we today and where are we going? Oddly enough, that has had the biggest impact on actually creating that incentive for a lot of our And I think it's because it I mean, it is a framework, it is a foundation. And I think there's still a little bit of, I don't know, fear is not the right word.

[00:53:51] - [Speaker 0]
I don't know what it is. Like, you kind of are sticking your neck out there when you go to sales and say, what metrics do you want me to improve with this course? Because you're naturally implying that they should get better once they're done.

[00:54:05] - [Speaker 1]
And Yeah.

[00:54:06] - [Speaker 0]
So you're sticking your neck out there a little bit.

[00:54:08] - [Speaker 1]
Well, the reason I mentioned it is it the the obvious question with AI is going to be, well, what are we trying to change? How do we measure that this has actually been successful when we've done it? And if we're not in the routine, if we're not actually in that routine of asking those questions and getting those answers and measuring things when we do regular non AI interventions, then this stuff isn't gonna get better once we start throwing additional tech at it.

[00:54:34] - [Speaker 0]
Yeah. Then you're the same company that spent a million dollars on AR headsets just because you wanted to see what it would do. I mean, that's never the biggest strategy. Well, what a great way to end an hour, you know, coming all the way back to the beginning, almost like a Christopher Nolan directed podcast. Thanks, Ashwin, for this series and those that like this.

[00:54:58] - [Speaker 0]
Again, this is a series. Please do look at our channel on Spotify and go to our social on LinkedIn. You'll see our other sessions posted here. Absolutely phenomenal content. Thank you again, doctor Ashwin Mehta, for joining us today.

[00:55:11] - [Speaker 0]
Look forward into the next one.

[00:55:14] - [Speaker 1]
Thank you, Nobel. Thanks for having me.