Coaching Wants Evaluation: Designing The ROI
The importance of learning assessment and training needs analysis in measuring L&D success
This article is part of a series that looks at the seven elements of needs analysis. The elements are: curriculum mapping, knowing / doing / believing, defining the target group, designing a learning brand, defining the return on investment, building the team and the scope of the project.
What you are reading right now is a fable. AshCom is a fictional company, but the learning challenges that Kathryn, AshCom’s CLO, and their team face are real and shared by learning teams across corporations, nonprofits, associations and educational institutions. We hope you can connect with the characters, their challenges, and the solutions they discover. Building and following a needs assessment system is critical to the success of the learning opportunities we create.
eBook release
How to Change Your Approach to Needs Assessment: A Story Around the 7 Essential Elements
How does the fictional company AshCom adapt to changes and develop innovative L&D solutions? Discover the seven crucial components of a successful needs analysis.
Analysis of Training Needs and ROI: The Last Minute Meeting
Monday was hectic. While Kathryn was preparing for a day of back-to-back meetings, a text message interrupted her thoughts. It was Amy.
We need to talk. Do you have an hour today to discuss the acquisition?
Amy didn’t usually do anything like that. Kathryn’s day was already planned, but she didn’t want to keep Amy waiting. Something important has to be on her mind.
Kathryn texted her.
I can squeeze into a work lunch. I’ll have salads delivered. Sounds good?
Amy replied immediately.
That would be great. See you.
Amy wasn’t technically an AshCom employee, but she was a trusted member of the L&D team. As Chief Learning Officer, Kathryn hired her as an ongoing consultant who was committed to 15 to 20 hours a week on a two-year contract that would expire in three months. Kathryn had every intention of renewing it.
Amy had other customers. She has worked with some of the largest corporations in Minneapolis and across the country. She also had five years of experience as a learning leader and instruction designer at a pharmaceutical company in Kalamazoo, Michigan, and advised on learning, training and development projects in a variety of industries for the next 18 years. Amy knew the trends and had several touchpoints to evaluate what other companies were doing. If necessary, she also jumped into teaching design work.
Amy arrived punctually at 11:45 am with a burgundy folder in hand. She sat down and took out a copy of the EU’s 2020 Business Chief article: “Why do up to 90% of mergers and acquisitions fail?”
Game changing insights
The article was short, but Amy had taken the time to run a yellow highlighter over the first few paragraphs.
The first simply stated that:
According to pooled research and a recently published Harvard Business Review report, the failure rate for mergers and acquisitions (M&A) is between 70 and 90 percent.
The second read:
Mergers and acquisitions should never be taken lightly. Not only do they challenge two companies to integrate into a corporate mission, but they also bring together large groups of people with their own personalities, ambitions, behaviors, and ways of working.
Amy gave a little background. She had seen some acquisitions that worked, but she had also seen more bugs that were miserable for everyone involved. So many systems had to work well together to be successful, and learning teams were often critical to the outcome.
As Amy spoke, Kathryn felt her fear increase. When she thought of her study team and the progress they were making, she felt confident. When she thought about the big picture and what kind of mission it was that usually happened at night, she was restless and a little unsettled.
Part of Amy’s value that went beyond her expertise was that she really cared about Kathryn, the learning team, and learners at AshCom (including those who would join them from Globex). Amy felt responsible as part of the team for what she communicated by spending the weekend reading articles about mergers and acquisitions.
“That,” said Amy, “was the reason for the early morning text.” I’m sorry if I disrupted your normal morning routine, but I had to speak to you. “Kathryn was accurate in her assessment; That was important to Amy and that made it important to her.
“Okay,” said Kathryn with a smile, “you all got me down while I was trying to eat my lunch. I’ve worked with you for almost two years and believe I got to know your patterns. You are not the kind of person to raise a potential problem unless you also have thoughts of possible solutions. So calm my stomach and tell me what’s on your mind. “
Amy appreciated Kathryn’s directness and trust. “What I’m about to tell you isn’t just a problem when two companies merge or one acquires another,” said Amy. “I’ve seen this in almost every company where I’ve consulted with a few exceptions.”
Measure ROI
Amy, feeling a little encouraged, went to Kathryn’s whiteboard and wrote “ROI: Return on Investment”.
Amy stood while Kathryn finished her salad and said, “Demonstrating a solid return on investment could be one of the most important tasks for a corporate learning team. But it is often given somewhere between little and no attention, at least when learning needs are being analyzed and systems are being updated or built. This is not easy to do at the beginning and is therefore often ignored at the beginning of a large initiative. This can be a costly mistake as I’ve seen in other companies. AshCom’s takeover of Globex increases the stakes even more. “
Kathryn replied, “Amy, I really appreciate your expertise and care for our company. I also respect your ability to speak the unvarnished truth with just the right amount of grace. So, looking back at the projects you’ve been involved in, I have to ask what grade you would give us to help clarify, measure, and report our ROI to the owners of the company. “
“B-” Amy said as if she had anticipated the question and already thought about it. “Learning from AshCom has done enough ROI to be successful and to justify the owners with the budget we receive each year. But with the acquisition, the budget will be in line with expectations. I like where we are going. In fact, I can’t think of many learner leaders who have done better at familiarizing themselves with the company’s learning needs over the past few weeks than you have. “
“But …” Amy hesitated.
Kathryn finished, “But we’re not where we need to get ROI. Or to put it more positively: We are not where we could be. “
“Yeah,” said Amy. “That’s how I see it, and I think now is the time to ask the ROI question instead of waiting for someone else to ask and be unable to answer.”
“I assume you already did this at the pharmaceutical company when you were on this team and with many of the clients you have had over the years. If you could put this into a question – and I’m going to ask you to do just that on the whiteboard – what question would it be? “
Define success
Amy didn’t hesitate this time. She wrote with confidence: “What does success look like?” She underlined the word “success” a little unnecessarily.
Amy stated, “The same question applies on several levels. For every module or course we develop, we need a clear answer. The same applies to the entire series of courses. What I am asking has to do with the entire tutorial at AshCom. “
“We don’t start with nothing. We have the curriculum map which has helped us to relate our learning plan to AshCom’s business goals. We also have the three buckets KNOW / DO / BELIEVE. We gain a solid understanding of our learners even though we are still dealing with the Globex people who will be joining us. The planning so far will help us avoid problems, solve problems and create opportunities. “
Amy continued, “My question is, how are we going to measure these things? Or more precisely, which metrics will we measure against? We should also think about when to take our measurements. How we measure will also play a role. We have to do this for the learning we create, but we also need a system that will convert those individual course metrics into a larger metric so that we can demonstrate and quantify what we’ve done and what difference it makes in achieving the AshCom- Has made goals. How were everyone involved successful? “
Kathryn had finished her lunch. With only 10 minutes left in an hour, she asked Amy to at least work out her big ideas for tracking ROI. Amy guided Kathryn through the 5 levels of assessment, based on Kirkpatrick’s 4 levels, which considered both quantitative and qualitative metrics, to get a fuller picture of the impact on the learners and ultimately on AshCom.
Amy knew the time was up and that Kathryn had to move on to their next meeting. “I really appreciate that you took the time to spend an hour with me and listen to me. Later today I will be sending a table that will give you a picture of the 5 levels. It will become a lot clearer to you and the team when you share it with them. “
“Trust me,” replied Kathryn, “that will be the topic of our next team meeting in two days. Glad you spent the time thinking and reading. I also appreciate your willingness to push myself and be honest with myself. I look forward to seeing the picture. “
When Amy returned to her cabin, she forwarded the following resources to Kathryn …
Needs analysis: shaping the ROI – what is being achieved and how do you know that?
Conclusion
To read the rest of the chapters in this series on needs assessment and see how Kathryn and her team handle the other basics, please download the eBook. How to Change Your Approach to Needs Assessment: A Story About the 7 Essential Elements. You can also attend the webinar to analyze your training needs analysis and develop a culture of continuous improvement.
References
Why do up to 90% of mergers and acquisitions fail?
Inno-Versity
Inno-Versity creates individual e-learning for some of the largest companies in the world. We remove complexity from critical e-learning projects. We are an in-house team of talented and experienced instruction designers, artists and learning experts.
