June 9, 2022
Focusing on knowledge retention, setting baselines and keeping track of quality of life metrics can help assess a program's efficacy, members of Protocol's Braintrust say.
Good afternoon! Workplace upskilling and reskilling programs have become a major focus in the evolving world of work, especially in a tight labor market. In today's edition, we asked the experts to let us in on the best ways to evaluate a company's training programs. Questions or comments? Send us a note at email@example.com
Chief technology officer at Skillsoft
In the areas of upskilling, pre- and post-assessments play a key role in concretely measuring the effectiveness of a program in delivering mutual employer-employee growth and workforce transformation. For example, let’s say a cohort of IT professionals need to upskill on DevOps. A training program must have three components: concrete, objective ways to measure DevOps skill levels before the program starts; individualized learning paths based on skill levels for efficient skill development; and objective measurements after certain milestones are completed. This periodic approach allows for progression of the cohort through its skilling journey in an absolute scale of proficiency. Traditional metrics of content consumption and utilization are activity-oriented rather than outcome-oriented: They do not correlate with business results.
Based on these assessment metrics, organizations should further layer in a level of business metrics. Several indicators of success include buoyed employee retention, a higher percentage of roles filled internally and increased velocity of new projects staffed. These metrics can directly correlate to a bottom-line financial metric, enabling a concrete ROI calculation for the upskilling program.
Knowledge retention and refresh is also an important area. Science shows knowledge is better retained by re-enforcement and practice. An upskilling program must have time-lapsed sampling measurements for learners to assess how they are retaining their knowledge. These can be delivered as flash cards or small case studies conveniently via email or a mobile application.
Learning platforms that combine all of the above techniques and measurements better deliver concrete business outcomes.
Chief enterprise officer at Coursera
To successfully generate and track ROI of upskilling efforts, it is imperative for businesses to first ensure L&D programs are structured the right way. Enterprises should start by carefully mapping their business goals to skills objectives, and identifying what skills are needed at the most macro level to achieve overall company goals. Take an objective look at your employee base to inventory existing capabilities, with a goal to gain a deeper understanding of where your teams are today relative to the skill proficiency goals you have identified. Next, assign specific job roles to the specific skill objectives. Then, build a training program around those roles and the skills needed to fill them today and in the future, while mapping to overall goals for innovation and growth. Additionally, think holistically about the long-term ROI of your L&D programs, measuring employee impacts on retention, morale and productivity. For example, we commissioned a customer study with IDC to look at L&D programs, revealing a 7.46x ROI over three years with specific benefits including:
- Higher employee retention — Employee attrition rates dropped 38%, producing $1.12 million in recruitment cost savings per year.
- Boosts in employee morale and mobility — Employees experience higher morale and feel more empowered, seeing a 20% increase in likelihood of promotion.
- Increased employee productivity — Productivity increased 25%, driving an additional $20,1000 in productivity value per user.
- More employees trained in less time — Companies can train 94% more employees using 40% fewer training resources.
CEO at Articulate Global, LLC
Upskilling employees should be part of your organizational DNA regardless of the economic environment because it drives employee engagement and thus productivity. Millennials in particular say that opportunities for development are critical to their job satisfaction. And if you’re responding to the economic slowdown by laying off workers, you’ll likely need to upskill those who remain so they can help fill the gap or you simply won’t be able to execute. But how do you know whether you have the right upskilling program in place?
Define the outcome you are trying to create with your upskilling program, then look at correlating KPIs to see if you’re moving the needle. If you want to reduce expensive employee turnover, compare retention rates for employees who have been upskilled with those who haven’t. You can also measure satisfaction of upskilled cohorts versus their peers using engagement surveys, and track promotion rates against your baseline.
If the economic downturn requires that you do more with less, then your upskilling program will need to drive productivity for your remaining employees. You can improve productivity of your teams by 14% by simply increasing their engagement. Again, look at your employee turnover rates, absenteeism and engagement scores for upskilled cohorts versus their peers. If your upskilling program is driving those metrics in the right direction, you’re on the right track.
CEO at Pluralsight
The most effective upskilling programs use an intentional, programmatic and data-driven approach to each stage of the team’s upskilling journey. The measurement stage is critical in every successful upskilling program. Organizations that effectively measure the progress of their team members are able to more closely align upskilling programs to company needs and then tailor the program to ensure those needs are met and the program is delivering ROI.
Assessments are an important first step of the measurement process, and should be a critical component of every upskilling program. Assessments can be an effective means to allow organizations to identify and close technology skills gaps, measure progress over time and establish customized learning paths that help individuals progress in their roles.
When the upskilling strategy is closely aligned to the overarching company and technology strategy, organizations can tie other important benchmarks to the measurement process. The most successful Pluralsight Skills customers know that upskilling programs have a direct impact on business-critical touchstones including employee retention, accelerated time to project execution and time to onboarding new technology hires. Pluralsight Flow customers can also measure and connect skill proficiency to things like code impact. These important factors should also be measured as part of the success of the program because effective upskilling efforts have a direct impact on each of these critical areas.
Above all, when companies measure the factors that are most aligned with the larger company strategy, they will know that their upskilling programs are contributing value to the business.
Director of Diversity, Equity & Inclusion in Tech at Intuit
See who's who in the Protocol Braintrust and browse every previous edition by category here (Updated June 9, 2022).
At Intuit, we use reskilling programs as a way to hire and retain top, diverse talent. Our reskilling programs include our Intuit Again returnship, which enables tech workers to on-ramp back into roles by revamping their skill sets, and our Apprenticeship Pathway Program, which springboards individuals from underrepresented groups into long and prosperous careers as technologists.
The way to measure these reskilling programs is to measure conversion from program participant to a full-time employee (acceptance rate), as well as engagement rates of individuals going through the programs. We also measure program growth, which helps us gauge the desire of our business units to invest in upskilling; a 4x growth rate tells us our hiring managers and teams are satisfied with the quality of candidates these reskilling programs have moved through the pipeline into roles on their team.
Beyond measuring the success of our initiatives, we aim to have an impact beyond Intuit, aligned to our goal of propelling top, diverse talent into prosperous careers. To measure this, we consider data at the individual level, such as salary and benefit increase (quality of life) compared to participants’ previous career and percentage of participants that desire to continue learning, whether that means exploring other areas of tech or pursuing a degree in computer science. We also track and measure Intuit’s impact on underrepresented groups overall with data such as how much we’ve invested in education and exposure of tech, and the overall percentage of women of color in the tech industry.
Kevin McAllister ( @k__mcallister) is a Research Editor at Protocol, leading the development of Braintrust. Prior to joining the team, he was a rankings data reporter at The Wall Street Journal, where he oversaw structured data projects for the Journal's strategy team.
More from Braintrust