Canada Association of Tourism Employees

Setting The Passing Rating In eLearning Programs 

Is there a reason behind the 80%?

In all of my more than 20 years of designing eLearning courses, I have always set the passed score at 80%. Most of the customer teaching designers also insist on this percentage. 80% has become something of a standard industry standard. But what is the rationale for this number? Is there any logic behind it? Or is it something that learning specialists have chosen arbitrarily and the rest of us will follow suit? Let’s see if this magic number is anchored in learning theory. And if not, how should we set the final passing score?

When do you frame the last quiz?

The logical sequence in designing an eLearning course begins with identifying a business challenge that can be addressed through performance improvement training, followed by breaking the training needs into measurable learning objectives (LOs) that are performance-based.

Once the performance-based learning objectives have been set, the next step should be to translate them into appropriate assessment or test questions that measure the achievement of the learning objectives. Most IDs skip this step and jump into content creation and return to formulating a final quiz at the end. This order is not advisable as it can mislead the designer into asking quiz questions that are not properly aligned with the LOs, resulting in the quiz not properly measuring the effectiveness of the course.

Therefore, the correct order would be to start with LOs, set the final quiz questions, and then select the content. This ensures a perfect match between the LOs and the final quiz and ensures that the most appropriate content is selected to address the LOs.

Are some LOs more equal than others?

Bloom’s taxonomy of classifying human cognition into different levels encompasses the most basic level of cognition to the highest: knowledge, understanding, application, analysis, synthesis and evaluation. Recently, these nouns have been revised to verbs: remember, understand, apply, analyze, evaluate, and create.

As we all know, Bloom’s taxonomy gives us the framework for setting learning objectives at the appropriate level of knowledge. It also shows that the learning goals are ordered from enabling goals to end goals; that not all LOs can be considered the same, some allow; and some are terminal. Small, enabling goals cannot be given the same score as an end goal, as the latter is critical to achieving the desired performance.

Assessment based on the level of cognition

LOs can be classified on the basis of the Bloom taxonomy according to their activating and concluding character and the evaluation of their corresponding performance based on the parameters frequency, learning difficulty, criticality and the associated risk. As soon as this is done, the corresponding quiz questions can be assigned depending on the type of previous LO points.

Now, this way of classifying and assigning ratings for each test question based on multiple parameters can be difficult and time consuming. An easy and practical way to do this is to categorize the test questions into three levels – Elementary, Intermediate, and Advanced – and assign the same score to the questions at each level. The score increases with the level.

For a final quiz with 12 questions, for example, questions 1-4 (basic knowledge) can be assigned 1 point each, questions 5-7 (intermediate level) 2 points each and questions 8-12 (advanced) 1 point each 4 points, whereby the total quiz endpoint is 30 points.

Authoring tools such as Articulate Storyline and Lectora Inspire can provide this output with small tweaks.

Are all services critical?

Some tasks are difficult to complete, some are infrequently (and some on a regular basis), some are critical to performance, and some are less. Some performances are risky in that, if improperly performed, they can cause damage or loss to the performer and / or the organization. Hence, it stands to reason that not all tasks are the same and therefore cannot be assigned the same number of points. Some questions warrant a higher score based on the frequency, ease of learning, criticality and associated risk of the respective tasks / achievements.

They would require the learners to achieve 100% on all questions relating to critical (such as operating a business-critical machine) and high-risk tasks (such as handling hazardous chemicals), since in these situations even a 1% probability of error is not would be acceptable. When it comes to tasks that are done frequently (such as daily routine tasks), or are simple or less important and pose no risk, you can set a milder pass value, such as 50 or 60%.

Set the registration result

So how do we bring all of these together to arrive at a passing score based on the type of performance? Once the goals and corresponding quizzes are formulated, we can assign points to each LO based on its basic nature, activation or completion. Then identify each LO’s corresponding performance task and assign it a weight based on parameters such as criticality, risk, and learning difficulty. By multiplying the points by performance weights, we can get a minimum acceptable score, which is the passed score.

• Question 1, basic level (ability level), 1 point, performance weight 0.3, passing point 0.3.
• Question 2, basic level (ability level), 1 point, performance weight 0.3, passing point 0.3.
• Question 3, basic level (ability level), 1 point, performance weight 0.3, passing point 0.3.
• Question 4, basic level (ability level), 1 point, performance weight 0.3, passing point 0.3.
• Question 5, intermediate level (enabling), 2 points, performance weighting 0.5, passing point 1.0.
• Question 6, intermediate level (enabling), 2 points, performance weighting 0.5, passing point 1.0.
• Question 7, intermediate level (enabling), 2 points, performance weighting 0.5, passing grade 1.0.
• Question 8, intermediate level (ability level), 2 points, performance weight 0.5, passing grade 1.0.
• Question 9, advanced (terminal) level, 4 points, power to weight ratio 0.8, existing result 3.2.
• Question 10, Advanced (Terminal) Level, 4 points, Performance Weight 0.8, Existing result 3.2.
• Question 11, advanced (terminal) level, 4 points, performance-to-weight ratio 1.0, existing result 4.0.
• Question 12, Advanced (Terminal) Level, 4 points, Performance Weight 1.0, Passing Score 4.0.

Total score 30, existing score 19.6 / 30 (65%).

That way, you can get to a more logically derived end result that is solidly based on learning theory and final achievement, rather than going with an arbitrary, deranged number like 80% that gives equal weighting to all questions and then sets a score to exist without any real basis.

CommLab India

Working with multiple outsourcing providers can be stressful and a waste of time and money. With CommLab India, all of your online training needs will be met quickly and inexpensively.