Is Your Online Course Good? Evaluation methods
Evaluations methods was my favorite subject in grad school, while I was pursuing a degree in adult education, training and staff development. My training geek hero is Dr. Donald Kirkpatrick.
Side story: a year or so before Dr. Kirkpatrick passed, he spoke at the Orlando Association for Talent Development (ATD) meeting. I participated with him in a role play, live. It was so cool! His genius and genuine love for his body of work was on full display. Someone took a picture of us, and it is a treasured item. I won’t post the picture here because of my previous article on Copyright Infringement. :)
Back to today’s story…
Evaluation methods are my favorite, because my focus when I train, write, or do pretty much anything in life, is “Did it work?” and “What happened?” (That is why I am obsessed with finding old classmates or coworkers on social media. LOL)
measure results to gain your participants’ Trust
If your mission in life is to develop people, either in a personal or professional fashion, it is a question you would do well to ponder. That will be the Number One question of potential participants. They want reassurance: “if I trust my investment with you, will you deliver on your promise? Will you solve my problem or help me improve?”
Kirkpatrick’s Evaluation Model
This model, developed by Donald Kirkpatrick in 1959, is used widely in the learning and development space. The following will be my short version of his model and how you can use it in your online course, or really any course. At the end of the article, I will share a link to his company’s website for more information.
Level 1 - Reaction
In the training biz, we call these “smiley sheets.” We all know these sheets. You finish facilitating an event (live or on demand) and give the audience a survey to evaluate their experience. It doesn’t tell you much, but it does tell you if at least in the “class” environment, you are doing something right. It usually is good for the ego.
Online course execution: When the participant finishes, you program one last survey asking about their experience. Did they like it? Too long? Too short? Caution here on asking too many questions. Remember, they are doing you a favor by answering the questions, so only ask fewer than five and make them count.
Level 2 - Learning
This is a quiz type experience of the content. Did they academically understand your content? Think of it as when you get your driver’s license. There is a theory test to see if you understand the knowledge of the road before you need to put them in action.
Online course execution: Quizzes can be sprinkled throughout the course. Most hosting platforms give you access to individual participant scores. The more sophisticated, or costly, platform options will offer reporting. This is helpful, as you can see trends and at what point do participants struggle, allowing you to update the course accordingly.
Caution here: if you set the quiz for multiple tries, so they cannot fail, this is pseudo information. When developing the course, consider the strategy of your knowledge checks.
Level 3 - Behavior
This gets friskier. This is evaluating whether or not the participant can actually do in real life what you taught them. In the analogy above, it would be successfully driving a car. A business example is diversity training: are people after the class demonstrating the correct behaviors everyday, when it counts?
Online course execution: Obviously, it may be difficult to ever see your participant successfully change. However, you can have them self-report results.
For example, you create an activity like this: “In lesson one, you were taught how to say a successful phone greeting. For this activity, try out your new greeting on the next five people who call you. Write in the comment box below the results of your experience.” Not perfect, but closer.
Another example to encourage behavior change is to send follow-up emails during the course and after with a “You can do this” attitude, and offer for them to contact you directly. Always keep your door open for feedback.
Level 4 - Results
This is the friskiest. Every training event has an expectation tied to it. An example, if you hire a personal trainer for physical training, you have an expectation of body composition change. In corporate training, this would be whether you solved the intended problem. In an online course, it would be whether your participants experience the intended result of your sales promise.
Online course execution. This one is the most difficult to measure. It takes resources and time. In the corporate world, it is rarely done because the next fire pops up and needs attention. However, in the online space, you have a chance to do it. Contact participants 30 days post course and ask if they have time to answer a few questions. To conduct a large-scale Level 4 evaluation, you could contract a call center vendor to help. (I have a resource if you are interested).
People who successfully sell an online course almost always include a one-off story of participant success, a result.
However, if you could prove that you didn’t have just a one off success, but that most of your participants achieved a positive result, that is GOLD for you. You would separate yourself from the competition.
That is all for Professor Stacey’s short lesson in training evaluation. You can go down the rabbit hole of training metrics and ROI (Return on Investment). But if that isn’t your thing, no worries. Familiarizing yourself with the four basic evaluation methods above will allow you to smartly answer potential client’s questions.
Now if you are ready for the rabbit hole, visit The Kirkpatrick’s Partners website. Have fun!
“Your participants want reassurance: “if I trust my investment with you, will you deliver on your promise? Will you solve my problem or help me improve? ”
