Top ten tips for writing effective questions - Saffron Interactive

• Friday, August 7th, 2009
• Written by Stephanie Dedhar

Whether they’re for an assessment or part of the training course itself, writing questions can be a tricky business. Read on for Saffron’s top ten tips for creating effective questions that test learners in the right way, on the right thing.

1. Draft the assessment before the content
If step one is defining the objectives and learning outcomes, step two is drafting the assessment. Make sure each question maps back to the outcomes and then focus the training content on the assessment.

2. Randomise the assessment
Ideally an end of course assessment will have a pool of questions, with a random selection drawn on each attempt. This means, if learners fail once, they can’t simply memorise where they went wrong but are actually tested again.

3. Remember: quality not quantity
When it comes to assessments, it’s better to have 10 excellent questions than 40 substandard ones. Likewise, multiple choice questions generally work best with four options - don’t go below three or above five without a very good reason.

4. Use questions to drive the learning
We’re big believers in the value of the test and tell approach. Don’t just give learners information and then test them on it. Instead, ask them to think for themselves with a question before correcting or confirming their answers.

5. Focus on behaviours
We’re in the business of changing behaviours. This means we need to enable learners to do the right thing, not just to remember facts and figures. Make sure every question relates to a choice or decision learners will face in real life.

6. Make it challenging…
If the right answer is three times as long as the rest, or the wrong answers are likely to make learners laugh than think, there’s no point even asking the question. Yes, coming up with plausible wrong answers is hard, but it’s worth the effort.

7. …but keep it fair
Don’t try to trick learners or give them an impossible choice. Negative questions, options that are identical except for one word, and questions on topics that weren’t actually covered in the course are neither fair nor effective.

8. Avoid yes/no questions
Giving learners a 50% chance of guessing correctly is not the most effective test. If you must use them, include a couple more options and make sure that ‘yes’, ‘no’ and ‘maybe’ are qualified, so learners have to justify their choice.

9. Help people learn from their mistakes
Don’t just use ticks and crosses to show how learners have done. Provide constructive feedback to reinforce the message if they answer correctly, and explain where they went wrong if they answer incorrectly.

10. Keep the learners’ experience in mind
Focusing on behaviours is one way to answer the ‘what’s in it for me?’ question. But there are other ways - for instance, pre-tests assess existing knowledge and point towards the most relevant training units. Time efficient and effective!

You can read the original post here.

From click next to the next level -The eLearning blog

An informative post on making e-learning interactive.

The promise of rapid elearning has for many ended up with a deluge of page turners. What went wrong? Many training honchos thought SMEs could learn the tools in a day and then they could eliminate an entire industry of instructional design out of the production process. So most rapid elearning ended up without instructional design and more importantly, without industrious design.

Page turners, by definition, are devoid of activities-practice and assessments. Without activities, learners miss out the feedback loop of interactive communication. Minus the practice activities, learning is at the mercy of the Ebbinghaus curve of forgetting and the only way memory decay can be staved off is by regular repetition or the old school rote memorization.

When instructional designers create page turners, they do more than just create presentations. For many, the fashionable-to-deride Click Next is a turn-the-page signal. For the discerning designer, it is the least minimum divisor of curriculum structure and sequence.

Of all the clicks a learner will make, Click Next is the most empowering as it links information consumed with information awaited. This link also seeks to replicate itself in the information hierarchy developing the learner’s brain. Page turners are not ineffective because of Click Next but because developers regard it as doing nothing but scroll function.

What’s on each page and how it’s represented impacts memory more in a page turner than in any other elearning. Content chunking “do’s and don’ts” and media principles for use of text, images, and audio exert a considerable impact of memory encoding. Works of Fleming, Levie, and Mayer remain the foundation of design of instructional message. Without application of these, it doesn’t matter what level of courseware you create—the learner is saving the information to a soon-to-be-deleted temp file! Application of the media principles enables the user to save information to his ready-to-be-accessed “See” drive.

Besides content design, a rapid elearning designer cannot afford to neglect the Component Display Theory. Component Display Theory maintains that to achieve higher levels of mastery, generalized content such as concepts, principles, and procedures need to be elucidated using a divergent range of examples and non-examples. This repository of examples and non-examples is the real interactive element of any learning.

Armed with a range of examples and non-examples, instructional designers can present the information using the inquisitory (Ask-then-reveal) method rather than the expository (Show-to-tell) method. Usually, it is the limited range of examples and non-examples that forces designers to rely of the standard rule-example sequence of presentation-activity-feedback routine. With more examples, presentation and feedback become integral to the learning activity.

To make your training more interactive, focus on the content development efforts for developing more examples and non-examples of the generalized content. What gets touted as Level 1, Level 2, Level 3 and Level 4 courseware is nothing but a description of the quality of examples embodied in a course. The more detailed and enriched an example, the closer it is to a reality-like simulation. Scenario-based practice and assessments utilize examples with the relevant details accentuated to make learning decisions possible.

To summarize, Click Next is not the culprit. To make rapid elearning more engaging, you need to work assiduously with the SMEs to develop more examples. Examples create opportunities for activities and activities yield better learner performance.

Article courtesy: http://tinyurl.com/mrncq3

No time for design - Cathy Moore

Another brilliant post by Cathy Moore on Instructional design.

Do we still care about instructional design? This graph from Google Trends compares searches for “elearning” with searches for “instructional design.”

At first, “elearning” followed “instructional design” in a sad slope downward. But in the last couple of years, “elearning” has perked up again, while its friend “instructional design” continues its descent into obscurity.

Maybe fewer people are searching for “instructional design” because it’s no longer a new concept (”usability” suffered a similar decline). Or, possibly, fewer people are searching for “instructional design” because fewer people care about it.

Did “rapid” kill ADDIE?

Here’s what can happen to the ADDIE approach when we care more about speed than anything else.

1. Analysis: “The client wants a course, therefore the client needs a course.”

2. Design: “Let’s use the template we used for the widget course, but with a blue background.”

3. Development: “Clean up the client’s PowerPoint slides and add a Jeopardy quiz.”

4. Implementation: “Put it on the LMS.”

5. Evaluation: “Did everyone look at every screen?”

I’m not saying rapid tools are evil. You can use them to create powerful elearning. It’s rapid design that’s the culprit, because it’s not really instructional design. It’s just content presentation. We end up putting lipstick on a pig.

But ADDIE takes too long!

When we treat ADDIE as a step-by-step process, it’s inefficient. By the time we’ve created our 39-page cross-referenced design document, we could have delivered a prototype of the course and gotten feedback from the client and learners, as Sumeet Moghe points out in his description of an agile approach to elearning design.

It would also help if we let go of our obsession with looks. Unfortunately, a text-only branching scenario that profoundly changes your employees’ approach to complex sales won’t get an award—it won’t even be submitted. The award submission will be a glitzy “course”/slideshow with redundant narration, flying pie charts, and a game-show quiz that has little effect on people’s performance.

But what about the lemurs?

I’m not saying that the Google Trends chart proves that instructional design is getting short shrift in elearning. I posted the chart because it happens to be a visual expression of my concern. There could be just as much correlation between the two searches as there is between “elearning design” and “lemurs” (which, happily, shows no worrying trends):

Check out the original post here and make sure to read the interesting comments.

3 Things to Consider When Building Your E-Learning Courses

Another significant post by Tom Kuhlmann.

The difference between effective and ineffective elearning is how you design the learning process. In this post, I’ll do a quick run through of some ideas centered on instructional design and three things that you want to consider when building your courses.

Focus on meaning and not information

We tend to equate learning with sharing information. Typically if there’s something we want to change, we think about how to get more information to people. In fact, most of the elearning courses I see are focused exclusively on sharing information. While sharing information is part of the learning process, it isn’t THE learning process.

If you’ve been reading this blog for a while, then the following point won’t come as a surprise. To build better elearning courses, the solution is to focus on the learner and not just the information. In fact, most disagreements we have with our subject matter experts and clients is usually because they tend to focus more on information and less on learning.

We need to take the information and then package it in a way that allows the learner to use it. That is when the learning happens.

When I was a young (and thin) video producer, I had to submit my projects for a peer review every Friday. I wasn’t allowed to explain the intent of the project. Instead, I had to play the video and then listen to what people got out of it. One of the goals of this exercise was to see if those watching the videos got the message I intended. In most cases they didn’t. Usually, the reason was that my videos were designed for me and not my audience.

Through those peer reviews I learned that my job wasn’t to give people information. Instead, it was to craft meaning out of the information. This applies to instructional design, as well.

The e-learning course is just one part of a complex process

As humans, we’re always in learning mode. We don’t turn learning on or off. In a sense, learning is like an ecosystem. We’re continually influenced by information, our social interactions, and experiences. These shape who we are and what we know. And this ultimately determines how we act.

We don’t learn just because someone gives us information or tells us that today we’re in a “course.” We learn because that’s how we’re wired. So when we do happen to take a course we fold it into our learning ecosystem and make it part of how we understand the world around us.

While we invest a lot of energy in building our elearning courses, those who take them aren’t as vested. To them, the course only represents one part of a larger process. How a person learns and builds understanding involves more than just taking a 30 minute elearning course.

Instructional designers are intentional

An instructional designer takes information and presents it to the learners so that they can develop context. When it’s done right, it can expedite the learning process which can save time and resources.

Here are a few simple examples. Look at the text below. It’s information. But what does it mean? Is it in reference to a storage bin? The sport of boxing? A shape?

In the example above, the meaning of the information is up to the learner to decide. But who knows what they’ll come up with? Since you could have multiple meanings, you end up having to correct perceptions that run counter to what you are trying to teach.

Now, let’s look at the next image.

You see an image of a box and the word. However, you’re not quite sure of the relationship, if there is one. This again forces the learner to think through the information. They might or might not see a relationship. And in turn, might draw the wrong types of conclusions. Just like the first image, if they draw conclusions that are not consistent with what you are trying teach, you’ll have to spend additional time and resources bringing more clarity to the information.

In the image below, because the box and the word are close together, you have an implied relationship. This brings clarity to the learner. You don’t need to explain as much because of the way the image is designed.

When it comes to instructional design, everything you put on the screen serves some purpose. In the example above, we started with the word “box.” By itself it can mean many things. Without some intentional design the learner can be distracted by irrelevant information or assumptions. In some learning environments that might be fine and exactly what you want. But in many it can be a waste of time and resources.

Adding the image of the box made it a little clearer. However, the final example with the image and text together brought meaning to the word. When you design your courses, think less about the information, and more about the relationship the pieces of information have to each other and most importantly to the learner.

While the above examples are simple, they do represent something essential to instructional design. You take information and present it in such a manner that the learner is able to see its relationship and context.

As an instructional designer, you want to focus on meaning. How is the information important to the learner? In what context? Considering that the elearning course is only one part of the learning process, what other ways can you support that learning happens? Is there a way to leverage the learner’s already existing learning ecosystem?

Finally, can you expedite the learning process by building the right types of relationships between the pieces of information? Are you using the right images, text, and multimedia?

These are just a few things to consider when building courses. The key takeaway is that learning is complex and because you build an elearning course doesn’t mean that learning happens. It involves an intentional act on your part and a commitment from the learner to apply what is learned. The better you design the information, the more likely the learner is to commit to using it.

Article courtesy:http://www.articulate.com/rapid-elearning

5 things IDs can think differently about! - Geeta Bose

I came across this blog post by Geeta Bose via twitter. An informative piece for Instructional designers.

Check it out here

The SME test by Norman Lamont

An excellent post by Norman Lamont of Lloyds Banking Group

The SME Test

View more presentations from normanlamont.

Video courtesy: http://www.slideshare.net/normanlamont/the-sme-test?type=powerpoint

3 Proven Techniques to Add Creativity to Your E-Learning Courses

An useful article by Tom Kuhlmann for instructional designers.

I like to download the TED videos and watch them when I travel. On my recent flight to Atlanta, I watched IDEO’s Tim Brown talk about the powerful relationship between playing and creative thinking. The essence of his presentation is how to design a better product and user experience.

He discussed three key elements: exploration, thinking with your hands, and role playing. I jotted down some thoughts about how they might relate to those of us who build elearning courses.

Exploration

Building an effective elearning course is more than just applying the mechanics of instructional design. It also involves a creative approach to visual design and how you present the content to the learner. This is as much art as it is science. So it’s important to have fresh ideas and renewed perspective.

As Brown asserts, you can get this with “exploratory play.” He talked about focusing on quantity and not quality of ideas. You don’t want to self-edit. Instead you want to push your creative thinking by generating ideas.

Teams I’ve worked with in the past have done this as a routine. For example, in one session we tried to come up with 50 ways to guide a person through a course. Some were kind of goofy, but we did end up having the foundation for quite a few future courses. At another session, we designed 30 ways for the learner to advance through screens without using an obvious button or instructions to “click here or there.” That was a challenge.

Most of our ideas didn’t make it into production. But that’s OK. The goal was quantity and not quality. Because we ran out of obvious ideas quickly, we really had to stretch our creative thinking to produce more.

Even if you don’t have a team for this type of brainstorming, there are some things you can do on your own. I have a folder on my computer where I collect ideas. When I read something, view a course, or find an interesting graphic style, I save it to the folder. I can go months without every looking at what’s in there. But whenever I need inspiration, I go to the folder and do a scan of what I have. It hasn’t failed me yet.

If you do follow this approach to collect ideas, OneNote and Evernote are two good tools because you can easily do screen and text captures. The captures preserve where you got the idea, and you can add your own notes and tags for quick reference and searching.

Think with Your Hands

Brown talked about “construction play” where you think with your hands. He gave some examples of prototyping. I see this as an extension to the exploratory play above. In exploratory play you generate ideas. And then you transition to making some of them real by thinking with your hands or attempting to actually build something.

A good place to start is by breaking down one of the ideas you’ve collected in your idea folder. It doesn’t need to be anything fancy. It’s just a working model or proof of concept. For example, Brown showed how IDEO prototyped a surgical instrument with a marker, a clip, and an old film canister.

In previous posts, I’ve talked about deconstructing the work of others. I showed you how to design your own templates and how deconstructing courses will help you develop the skills to build better courses. I see deconstruction as part of the “playing with your hands” process. You’re not trying to copy the work of others, but by pulling it apart and building your own versions you end up learning how to do things in a new way. These are techniques you can apply to other projects.

When I play with ideas, I don’t commit a lot of time to the details. I use simple shapes and test them out first. For example, one of my favorite elearning sites is Edheads*. They have some great elearning courses that are very interactive. Similar to the frog dissection demo, I’ve been playing with ways that I’d replicate some of the functionality of the virtual knee surgery course in PowerPoint without using Flash.

Click here to view the knee range of motion prototype.

As you can see, the prototype is nothing fancy. The main point is to play around with an idea and make it something tangible. You’ll learn new techniques which will help you be more creative and produce your courses faster.

The virtual knee prototype above only took a few minutes to build. If a customer wanted an example of a rapid elearning course with some interactivity, I could create a prototype like this on-the-fly because I’ve made it a habit of building them as part of my professional development.

Role Play

For me, one of the most striking parts of Brown’s presentation w as when he showed the image from the emergency room video. It was shot from the perspective of a patient. When they reviewed the video, all they saw was 30 minutes of the acoustic tiles on the ceiling. Immediately, I could empathize with the patient’s perspective. And I could already think of ways to improve the service. If I were trying to design a better experience for the patient, this type of role play would really help.

So, how does this apply to elearning? Glad you asked. Ultimately, you want to design a product that serves the learner.

When you sit down and look at the courses you build, are they ones that you’d enjoy going through?

When I was at the conference in Atlanta, someone asked me for some ideas on how to get her instructional designers and subject matter experts on the same page. She was struggling with how to get her subject matter experts to frame their expertise in a way that was relevant to learners. And then have her instructional designers understand how to build the right type of learning experience.

I referenced the role play ideas from Brown’s video that I had watched on the flight over. We talked about the need for the designers and subject matter experts to see the course from the perspective of the learner and not the information.

The goal is to design an experience for the learner. And in doing so, make the subject matter relevant. Perhaps there was some sort of role play exercise that could put the subject matter expert and instructional designers in the position of the new learner. Here are some ideas.

• Become a new learner. Sign up for an elearning course and then see what works for you and what doesn’t. There are always things to learn about navigation, the user interface, and interactions that engage you. Take what you learn and apply it to your own elearning course.
• Meet with recent learners and get their perspectives on how to structure the content. Ideally, you’d bring a recent learner into the process as part of your normal development cycle. They’ll help you keep the course real. If not, at least try to get their perspective at the beginning of the design process.
• You could quickly prototype a course and then watch a learner go through it. Find someone who has no vested interest in the course. Watch how they interact with the content. You might be surprised at what they do. You might also be surprised at what they feel about the content.
• For each piece of information you present, apply an expected action. “If the learner knew this information, what would it look like if I saw it?” Build your course around those expectations.

Role playing is about designing the best user experience. The key in all of this is to put yourself in the position of the new learner. And by doing this, you’ll make a course that is relevant and successful.
As you watch the video, you’ll probably get different insights. These are three that came to mind and ones that I think can be applied in a practical way.

Article courtesy:articulate.com/rapid-elearning

Instructional designer's nightmare

Video courtesy: elearning network ( designers only group) - Clive Shepherd