Quality Assurance Checks

Today, I was pulling out older bug reports to share formats for QA of a new course. This had me reminiscing. QA is a vital process before final roll out. Why is QA important? It helps identify the bug and errors in the course. It also helps check consistency, content misses, errors introduced during development, animation sequence, audio flow, and so on. QA helps us check the course on the whole whereas ID reviews, edit review, and graphic reviews check each aspect individually. When the course is developed, we get to finally see it function. Now, it is not rare to have one or two bugs in the application. It is important to identify these before the learner takes the course.

How do we conduct QA? We use an xls format to document slide numbers, the issue, the type of issue and so on. We put the course itself under the microscope and dissect it to see whether things are in place. We try out all options available to ensure that the course is behaving the way it should. Areas to be addressed during QA are:

• Functionality
  
• Are all buttons on the interface working properly?
• Are all the screens linked logically?
• Are all the tabs and click to know text working as they should?
• Are the exercises functioning normally?
  
• Is the navigation free of bugs?
  
• Audio
• Is the VO in sync with the OST?
• Is the audio audible, crisp, clear?
• Has any part of the audio got chopped off during integration?
  
• Alignment
• Are all elements aligned to the grid?
  
• Are feedback boxes popping up in the right area?
  
• Are the text boxes popping up in the right area?
• Are the graphics positioned appropriately?
• Has the white space been used appropriately?
• Graphics
• Are the graphics clear?
  
• Are they in sync with what was visualized in the storyboard? (Though this should be ideally taken care of during graphic reviews)
  
• Do the graphics seem to belong to the same family?
• Are they adequately sized and positioned on the screen?
• Do they follow the color scheme?
  
• Consistency and standardization
  
• Is the instruction text standard across the screens?
• Are the feedback boxes standard across the exercises?
• Are the tabs and click to know text consistent across the course?
• Is the font (color and size) consistent across the course?
• Content accuracy
• Are there any content misses?
• Are the screens too text heavy?
• Are there any new errors introduced in the course?
• Are there any edit issues in the content?
  

From prior experience, I know how crucial his stage. I have witnessed how this round can check bugs and ensure an error free course before learner testing. If work is not systematic, errors will come up with every round of QA. This can be very frustrating. Ensure that you always conduct at least two rounds of QA to ensure that your course meets the quality standards.

Evaluating a Course

What factors must you check while evaluating a course?

• Are the learning objectives obvious?
  
• Are the learning tasks clearly defined?
• Visual:
  
• Is the interface intuitive? Does it facilitate learning?
  
• Is the visual layout and hierarchy neat and logical?
• Are the icons and graphics used appropriate? Does it facilitate learning?
• Audio
• Is the audio sufficient, too much, or too little?
  
• Is it appropriately paced, too fast, or too slow?
• Does it have life or would it put the learner to sleep?
  
• Text
• Is the text on screen too much, too little or just right?
• Is the instructional flow maintained?
• Has the information been chunked and presented effectively?
• Are the instructional strategies challenging for the learner? Does it help meet the learning outcome?
  
• Does the strategy ensure high instructional interactivity?
• Do the exercises reinforce leaning?
• Are the exercises introduced at the right junctures?
• Is there an exercise to check each learning objective?
• Are the exercises forced or they motivating?

This is, however, only a surface level evaluation. A detailed evaluation will check the 'learnability of a course.' For detailed evaluation read article on Learability Testing.

Tailor-made Courses

Do you buy clothes that are three times your size? Do you buy shoes that are twice as big as your feet? Why not? I guess the logical explanation is that we prefer to buy things that are made for us. At school, why do we begin at the 1^st standard and proceed to higher classes? Why are all children not put in the same class? This is because each learner is at a different level and their learning needs are different. Right.

Wondering where I am going with this? The objective of this article is to stress the importance of designing courses that are tailor-made for the learner’s needs. Let me share an incident with you. As part of our LCM process, we conduct learner evaluation. In this phase, we ask our learners to take the course and we observe the learner. During one such learner testing, we found some interesting truths. Before I proceed, let me give you some information about the course. A training company had conducted a two-day workshop and this had to be converted into a refresher emodule (that’s where we came in) for managers who had attended the workshop. Seven learners volunteered to help test the course. 3 out of the 7 learners had not attended this workshop. The results showed that these three learners were lost and very confused. The other four learners were extremely comfortable and were motivated to complete the course. This was because they remembered the cues provided during the workshop and therefore, found the course interesting. Why the discrepancy you ask? This course was meant to be a refresher course and catered to a very specific audience. It is therefore effective only for that audience. Most learners claimed that this was the most effective elearning program that they ever taken. Now, that’s something...

Each learner is at a different level and their learning needs are different. It is extremely important to ensure that we chunk similar learners together and design a course that caters to their common need. For example, when designing a course on how to use a software application, we will design this keeping in mind the primary learners/users and not the secondary and tertiary learners/users. This way we are ensuring that we make this course effective for the learners who interact with the software more frequently. If you really have to make the course for everyone coming in contact with that software, there is a way out. Have three learning paths, one for the primary, another for the secondary and third for the tertiary audience. This will ensure that the learner goes through only that information that is relevant to him/her.

Why is it so important to create courses for a specific audience?

• Maintain motivation: Most elearning courses do not meet the ‘What’s in it for me’ criteria. Learners want to know how the content you teach can be applied in their jobs. Learning can not be forced. The learner has to see value in the course. Show them the value and the battle is have won.
• Provide relevant information: Imagine this scenario. Your learner is working in a software company. He/she comes in at 9:15 in the morning and leaves at 8:30 in the evening. He/she dedicates half an hour everyday to a training course. Do you actually think he/she will have the patience to go through a course and search for information relevant to him/her? Don’t give a sea of information and ask them to find what they want. Give them exactly what they want. Do some research to identify this.
  
• Effective learning: The course is definitely more effective when it is specific. For example, you are teaching learners to deal with conflict situations at the workplace. We know that conflict situations can occur at any level in the organization. If the learner is a middle level manager, the conflicts situations he/she is likely to face and the power he/she has to tackle such situation will be different for a senior manager and or an executive staff. Conflicts could be between manager-employee, manager-manager, and employee-employee. The employee need not (and should not) have to learn how manager-manager conflicts are tackled. Another interesting thing is that when designing a course for an employee, the content for manager-employee will be different as opposed to that designed for a manager.
• Cognitive overload: Excess information or detailed information will cause cognitive overload. Designers (or should I say SMEs) end up stuffing everything (to be safe) in a one hour module. Trust me, nothing will register in the learner’s mind. He/she will click the pages away. Stick to the learning objectives and the Bloom’s levels.
• Increase completion rates: By giving the learner what he/she wants, you are going to ensure that dropouts decrease. If the learner sees benefit in taking a course, why wouldn’t they complete it?
• Transfer of learning: The chances of bringing about a behavioral change are higher when the course is designed for a specific audience. Retention of concepts will be higher. Therefore, transfer of learning at the workplace will be higher.

Remember that you wouldn’t finish a book or a movie if it did not interest you. It would interest you only if you can relate to it.

Duration of a course

How long do you think a good course should be? The answer is "It depends". I don't believe in standards such as a good course should not be too long, too short, and so on. How can we reach a conclusion without doing our research and understanding the requirements? A good course should be as long or short as it should be. Not helping, am I? Let me try and explain.

The duration of a course is decided based on stakeholders' requirements, learner profile and requirements, training gaps, table of content, organizational objectives, and learning objectives (if I have missed anything important, please do share). The more focused the objectives, the easier it would be to allocate duration. If it covers everything, it is bound to be ridiculously long and not very effective. For these kind of situations, we can suggest that the course be split into focused sections or parts. For a particular project, we came across a situation where the learner was not be able to spend more that ten minutes at a stretch for learning. We designed the course such that the content was divided into learning nuggets of 5-10 minute duration. You may have also come across situations where the learner is able to devote only half an hour per day for training. Now, this does not mean that we cramp everything within that half hour.

Think of a really good movie you saw. What is not important is how long is the movie but how good it is. I have seen two hour-movies that have been extremely painful to sit through. On the other hand, I have also seen 3 hour movies that I have enjoyed. When the movie is good, you don't realize how much time has passed. Now, this does not mean that we make our courses really long. Remember, how irritated we get when an unnecessary song or scene is a part of a movie. This is what I meant by courses should be as long or short as they should be.