How are these evaluations conducted?
An expert (usability/learning) judges the effectiveness of the course. The expert will go through the course and try everything a learner would if he/she were to take the course. They look for obstacles, ambiguity, functionality, and several other issues that hinder progress. Detailed reports are generated at the end of the evaluation.
What is an expert usability evaluation?
Using this technique, you evaluate the usability of an online course. An expert lists the parameters based on which the evaluation will take place. These could include:
- Navigation: What is the primary form of navigation? Is this intuitive enough? This would ideally mean that we do not include 'Click Next to proceed' kind of instructions. The learner should intuitively know what the primary navigation is.
- Visual hierarchy: Is the information organization in a logical manner? Eye movement is typically from left to right and from top to bottom. Are all elements positioned keeping this in mind? Will the learner know where the information starts and where it ends?
- Accessibility of information: Are important elements placed upfront? Will the learner be able to access the most important information easily.
- Affordance: Do buttons have the affordance of a click? Will the learner know when a click is required? Will he/she know what is expected on an interactive screen? During learner testing, I have seen learners click images that are not clickable or miss buttons that need to be clicked. This is because the element does not have the affordance of a click. Therefore, it is important to identify such issues.
- Fonts and font sizes: Will the learner be able to read the text easily? Do font colors hinder readability? Are these fonts and font sizes consistent across?
- A QA checks whether the online course maps to the signed off storyboard/wireframe. It also checks functionality, consistency, and whether the course has any bugs.
- Expert usability evaluation, on the other hand, checks whether the elements in the course are usable. It also takes into account user experience. Does the eLearning application cater to the five principles of usability - learnability, efficiency, memorability, errors, and satisfaction?
What is an expert learning audit?
Using this technique, you can evaluate the learning effectiveness of a course. An expert lists the parameters based on which the evaluation will take place. These could include:
- Learning objectives - content mapping: Are the learning outcomes addressed? Can the content be directly mapped to the learning objectives? Is there more information than is stated in the learning objective?
- Learner-content mapping: Is the content specific to the learner profile? Is it relevant? Will it help the learner meet the learning outcomes?
- Learner motivation: Is the course motivating enough for the learner? Why will he/she complete the course? Will they find it interesting? Will he/she be motivated to take an exercise?
- Vizualization: Do the visual elements aid learning? Are they similar in look and feel across the course?
- Language: Will the learner understand what is written? Is there any ambiguity?
By the time the course is developed, several ID reviews have already been done. The learning audit is conducted by a third person who has not been a part of the design phase. Therefore, the course is looked at by a fresh eye and this makes a world of difference. The expert looks at the course without considering the constraints. I believe this is a good thing because several times we compromise on learning because of we are thinking about the constraints. He/she looks to identify the obstacles that will hinder learnability. An expert can help us identify where we have compromised.
Keep the following in mind if you are evaluating:
- Ensure that there are no distractions. This requires a lot of concentration, else you may miss a crucial issue.
- Try everything. What is the learner were to click this? What would happen if I go here instead of there?
- Use screen grabs to highlight issues. This is helpful as the reader will not have to shuffle between a report (xls, word) and the course.
- Include suggestions wherever possible. Provide two or three alternatives if possible. It would be very effective to show a suggestion visually.
- Ensure that you mark the repetitive issues as a global comment. But, it is also important to identify all those screens in which the issue is present. This will help save time when the reader is fixing the issues.
- If you have set parameters, you could check each screen for each parameter in a logical manner rather than just scan a screen. This way you will not miss anything.
- It can become very tedious, tiring, and repetitive. So, be prepared.