Testing eLearning Products

Testing E Learning Products

View more OpenOffice presentations from Kern Learning solutions.

A presentation by Kern Learning Solutions.

Issues with Second Life

Virtual worlds (VWs) must take user experience seriously. Are these VWs usable? Let us look at Second Life. This is based on my experience and are only thoughts (not expert opinions). I wish Second life would relook at the following:

• Navigation: Why does the user have to spend time learning how to use the controls? Why isn't the navigation intuitive enough? Why is the Search option so confusing? As a new user, what do I search for? How do I decide where I would like to go? How do I know how big the VW is?
  
• Editing Appearance:

1. Privacy: What is the first thing a user does? Most users edit their appearance. Why is this visible to the rest? I can see another user editing the appearance. The stance is weird and the appearance and disappearance of clothing items is downright funny! Can't the user have the privacy to change their appearance?
2. Filtering: Like other social networks, I wish SL provided the option to view a user's profile. This helps understand more about the person. This may help avoid awkward conversations and situations.
3. Usability: The slider widgets used to readjust length of clothing are not efficient. Removing a part of clothing is tricky. I had trouble getting rid of a skirt my avatar had on on top of her jeans. I have also seen my friends struggle to with hair, clothing, shoes, and so on.
   

• Conversation: Why should avatar's type as you type on your keyboard? Let's admit it looks very funny. Is there no other intuitive way of letting another user know that you are typing?
• Actions: It is hilarious to watch a user master the art of sitting at SL. Most times, they face away from us or just run around the place. :) This is a usability issue. A user need not have to practice several times to sit to get it right.
  

In terms of SL for learning, I like the thought of letting the learner create something. But, most objects in SL (atleast those I have come across in my brief time there) are PPTs. SL also has virtual classrooms. When we avoid making the learner read lengthy notes and sit through lectures, why use it in SL? When the fancy of the VW wears off (as it did with me), learner motivation is bound to dwindle. What then? Is there no other interesting format? Check these posts by Karl Kapp on how VWs effectively.
Examples of usaing virtual world 3d spaces learning
Virtual Hospitals Protocol
Three Virtual World Learning Best Practices

Quauntifying Time Spent on Web 2.0 Tools

Is this a trick question? ;) My response to this month's Big Question:

How much time do you spend and how did you find time for all the relatively newer things like reading blogs, twitter, social networks, etc.?

I use TweetDeck and receive tweet notifications. If I am working on something that requires high concentration, I close this application. Else, I logically close a thought and quickly check what others are saying. If the tweet inspires me, I tweet back immediately. If it is an interesting link and a longish blog, I keep it open and read it during my next break. I blog has to be either really compelling or really short for me to read it immediately. I check Facebook only once or twice a day as I use it only for personal networking. Without a doubt, Web 2.0 makes us more effcient. Everyone values time. When a blogger blogs, he/she tries to keep it crisp. I scan through a blog before i decide if I want to invest time. Twitter is popular because writing/reading short tweets does not take time. Coming back to the question, I will not be able to say exactly how much time I spend networking/learning. It various as per project deliverables, my moods, blog updates, and twitter traffic.

What are you doing less of today than you were 3-5 years ago?

• I used to take longish walks to breath in some fresh air when I was at office. Now, that I work from home, I tweet or read blogs. I don't go through my Google reader as most bloggers I follow share links on Twitter.
• Earlier, I used to surf the Internet for hours together to find information that interests me. Today, I go looking for information only when my research is very specific to something. Most times, information comes to me (courtesy web 2.0). This surely means my search is more effective and productivity high as I spend lesser time surfing the Internet for information.
• I also hardly spend time text messaging and talking over the phone (several relatives, including my sister, complains of this).
• I am also a member of ASTD, Social Issues and Serious Games fourms. These discussions happen over mail and therefore, I don't read these anymore. Today, I prefer real time discussions.
  
• Finally, do I neglect my work? No, I manage time better than I did 3-5 years back.

Do you have less of a life with all of these new things?

No, I still shut my computer down by max 7. I don't tweet or read blogs over weekends. My weekends are reserved for my family and myself. Anyway, I don't think people who are online all the time, don't have a life. (Unless ofcourse if you are recluse and unsocial, but who am I to judge ;)) I also think my life is 'more happening' now as I 'know' more people than I did before these new things were introduced.

Expert Usability Evaluation and Learning Audit of an Online Course

You do not have access to learners or you are really short of time. However, you want to find out if your online course is easy to use and learn. What do you do? You can conduct an expert usability evaluation and expert learning audit. Both these evaluation techniques have their roots in usability inspection. Before I proceed further, these techniques have evolved from usability, but have been modified to suit the requirements of the learning field. If you use some other techniques, please do share. We are always eager to learn more effective ways of doing things.

How are these evaluations conducted?
An expert (usability/learning) judges the effectiveness of the course. The expert will go through the course and try everything a learner would if he/she were to take the course. They look for obstacles, ambiguity, functionality, and several other issues that hinder progress. Detailed reports are generated at the end of the evaluation.

What is an expert usability evaluation?
Using this technique, you evaluate the usability of an online course. An expert lists the parameters based on which the evaluation will take place. These could include:

1. Navigation: What is the primary form of navigation? Is this intuitive enough? This would ideally mean that we do not include 'Click Next to proceed' kind of instructions. The learner should intuitively know what the primary navigation is.
   
2. Visual hierarchy: Is the information organization in a logical manner? Eye movement is typically from left to right and from top to bottom. Are all elements positioned keeping this in mind? Will the learner know where the information starts and where it ends?
   
3. Accessibility of information: Are important elements placed upfront? Will the learner be able to access the most important information easily.
   
4. Affordance: Do buttons have the affordance of a click? Will the learner know when a click is required? Will he/she know what is expected on an interactive screen? During learner testing, I have seen learners click images that are not clickable or miss buttons that need to be clicked. This is because the element does not have the affordance of a click. Therefore, it is important to identify such issues.
   
5. Fonts and font sizes: Will the learner be able to read the text easily? Do font colors hinder readability? Are these fonts and font sizes consistent across?
   

What is the difference between a QA and an expert usability evaluation?

• A QA checks whether the online course maps to the signed off storyboard/wireframe. It also checks functionality, consistency, and whether the course has any bugs.
  
• Expert usability evaluation, on the other hand, checks whether the elements in the course are usable. It also takes into account user experience. Does the eLearning application cater to the five principles of usability - learnability, efficiency, memorability, errors, and satisfaction?
  

A QA is more content-centric while an expert usability evaluation is more user-centric. This is the main and the most crucial difference.

What is an expert learning audit?
Using this technique, you can evaluate the learning effectiveness of a course. An expert lists the parameters based on which the evaluation will take place. These could include:

• Learning objectives - content mapping: Are the learning outcomes addressed? Can the content be directly mapped to the learning objectives? Is there more information than is stated in the learning objective?
  
• Learner-content mapping: Is the content specific to the learner profile? Is it relevant? Will it help the learner meet the learning outcomes?
• Learner motivation: Is the course motivating enough for the learner? Why will he/she complete the course? Will they find it interesting? Will he/she be motivated to take an exercise?
• Vizualization: Do the visual elements aid learning? Are they similar in look and feel across the course?
  
• Language: Will the learner understand what is written? Is there any ambiguity?
  

What is the difference between an ID review and learning audit?
By the time the course is developed, several ID reviews have already been done. The learning audit is conducted by a third person who has not been a part of the design phase. Therefore, the course is looked at by a fresh eye and this makes a world of difference. The expert looks at the course without considering the constraints. I believe this is a good thing because several times we compromise on learning because of we are thinking about the constraints. He/she looks to identify the obstacles that will hinder learnability. An expert can help us identify where we have compromised.

Keep the following in mind if you are evaluating:

• Ensure that there are no distractions. This requires a lot of concentration, else you may miss a crucial issue.
  
• Try everything. What is the learner were to click this? What would happen if I go here instead of there?
  
• Use screen grabs to highlight issues. This is helpful as the reader will not have to shuffle between a report (xls, word) and the course.
  
• Include suggestions wherever possible. Provide two or three alternatives if possible. It would be very effective to show a suggestion visually.
  
• Ensure that you mark the repetitive issues as a global comment. But, it is also important to identify all those screens in which the issue is present. This will help save time when the reader is fixing the issues.
  
• If you have set parameters, you could check each screen for each parameter in a logical manner rather than just scan a screen. This way you will not miss anything.
  
• It can become very tedious, tiring, and repetitive. So, be prepared.
  

The reports generated from both the expert usability evaluation and the learning audit is valuable source of feedback. You will be able to identify the things that can be worked on based on the suggestions provided by the expert. Use these techniques to evaluate your online course. Try it once and see how much of a difference it actually makes. But, remember, this is still no match for direct feedback from the learner.