Quauntifying Time Spent on Web 2.0 Tools

Is this a trick question? ;) My response to this month's Big Question:

How much time do you spend and how did you find time for all the relatively newer things like reading blogs, twitter, social networks, etc.?

I use TweetDeck and receive tweet notifications. If I am working on something that requires high concentration, I close this application. Else, I logically close a thought and quickly check what others are saying. If the tweet inspires me, I tweet back immediately. If it is an interesting link and a longish blog, I keep it open and read it during my next break. I blog has to be either really compelling or really short for me to read it immediately. I check Facebook only once or twice a day as I use it only for personal networking. Without a doubt, Web 2.0 makes us more effcient. Everyone values time. When a blogger blogs, he/she tries to keep it crisp. I scan through a blog before i decide if I want to invest time. Twitter is popular because writing/reading short tweets does not take time. Coming back to the question, I will not be able to say exactly how much time I spend networking/learning. It various as per project deliverables, my moods, blog updates, and twitter traffic.

What are you doing less of today than you were 3-5 years ago?

• I used to take longish walks to breath in some fresh air when I was at office. Now, that I work from home, I tweet or read blogs. I don't go through my Google reader as most bloggers I follow share links on Twitter.
• Earlier, I used to surf the Internet for hours together to find information that interests me. Today, I go looking for information only when my research is very specific to something. Most times, information comes to me (courtesy web 2.0). This surely means my search is more effective and productivity high as I spend lesser time surfing the Internet for information.
• I also hardly spend time text messaging and talking over the phone (several relatives, including my sister, complains of this).
• I am also a member of ASTD, Social Issues and Serious Games fourms. These discussions happen over mail and therefore, I don't read these anymore. Today, I prefer real time discussions.
  
• Finally, do I neglect my work? No, I manage time better than I did 3-5 years back.

Do you have less of a life with all of these new things?

No, I still shut my computer down by max 7. I don't tweet or read blogs over weekends. My weekends are reserved for my family and myself. Anyway, I don't think people who are online all the time, don't have a life. (Unless ofcourse if you are recluse and unsocial, but who am I to judge ;)) I also think my life is 'more happening' now as I 'know' more people than I did before these new things were introduced.

Expert Usability Evaluation and Learning Audit of an Online Course

You do not have access to learners or you are really short of time. However, you want to find out if your online course is easy to use and learn. What do you do? You can conduct an expert usability evaluation and expert learning audit. Both these evaluation techniques have their roots in usability inspection. Before I proceed further, these techniques have evolved from usability, but have been modified to suit the requirements of the learning field. If you use some other techniques, please do share. We are always eager to learn more effective ways of doing things.

How are these evaluations conducted?
An expert (usability/learning) judges the effectiveness of the course. The expert will go through the course and try everything a learner would if he/she were to take the course. They look for obstacles, ambiguity, functionality, and several other issues that hinder progress. Detailed reports are generated at the end of the evaluation.

What is an expert usability evaluation?
Using this technique, you evaluate the usability of an online course. An expert lists the parameters based on which the evaluation will take place. These could include:

1. Navigation: What is the primary form of navigation? Is this intuitive enough? This would ideally mean that we do not include 'Click Next to proceed' kind of instructions. The learner should intuitively know what the primary navigation is.
   
2. Visual hierarchy: Is the information organization in a logical manner? Eye movement is typically from left to right and from top to bottom. Are all elements positioned keeping this in mind? Will the learner know where the information starts and where it ends?
   
3. Accessibility of information: Are important elements placed upfront? Will the learner be able to access the most important information easily.
   
4. Affordance: Do buttons have the affordance of a click? Will the learner know when a click is required? Will he/she know what is expected on an interactive screen? During learner testing, I have seen learners click images that are not clickable or miss buttons that need to be clicked. This is because the element does not have the affordance of a click. Therefore, it is important to identify such issues.
   
5. Fonts and font sizes: Will the learner be able to read the text easily? Do font colors hinder readability? Are these fonts and font sizes consistent across?
   

What is the difference between a QA and an expert usability evaluation?

• A QA checks whether the online course maps to the signed off storyboard/wireframe. It also checks functionality, consistency, and whether the course has any bugs.
  
• Expert usability evaluation, on the other hand, checks whether the elements in the course are usable. It also takes into account user experience. Does the eLearning application cater to the five principles of usability - learnability, efficiency, memorability, errors, and satisfaction?
  

A QA is more content-centric while an expert usability evaluation is more user-centric. This is the main and the most crucial difference.

What is an expert learning audit?
Using this technique, you can evaluate the learning effectiveness of a course. An expert lists the parameters based on which the evaluation will take place. These could include:

• Learning objectives - content mapping: Are the learning outcomes addressed? Can the content be directly mapped to the learning objectives? Is there more information than is stated in the learning objective?
  
• Learner-content mapping: Is the content specific to the learner profile? Is it relevant? Will it help the learner meet the learning outcomes?
• Learner motivation: Is the course motivating enough for the learner? Why will he/she complete the course? Will they find it interesting? Will he/she be motivated to take an exercise?
• Vizualization: Do the visual elements aid learning? Are they similar in look and feel across the course?
  
• Language: Will the learner understand what is written? Is there any ambiguity?
  

What is the difference between an ID review and learning audit?
By the time the course is developed, several ID reviews have already been done. The learning audit is conducted by a third person who has not been a part of the design phase. Therefore, the course is looked at by a fresh eye and this makes a world of difference. The expert looks at the course without considering the constraints. I believe this is a good thing because several times we compromise on learning because of we are thinking about the constraints. He/she looks to identify the obstacles that will hinder learnability. An expert can help us identify where we have compromised.

Keep the following in mind if you are evaluating:

• Ensure that there are no distractions. This requires a lot of concentration, else you may miss a crucial issue.
  
• Try everything. What is the learner were to click this? What would happen if I go here instead of there?
  
• Use screen grabs to highlight issues. This is helpful as the reader will not have to shuffle between a report (xls, word) and the course.
  
• Include suggestions wherever possible. Provide two or three alternatives if possible. It would be very effective to show a suggestion visually.
  
• Ensure that you mark the repetitive issues as a global comment. But, it is also important to identify all those screens in which the issue is present. This will help save time when the reader is fixing the issues.
  
• If you have set parameters, you could check each screen for each parameter in a logical manner rather than just scan a screen. This way you will not miss anything.
  
• It can become very tedious, tiring, and repetitive. So, be prepared.
  

The reports generated from both the expert usability evaluation and the learning audit is valuable source of feedback. You will be able to identify the things that can be worked on based on the suggestions provided by the expert. Use these techniques to evaluate your online course. Try it once and see how much of a difference it actually makes. But, remember, this is still no match for direct feedback from the learner.

Informal Learning

Ever since I joined Twitter, my account has been bombarded with tweets on informal/social learning. If you don't know or you need a quick refresher on what informal learning is read What is Social Learning? Is this a new type of learning? The answer is a sure no. Informal learning has been around for ages, even before formal training came into the picture. People have always been exchanging notes to perform better.

Read more about this on my latest post at Learnability Matters: Informal Learning - 7 reasons why organizations must promote it

How important is the SME?

You would have interacted with several SMEs. I have attempted to classify the SME. Five prominent personae emerged. (These are based on my experiences. Please feel free to add your thoughts.)

1. Temperamental SME: This SME loves to throw her weight around. She demands that things be done her way, else she will not cooperate. She dons the role of the boss and tells you when she expects things. She expects you to work only on her project and on nothing else. She throws a tantrum every time she thinks she has been let down.

2. Easy Go Lucky SME: This SME agrees to everything you say. His typical response to a query is 'Sure! Go ahead.' You wonder whether he has actually heard what you asked him. He typically falls in with the process but expects you to keep reminding him to give you time. When he does, he signs off storyboards rapidly which leaves you wondering whether he really went through them.

3. Absentee SME: This SME is never around to share his thoughts. Whenever you call him, he is busy. He needs a call from a 'higher up' to ensure that he spares time for you. Eventually, when he is forced to spend time with you, he passes on his resentment of the situation onto you.

4. Doing Your Job SME: This SME thinks being an ID is no big deal. She thinks she knows the best way to teach and present things. She typically focuses more on how content can be presented rather than on the accuracy of content. This storyboard is her baby, you change anything and she freaks. You are only to make those changes that she suggests. She tries to design your course for you and you end up feeling like an assistant.

5. The Perfect SME: This SME respects timelines, works with you as a team, takes the learner's motivations and needs seriously, and gives valuable feedback. He researches and pulls out the best stuff to help you understand and transfer the thoughts to the SB. He encourages you to call anytime you get stuck.

In every ID's blog, you will find at least one post on interacting with the SME. Most of these posts cover in detail the problems they face with SME. In every discussion forum, you will find SME interaction listed as an important skill that any ID must master. Why is the SME so important?

1. SME is a library of information. He/she is an expert in the domain and has the knowledge that will make your training program effective for the learner.
2. SME can ensure that your course is relevant to your learner. In most situations, the SME is in the best position to share the learner's real life situations and happenings.
   
3. When the content is highly technical or unfamiliar, the SME becomes your walking stick. You have to interact with him/her to get comfortable with the content.
   
4. SME will always ensure content accuracy. This is really important. You might as well not teach covering something incorrectly.
5. From a sea of information, the SME helps decide what is absolutely necessary. SME can help prioritize topics and concepts.

Regardless of the SME's persona, you need to ensure that you have a process in place. Remember to keep these in mind:

• Share the schedule with the SME. Let them know a day in advance that you are going to send them something. They can plan their reviews accordingly.
• If you are send them two or three things, clearly let them know which ones you expect to receive first.
  
• Most first time SME are not sure of what they need to do. Define their role clearly. If you send then a content dump, let them know what you expect from them. Let them know that they need to provide or validate examples.
  
• It is important to explain the concept of a sign off. Ensure that they understand that if the TOC is signed off, revisiting it at a later stage would mean a scope change.
  
• Build a rapport with them. You can going interact them for a long period of time. Ensure that this time is pleasant and fruitful for both.
• Seek their opinion. Treat them like an expert.
  
• If you do not agree with a feedback, discuss. Share your thoughts and concerns and hear them out. Never fix anything just because you have been told to do so.
  
• Ensure that the SME also always keeps the learner in mind. Ask questions like 'Will the learner understand this?', 'Will the learner find this interesting?', 'Will the learner need this information?'
  

Exercises that help reflect on gray areas

Think about this:

1. The objective is to get your learners to design better forms. The exercise shows a form in which the user wants to change his password. The learner can add/modify elements in the form to ensure that the errors are minimized.
   
2. The objective is to let the learners know that consumers are loyal to particular brands for specific reasons. The exercises makes the learners reflect on their own brand preferences and their reasons for it.
   
3. The objective is to let the learners understand their personality style and their traits. The exercise requires them to respond to particular situations. Their personality traits are displayed based on their responses.
   

Can these kind of exercises have one correct answer? The answer is no. In most instances, the answers are bound to be subjective and/or there are several correct answers. So, do you avoid these completely? No. These exercises are extremely powerful. They make the learner think, reflect, and respond based on their experiences and knowledge.

How may times have you been told by a SME or a client that all exercises must have the right and wrong answer? I have heard this often. Why is it so important to tell the learner, 'That's correct/That's incorrect.' Everything cannot be classified under black or white. As in most cases, there is a lot of gray area. Why do we get scared of designing exercises for this area? I had an interesting discussion on twitter with @thoughts and @manishmo.
--------------------------------------------------------------------------------------------
@rnarchana: All exercises must have RIGHT answers. Why this mindset? There are times when there are no right/wrong answers. The aim to make them think!
@thoughts: @rnarchana Exercises without correct answers can be an activity; perhaps not an assessment.
@rnarchana: @thoughts 1. exercises that teach/reinforce 2. exercises that assess the understanding. With quesns that have no rite ans, u cannot assess.... Just elaborated on your thought I guess. What I am trying to say is 'Activity' is a powerful learning tool too
@manishmo: @rnarchana So what do you want the learners to think (exercises w/o rite answers)? There's always a direction you are trying to push toward
@rnarchana: @manishmo Yes. Ex: Exercise that reinforces that we form perceptions based on material things (cars/clothes) is subjective; no right ans
@manishmo: @rnarchana But even in this case we want to move learners to the direction of "don't form perceptions".
@rnarchana: @manishmo Actually no. This exercise was only meant to make the learner conscious of the perceptions they make unconsciously.
--------------------------------------------------------------------------------------------

This discussion encouraged me to blog this to get my thoughts on the matter together (Thanks Manish and Geetha). Further, I had this interesting discussion with Geeta (my boss and mentor). I was explaining how the SME had requested that we insert a right/wrong answer feedback even though there were none.
Me: Why does everyone assume that learners have absolutely no prior knowledge and experience that they can use to make an approriate reponse? Why do we not encourage them to think and reflect?
Geeta: This is because most SMEs and IDs still promote the educational system followed in schools. The learner is not allowed to think. They treat adult learners as K12

Why is it so important to control everything? We should start letting the learner take responsibility for his/her own learning. This does not go to say that we should avoid exercises that provide correct answers. Exercises can play one of the following roles:

1. Exercises to teach, reinforce, and reflect on a concept
2. Exercise that check the learner's understanding

Use a healthy mix of these in your training program. As Geeta rightly suggested, we should look at opening the learner's minds to new possibilities rather than restricting them to what we suggest is the right answer. When using exercises without right/wrong answers, remember:

• The aim is to make the learners reflect on a concept based on their experience and knowledge. Their answer is bound to be very subjective.
  
• Do not use this assuming that that will come up with the right answer that you have in mind. This may not be necessarily true.
• Use these are teasers. The learner may get curious and conduct their own search online to get their thoughts in order or even better discuss with a peer.
  
• Provide all the information that the learner may require to make an informed decision.
  

I am working on a really challenging project. We have included several instances where we leave the learner to come up with the answer. We encourage them to share their answers, thoughts, and responses with their peers and experts in a social networking forum.

Can Online Learning Environment Kill Motivation?

Ros Aini posed a question in my previous post, "Online learning environment can demotivate online learners. What do you think are the aspects that contribute to this matter?"

I am going to try an attempt answering this question based on my experience and what I have read/heard. If you think you disagree, please help me learn more by sharing your thoughts. If you agree and would like to add and give more clarity, please feel free to do so.

First let me clarify, online learning environment can demotivate online learners is a generalization. What aspects of online learning environment kill learner motivation?

1. Choosing the wrong learning environment for your learner
Your research should help you identify which is the most suitable learning environment in which your learner will learn. A virtual world such as second life may not be suitable for all online learners. You may realize that your learner prefers to read/write than actually 'be' in an environment where they can explore.
Tip: Always arrive at a solution (whether online or otherwise) based on research. You will be closer to getting it right.

2. When the learner control is zero
When you opt for a learning environment, ensure that it is designed such that the learner can decide their own learning path. Most online learners prefer to decide what they learn and how they learn. Avoid forcing your learners down a structured learning path. Several clients insist that the Next button be disabled till they attempt a practice. Give them the choice. If the practice is challenging enough, they are going to want to take it.
Tip: Give them several options such read, listen, do, experiment, share, and so on to learn.

3. When navigation is poor
Navigation plays an important role. Think about this, you have used a virtual world (VW) to teach. Your learner spends ages learning how to navigate within this VW and is not comfortable with the controls. Finally, he gives up frustrated about the fact that he is not getting it right. Poor navigation makes the learners feel dumb. And, no one enjoys this feeling. Navigation can kill motivation even if the training material is really good.
Tip: Ensure navigation is intuitive. The learner should spend minimum or no time learning how to navigate in the learning environment.

4. When system specifications are not shared upfront
Imagine this. You have provided several hyperlinks to blogs and wikis. The organization in which your learner works provides restricted Internet access. The learner tries to click on the link, but gets a 404 error. How frustrating would this be for the learner?
Tip: Always state the system and bandwidth requirements. Or design keeping the learner's bandwidth in mind.

Other than these, what else could demotivate an online learner?
- A know-it-all-peer who makes the learner feel very small, thereby making him reluctant to share his thoughts online again
- The learner may constantly doubt the authenticity of content and feel confused about what to internalize
- No access or an opportunity to interact with 'true' experts
- A learner may feel overwhelmed with the amount of information that is out there

How to tackle a demotivated learner?

A demotivated learner is any IDs nightmare. Don't we love it when learners are highly motivated, thirsty for knowledge, and realize the 'what is in it for me' even before the course begins?

Symptoms of a demotivated learner:

• During interviews, they frown over the concept that you are going to teach.
• They do not appreciate the fact that HAVE to take the course. (I can't blame them.)
• They try to convince you that they know everything they need to about the concept (you know otherwise through research).
• They tell you that this course is not going to be useful for them as this concept will not help hone their core skill.
• They are against the mode of delivery (elearning, ILT, or plain training) because of bad experience in the past.

The situation is very, very dangerous if you spot all the five symptoms in your learner profile. How do you tackle a demotivated learner? Find answers for the following:

1. What can I do to ensure that the learner takes the concept seriously? How can I make it interesting for them?
2. How can I show them that they do not know everything they need to know about the concept?
   
3. How can I show them that this concept if mastered is going to help them work better?
   
4. How can I change their bias against the mode of delivery?

To answer all these questions, I would first recommend that you have a nice long chat with your SME. This is important because my suggestions are going to require the SME's approval.

Suggestion 1: Challenge the learner
The learner thinks he/she knows all there is to know about the concept. Well, challenge him/her then. Design a very hands on course. The intention is to make the learner perform tasks designed keeping the theories of the concepts in mind. We want the learner to realize for themselves, 'Oh! I did not know that' or 'This is probably more effective than how I have been going about it'.

Suggestion 2: Do not bore them with theory
Please do not tell the learner blah blah blah. You will lose the learner even before the end of the screen. They do not want to hear the theory. Stick to pure application. It is definitely easier to describe a theory. Try teaching a theory with absolutely no words. Use tasks and examples. This is incredibly challenging for the ID and SME. But, trust me, the experience is absolutely worth it.

Suggestion 3: Ensure that your course is visual
This learner profile is not going to read anything more than two lines. Avoid content. Make it visual by displaying images/animations of examples. I avoid content heavy screens by using a bigger fonts. Try it sometime.

Suggestion 4: Encourage social learning
Introduce videos from YouTube, share blog links, create or encourage learners to join discussion forums. Do all that you have to and bring them in contact with other people. Encourage them to share their ideas with peers and experts. I do not have to stress the important of social learning.

Suggestion 5: Design challenging knowledge checks
Design the tasks keeping in mind the learner's reality. Make them curious about things. Ensure you grab their attention. Ensure that the answers are not obvious. Design assignment that have no correct answers. Encourage them to post reports or assignments on blogs/forums. Allow them to discuss their ideas and answers with others.

These are my list of suggestions. If you think there are other suggestions that help tackle a demotivated learner, please share them.