Usability Assignment, this year's system: Instant Messenger ™ Microsoft

Your situation

Your group is a team of usability experts employed by the Ace Consulting Company (™) to evaluate the usability of Microsoft's MSN Messenger program (version seven¹).  You have been assigned by the VP of your company to determine if Messenger meets the needs of the people who currently use the system.  In addition, your VP may be interested in seeing how quickly new users learn the use of this service. Your job is to study the program and find any problems that may arise when people are using the system that are caused by flaws in the design of the interface.  This will be done by observing people who are using the program while they are carrying out a set of pre-created tasks and then recommending ways in which these flaws may be corrected.   Try to get as wide a mix of test participants within this range as possible:

• Non-users, people who have never used any instant messenger program (including competitor programs such as Yahoo™ Messenger).
• Novice users, who just use the basic features of MSN Messenger e.g., they only send and receive messages from a list of contacts and they seldom change this list or use any of the other functions.
• Intermediate users who use many of the features available through Messenger (e.g., automatically tracking past conversations with contacts, logging in but appearing as offline to your contacts) but don't know them all.
• Power users, someone who is familiar with and uses on a regular basis all (or close to all) the features and functions available through Messenger.
  

It is up to your team of consultants to come with a set of typical tasks that should be completed by the test participants.  Although I have started all the groups off by providing a list of sample tasks, each group should add to this list.  The assignment sheet has a section that indicates how you can go about this but you should already be familiar with task descriptions from Assignment 1.  In the case of the usability study I suggest that you break down a full task (a complete interaction with the system) into sub-tasks (like the ones that I have below).  Presenting a long and complex task may add extra level of complexity and unnaturalness that is not faced by the typical user and may skew your test results.  As well, the experimenter should try the system ahead of time, becoming as familiar with it as possible.  Your group should come up with at least three other tasks to give to participants; preferably, you should come up with even more.  (Just don't give out so many tasks to each participant that your test takes too long to run).  A good task is something that many end-users are likely to complete with Messenger; tasks should also be selected to investigate different (but still important or heavily used) parts of the system's features.

Here is a list of some sample tasks to start you all out.  Feel free to use them in your study but again make sure that you write up some additional tasks of your own:

Sample tasks:

Note: Due to privacy concerns you may have to have your test participants create a new Messenger account that will be used just for the study.  (Some test participants may not feel comfortable having you see messages from their friends occur come up during the study).   If this is the case, then you have to get things set up for them ahead of time (maybe use a test account for all test participants) and add have the person add members of your group as sample contacts.  Before you run the study ask your test participant if he or she feels comfortable using their own Messenger account (be sure to mention that messages from contacts may come up during the study).  The person must be free to either quit the study or to proceed by only using a test account that you have already created.  Furthermore if some of your tasks involve viewing additional information (e.g., seeing additional personal information about contacts or viewing past archived messages) and if you test participants refuses to let you see this information then you must respect this and either end the study or to drop the tasks that require this information to be seen.

Task 1. Send a message to a contact.  You see that your friend (you should customize this for your test participant) is online.  Say 'hi!' to this person through Messenger.

Reason for choosing this task: Sending an instant message is one of the core functions of the system and is often the primary (or sole) reason for using this program.

Task 2. Change your online status to 'busy'.  You want to stay logged onto Messenger in order to stay aware of the online status of your contacts but now you want to do some work and you don't want to be bothered by contacts who just want to IM chat.  Change your status to busy in order to reflect this.

Reason for choosing this task.  Although Messenger can monitor the activities of the user and try to automatically set their online status accordingly this process is far from perfect.  Nor does the change in status occur instantaneously so there are times that the user may want to explicitly and immediately indicate to others what state that he or she is in.

Task 3. Pass a message to all your contacts through your personal message.  You have some information that you want to pass onto all your friends that is time sensitive in nature so you can't email it because not all of them may check their email in time but because most or all of them stay logged into IM continuously (or at least check their IM more frequently than they check their mail) so by doing it this way you've got a better chance of someone noticing your message.

Reason for choosing this task.  Some IM users never customize their personal message or provide an IM picture.  Others not only change both on a frequent basis but also do so as a means of communication as well as a method of self-expression.  This task can be used to determine how well IM supports this rare, but to some users important scenario, and passes on the information to contacts who always remain logged into IM and the ones that only do so on a sporadic basis.

Preparing Equipment

Test participants can either access Instant Messenger from their own computer (ideal - but they must be comfortable with this) or they can try to access it from your own account (if you find it more convenient to use the computers in the lab). 

Parts of the system to exclude from your the usability study

• Applications that support Messenger but are not a part of the application e.g., games, remote assistance, the 'Today' window that users may see when they login to Messenger, "Windows Live Spaces", all tabs that aren't directly relevant to the use of Messenger (that excludes all of them except for the one to check out who is online)
• May include: video call (if your participants are okay with this and you have a camera), 'Share fun activities' may be okay for some participants but leave out 'My activities'.

^{1 It's important that everyone evaluates the same version of Messenger because each version has different features.  I picked this version because it is the one that is installed in the WinXP labs in Math Sciences.  Make sure that your evaluation is not performed on a different version of Messenger.}