The web pages for the University of Calgary
library (http://www.ucalgary.ca/library/)
were developed to provide staff and students access to some of the services
offered by library without having to actually come into library itself.
Your company has been hired by the University
of Calgary in order to evaluate the usability of the library web pages. You are to take the role of consultants who
have been assigned by the VP of your company in order to determine if the
existing user group of the library web pages will be able to use the web site
in order to complete some typical library-related tasks. In addition, your VP may be interested in
seeing how users who are new to the system will interact with the site as well.
Your job is to study the site and find any problems that may arise when people
are using the site caused by flaws in the design of the interface. This will be done by observing people who
are using the site doing a set of pre-created tasks and then recommending ways
in which these flaws may be corrected.
Try to get as wide a mix of test participants within this range as
possible (e.g., regular library users who have never used the web site before,
regular and experienced users of the site, someone who has never went into the
library but may be future users etc).
It is up to your team of consultants to come
with a set of typical tasks that should be completed by the test users. The assignment sheet has a section that indicates how you
can go about this and should already be familiar with task descriptions from
Assignment 1. As well, the experimenter should try the system ahead of time,
becoming as familiar with it as possible.
Your group should come up with at least four other reasonable
tasks to give to participants; preferably, you should come up with even
more. A good task one that is likely to
be used by many end-users. Tasks should also be selected to investigate
different (but still important or heavily used) parts of the system
functionality.
Just to start groups out, I have included a
list of some example sample tasks:
Task 1.
Finding the site. You heard from a classmate that the university has created a web page
that allows you access many of their services through the Internet. However, your friend cannot remember the web
address. Use whatever mechanism you
want in order to find the main homepage for the library and display it in your
web browser.
Reason for choosing this task: No matter how good the website
may be designed it is only of value if people can actually find it. The main point of this task is to determine
the different ways that people will employ to find the site (e.g., use a search
engine to find it, type the url in) and how successful they are at it (if they
can easily find it with search tool, if they are able to correctly type in the
url on the first try etc.).
Task 2.
Searching for a book. You are student registered in the introductory undergraduate
Human-Computer Interaction class, CPSC481, this semester. During the first day of class the Prof.
mentioned that there is no required textbook for the course. However he said that there are many optional
readings that will come from the book “Human-Computer Interaction: Toward the
Year 2000”. Unfortunately when you go
to the bookstore you see that the cost of the book will overdraw your budget
for this month – even the prices listed in the used bookstore are just too
expensive for you to afford. But since
you are really keen on learning everything you can about HCI you
still want the book and decide to see if it is available in the university
library and if so where you need to go to find it (i.e., what building it is in
and exactly where in that building is the book located).
Reason for choosing this task: Finding a book in library listing is definitely a core task. However, in addition to knowing whether or not
the book is available the person also needs to be provided with all of the
information that he or she needs in order to find the book (if it is currently
free and if so where to find it).
Task 3: Finding information on a topic. You
want to find out everything that you can about CPSC481. Find out if the library has any recent
textbooks (other than the one listed in the second task) on the topic. Find two textbook titles that you would
think would be worth getting from the stacks.
Reason for choosing this task. This is a core task. However catalogs may not use the same words
for defining a subject so that the person who is performing the search will
have to define their own search terms.
Task 4.
Looking for an old newspaper article. You
heard from your cousin Henry about an interesting article that was written up
in the Montreal Gazette sometime last April (2001). Find out if the library subscribes to the Montreal Gazette and if
so do they have available any newspaper articles from that month/year.
Reason for choosing this task: Quite often when people are
looking for old periodicals they will search for them in their local
libraries. Many publishers (such as
newspapers) do not archive their old stories (or at least they do not make this
information available to the general public).
Either test participants can use a browser from
their own account or you can set up a browser from your own account. If you are reusing a browser (e.g., in your
login account) clear the browser's cache before each session, as you don't want
the link highlights to give the user clues of where to go. Depending on how you
run participants, you may want to have a browser (e.g., Netscape or Internet
Explorer) up and ready at the home page for the library (http://www.ucalgary.ca/library/),
although I recommend that you have people start cold for at least the first
task (e.g., how to find the site.).
The site may be busy or there may be slow
network connections. Slow system
response times are a part of system usability and should not be ignored. (These are the same issues that actual
users of the system will have to face).
Administer the pretest questionnaire. There should be a reason for asking the
questions that you ask – don’t ask for information that have no bearing on the
test and which you won’t be using.
Typical questions in the pretest will ask about the person’s computer
experience, their internet and browser experience, their experience with this
particular web site and if this person actually does use the university library
or other libraries.
At the end of the test, administer the
post-test questionnaire. This second
questionnaire should include questions that ask about people’s satisfaction
with the system both in broad terms e.g., “I would rather use the online
library service rather than go into the library myself” e.g., “I would never
want to use the online system”, and with more specific ratings (Strongly agree,
agree, neither agree nor disagree, disagree, strongly disagree). Also, be sure to administer the
usability instructions to participants, as indicated in the handout.
Note: the reason you are doing this is to see what initial conceptual model
people have of the system, based upon their prior experiences and their
interpretation of the visuals on the screen. You are looking for places where
the model is incorrect. Start doing this as soon as they get to the homepage of
the library.