Lecture Topics in HCI, by Saul Greenberg
Contents for:	All Topics	CPSC 481	SENG 609.05	Industrial Course

Evaluating Interfaces with Users: Quantitative Methods

This section introduces quantitative methods for evaluating interfaces with users, concentrating on experimental design, controlled experimentation, and simple statistics.

An assignment on quantitative evaluation gives students practise in a controlled experiment.

Overheads

• Quantitative methods: [PDF] or [Powerpoint]
• Analysis of Variance [PDF] or [Powerpoint]

Topics Covered

• Quantitative evaluation methods
  • User performance data collection
  • Controlled experiments
  • The experimental method
  • Planning an experiment
  • The value of statistics
  • Example: T-test
  • Significance levels, types of error
  • Other statistical tests
• (Optional) Analysis of variance
  • terminology
  • factorial design
  • case study

Readings from Baecker Grudin Buxton and Greenberg

• Chapter 2: Design and Evaluation, p.73-91. The sections directly relevant to quantitive evaluation are c) Controlled experiments.
• Methodology Matters: Doing Research in the Behavioural and Social Sciences, p.152-169, covers many fundamental issues in empirical research methodologies.

Videos

Touch Typing with a Stylus, shows a pen-based character input system. This is presented as a radically different alternative to the screen-based keyboards evaluated in the example assignment.

• Touch Typing with a Stylus, by Xerox (1993, SGVR 88)

In-Class Teaching tips

In-class quantitative controlled experiment. In a short HCI course I once offered, there was no time for students to do the related assignment. Instead, I run a subject in class through a controlled experiment to give people an idea of what they have to do (their task was mouse-typing on two soft keyboards with different layouts, specified in the assignment 1). Students then run each other as subjects out of class (since the software is set up for them, it takes them only 15 minutes). They hand in the data, and we analyze and interpret it in a later class.

Major sources used to prepare lecture material

• For quantitative evaluation, the reading Methodology matters provides good general background. Other sources I used include general psychology statistics books, and chapters in HCI textbooks dealing with controlled experiments (e.g., Chapter 11.5.1 in Human Computer Interaction by Dix, Finlay, Abowd and Beale).
• The research planning chart came from an old CHI tutorial, but I've since lost track of who actually created it.