Laboratory 12

A/B Testing

A/B testing is a simple way to test changes to your page against the current design and determine which ones produce positive results. It is a method to validate that any new design or change to an element on your web page is improving your conversion rate before you make that change to your site code. - [source]

Heuristic Evaluation

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface (UI) design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles (the "heuristics"). - [source]

Nielsen's Heuristics: Visibility of system status, Match between system and the real world, User control and freedom, Consistency and standards, Error prevention, Recognition rather than recall, Flexibility and efficiency of use, Aesthetic and minimalist design, Help users recognize, diagnose, and recover from errors, Help and documentation. See other heuristics like Gerhardt-Powals’ cognitive engineering principles and Weinschenk and Barker classification .

Usability Testing

Usability testing is a technique used to evaluate a product by testing it with representative users. In the test, these users will try to complete typical tasks while observers watch, listen and takes notes.

Depending on the usability questions that arise throughout the design process we can use methods like: Participatory Design, Paper Prototyping, Walk-through, Surveys, Focus Groups, Heuristic Evaluation, Usability Testing, Quality Assurance to get user feedback.

Usability testing is a black-box testing technique. The aim is to observe people using an application, website, product in order to discover errors and areas of improvement. Usability testing generally involves measuring how well the test subjects performed in these areas:

  • Performance - How much time (how many steps) is required for people to complete basic or given tasks?
  • Accuracy - How many mistakes did people make in completing a task?
  • Recall - How much does the person remember (about completing a task) after periods of non-use?
  • Emotional response - How does the person feel about the tasks completed? (confident, stressed) Would the person recommend this system to a friend?
usability testing

Figure above shows usability testing throughout the product lifecycle.

Example of a usability checklist for websites before and/or after designing the application/product Usability Checklist.

Use Cases

LearnFWD: Getting Started with User Research (Jinghua Zhang & Diane Loviglio) from Mozilla WebFWD on Vimeo.

Developing a Usability Test

Getting Started

How to prepare for usability tests:

  • Develop the test plan - What is the goal of the test? What are the tasks/scenarios/functionalities we wish to test? ;
  • Set up a testing environment - Software installed, Room prepared, Logistics;
  • Find and select participants - Besides participants select observers and moderators.
  • The best results come from testing no more than five users and running as many small tests as you can afford. According to Nielsen, J. (1994). Usability Engineering, Academic Press Inc, p 165

Conducting the Usability Test

How to conduct the usability test:

  • Prepare test materials - Logistics, Room, Questionnaires;
  • Conduct test sessions - According to the test plan;
  • Each participant/group of participants has a series of tasks to complete. The observer note the actions of the user and only intervenes as a last resort

  • Debrief the participants and observers.
  • Follow up on any particular problems that came up for the participant.

    Broad questions to collect preference and qualitative data.

Usability Test Results Analysis

What to do after the usability test:

  • Analyze data and observations;
  • Usability testing recording software to record the computer screen and the participant's voice and facial expressions during testing.

    By analyzing participant's facial expressions, the number of mouse clicks made, and the navigation path used to complete a task, we can identify the most frustrating parts of a task and suggest ways to improve the interface to better support the user.

  • Report findings and recommendations.