IADT MSc UX Design Assignment — Prototyping and Evaluation

Back to Assignment Introduction

Prototyping

Paper prototyping is a popular method to rapidly create and iterate designs for user testing, a simple and effective approach before the creation of wireframes, high-fidelity designs, or coding begins (Nessler, 2016).

Nielsen (2003) highlights the value of paper prototyping, estimating that earlier rather than later design exploration has ten times the usability value, and is 100 times cheaper to implement.

Simple and novel ways to replicate familiar mobile paper experiences were explored:

• The initial paper prototype was constructed by leveraging Balsamiq cut-outs and wooden sticks.
• The second and third iterations used pen and paper components to enable more rapid change and easy recording on camera.
• Marvel App augmented the paper prototypes; linking them together in a simulation of a mobile application.

Confirmation message alternatives in Marvel App.

The task for testers to complete was the handing in of school work, asked with a “how would you…?” request. A “tell me more about that…” post-task approach explored more. During the task, participants were asked to talk out loud as they stepped through the screens.

Paper prototyping options. Clockwise from top left: Paper stencils and sticks, paper-based interaction, team paper-based version of a common task, paper-based screens in digital format on smartphone.

Prototyping Video

The first prototype was constructed using cut paper stencils and wooden coffee stirring sticks:

Video walkthrough of an initial design using paper stencils and coffee stirring sticks, also identifying questions for later testing

The prototype task was iterated and using Marvel App published on a mobile phone that could be tested remotely by sharing a link to the app and observed using Zoom.

Hand-drawn paper components assembled into a digital flow using Marvel app

Prototyping Learning Lessons

• Paper prototyping enables rapid and low-cost creation and iteration.
• The simplicity of prototyping avoids the distraction of visual treatments and reminds users what they are seeing is still at the design stage.
• Digital tools are required for delivering prototypes during lockdown restrictions to remote testers.
• Road test the prototype before usability testing with recruited participants.

Evaluation

Evaluating our design using paper prototypes provided for validation by representative users. The collection of feedback data was part of the overall testing process (Goodman et al, 2012).

Testing took the form of a moderated remote usability study (Nielsen, 2014) using a prescribed task, conducted over one week, and exploring opportunities for improvement through iterative designs and testing. (Rubin & Chisnell, 2008)

Testing Metrics

The Microsoft Teams task and the prototype tasks were tested for task completion rates and error rates (effectiveness), task completion times (efficiency), and satisfaction ratings (satisfaction), as the basis of determining usability (Nielsen, 2001).

Participant Recruitment

• A simple screener (Arnould, 2015) was constructed to identify suitable testers from the population.
• A consent form was signed.
• Permission from guardians was obtained.
• The facilitator was vetted and certified as suitable to work with minors (An Garda Síochána, HSE Ireland, 2020).

Screening

Our participants were 16–18 years old, with at least 5 years of digital experience. Each was available in the evening after school, had access to a mobile phone, a laptop, was familiar with Zoom, and had used Microsoft Teams for school.

Recruitment Screener.pdf

Five (5) testing participants were selected.

Testing Method

A test script of introduction, task steps, and closeout was created. The script was aligned with the task steps in the prototype.

Test script introduction

Testing Delivery

Testing was conducted in-person and remotely. A camera and notebook were used to record actions and verbal comments of testers during the task.

Zoom testing. The participant shares their view of the prototype as the task is performed; talking out loud about their decisions.

Prototype Iterations

Throughout testing, the prototypes were modified using collected information and the iterations retested.

• The initial test task was effective, but there was some hesitancy that was explored further: the design use of an Upload button in two successive steps was confusing.

Initial prototype with first participant, exploratory questions included.

• For the first iteration, the second Upload button label was changed to Submit, which eliminated the hesitancy of testers.

First iteration with Upload button label modification

• For the second iteration, the number of steps to perform the task was reduced by two by removing Add File and Submit actions. This iteration also offered a progress bar modal dialog to increase system visibility during hand-in.

Second iteration, showing reduced steps and combined processing/confirmation modal.

Evaluation Learning Lessons

• Decide on a testing methodology and measurable criteria.
• Screen and recruit test participants. Be resourceful during the lockdown and get consent and buy-in.
• Make changes supported by feedback and test iteratively.
• During testing, listen actively, probe gently and restate to clarify, then summarize. (Quesenbery & Brooks, 2011).

Testing Results and Data

The combined testing results and source data have been provided for reference and completeness of the assignment.

Back to Assignment Introduction