Data gathering methods during the usability test

Get Complete Project Material File(s) Now! »

Usability testing

Usability testing is regarded as a core method for assessing and evaluating a service. According to Rubin and Chisnell (2008), the fundamental goal of usability testing is to gather data that would help to analyse and identify all the flaws in a service in order to improve it and implement adjustments. In other words, usability testing is a process that engages people as test participants who represent the target audience to evaluate the degree to which a service meets the usability criteria. Albert and Tullis (2010) identified ten most common usability study scenarios with respective metrics that they can be assessed with. According to them, the most valuable attributes for evaluating usability are performance metrics: task success, task time, errors, efficiency and learnability. Nevertheless, traditionally usability tests are conducted on single UI due to lack of assessment methods. In consequence, Majrashi (2016) for the purpose of his study developed an assessment model for evaluating the cross-platforms usability on multiple UIs through measuring efficiency, effectiveness, satisfaction, productivity and continuity.
Based on the relevant scenarios identified by Albert and Tullis (2010) and assessment models developed by Majarshi (2016), this study attempted to assess usability on cross-platforms by comprehending the traditional metrics used in usability testing with metrics that would allow to measure usability when users switch between devices.
Correspondingly, four relevant metrics measuring users’ experience were chosen based Findings and analysis on the design of usability test: task success, task time, efficiency and an additional metric, continuity, which is an essential component in cross-platform usability (Wäljas et al., 2010).

Assessment metrics

The metrics used in the study with respect to cross-platform tasks that were assigned to participants are described below (See Table 1). The descriptions of metrics and usability study scenarios are based on tables which were presented in the book by Albert and Tullis (2010) and in the research paper conducted by Majrashi (2016). x – metrics that measures the scenario

Task success

Task success concerns completing the task and achieving the assigned goal. The metric can be used in usability study that requires accomplishing any task. Prior to measuring the success rate of the assigned tasks, the criteria for task completion were predefined.

READ Approaches to solve mathematical programming problems

Task time

Refers to the time spent on a task completion. The participants’ time is measured between the start and the on the task, measured in minutes and seconds. In this study it was recorded when screen recording the usability study.

Efficiency

Efficiency is the amount of effort needed in order to complete a task, which is usually done by measuring the actions the participants take while performing a task. There can be many forms of actions, such as clicking on a link, pressing the button on mobile or turning on the switch. Each one of the actions taken by the participant defines a certain amount of effort. The more actions taken means the more effort needs to be involved.

1 Introduction
1.1 Background .
1.2 Problem Statement
1.3 Purpose and research questions
1.4 Delimitations
1.5 Outline
2 Theoretical background
2.1 Definition of Cross-Platform Service
2.2 Cross-Platform Usability and Task Continuity
2.3 The difficulty of designing and evaluating Cross-Platform Services
2.4 Information structure and organisational systems .
2.5 Conclusions
3 Method and implementation
3.1 Research design .
3.2 Usability testing
3.3 Assessment metrics
3.4 The design of usability test
3.5 The choice of websites for usability test .
3.6 Data gathering methods during the usability test
3.6.1 Think Aloud Protocol .
3.6.2 Screen recordings
3.7 Questionnaire .
3.8 Selection of participants .
3.9 Ethics
3.10 Validity and reliability
3.11 Data analysis
4 Findings and analysis
4.1 Think Aloud Protocol Analysis
4.2 Screen recordings analysis
4.3 Questionnaires analysis
4.3.1 Consistent and inconsistent websites
5 Discussion and conclusions
5.1 Discussion of methodology
5.2 Discussion of findings
5.2.1 Research question 1: How does consistent and inconsistent design
on cross-platform services affect its usability?
5.3 Conclusions
5.4 Further research
6 References
7 Appendices .