DoubleMap

Usability Evaluation of a mobile application that allows real-time tracking of shuttle services

OVERVIEW

We performed Usability Evaluation to assess the overall effectiveness and experience of the DoubleMap application for its target users. In this project, we identified pain points where users get confused and tasks which they were able to perform with ease. Evaluation methods like cognitive walkthroughs, think aloud observations, and heuristic evaluation was applied to measure the usability of the system. Design recommendations were suggested for all the usability issues discovered throughout the evaluation.

ABOUT DOUBLEMAP

DoubleMap is the web and mobile application designed to track the shuttle services and municipal transit systems across various campuses. Campuses include universities, corporate complexes, hospitals, and airports. The application is available on both Android and iOS platforms.

OUR STUDY OBJECTIVES

The goals of our evaluation study were as follows:

Assess the overall effectiveness of DoubleMap for different types of users
Identify features of the application which users easily learn
Identify obstacles in the application where users struggle
Determine opportunities for design improvements

THE EVALUATION PROCESS

UNDERSTANDING STAGE – TARGET USERS, APPLICATION TASKS & COMPETITIVE ANALYSIS

Target Users

The primary users of the application include students/staff, corporate/company employees, Hospital staff, Airports and Municipal transit users. Bus drivers and campus administrators who manage new and current routes or buses are secondary users of the system. We designed a persona that would sufficiently meet DoubleMap’s majority user base.

Fig. Susan’s Persona

Common Application Tasks

Finding the nearest bus stop
Add favorite stops
Searching for relevant bus arrival timings
Set up push notifications to get notified about bus arrival
Checking for service announcements or any delay in bus arrival due to unforeseen conditions
Provide feedback to the system (if any)

Fig. Typical user task flow

Competitive Analysis

Competitive analysis was conducted with the below competitors of DoubleMap

Transloc Rider: It is a similar app that provides real-time bus updates and announcements. It is used by many transit systems and university campuses
RideCell: It is a company that provides ride/car sharing applications. It also provides services for fixed routes fleet automation. Works similar to Transloc and DoubleMap

TESTING STAGE – CONTEXTUAL INQUIRY, COGNITIVE WALKTHROUGH & HEURISTIC EVALUATION

Contextual inquiry sessions were conducted with some students on the IUPUI campus and some were conducted in their individual apartment. The sessions followed a method where the user performed the tasks and the observer only interrupted in case the user was confused. The following things were recorded:

Amount of time taken to perform a task
Patterns of behavior, pain points, and places where users face difficulty
Alternate approaches to performing a certain task

Fig. Contextual Inquiry sessions

The focus of the cognitive walkthrough session was on understanding DoubleMap’s learnability for novice users. In this session, we went through all the sequence of tasks involved and asked questions from the perspective of the user.

Fig. Cognitive Walkthrough session

For the Heuristic Evaluation, we judged the application based on Jakob Neilson’s ten usability heuristics of interface design. The heuristics can be found here. https://www.nngroup.com/articles/ten-usability-heuristics/

FINDINGS STAGE – QUANTITATIVE & QUALITATIVE RESULTS

Ten participants were recruited for the usability study out of which five were novice users and others were experienced users of DoubleMap.

The time taken by users performing basic application tasks were recorded and plotted on graphs

The overall success rates of novice vs experienced users was measured.

After completion of all the tasks, we asked users a couple questions regarding DoubleMap in the post-task questionnaire. We received the following responses.

Does DoubleMap solve the purpose of real-time tracking of buses?

85% participants agreed that DoubleMap solves the purpose of real-time tracking of the bus locations while 15% disagreed with it.

Did DoubleMap meet your expectations?

85% of the participants agreed that DoubleMap meets their expectation level whereas only 15% of the participants disagreed on this.

Quantitative Insights

The following insights were discovered:

Novice users completed 84% of all the tasks whereas experienced users completed 96% of all the tasks
Finding the estimated arrival time of the bus (ETA) had the lowest completion rate of 70% as compared to all other tasks
On average, new users took 20 seconds longer than experienced users to find the nearest bus stop from their location & selecting a bus route to the desired destination stop
100% success rate was observed for all the participants for finding the IUPUI campus system on the application, finding the nearest bus stop from the current location & tracking the location of their bus

Qualitative Insights

DoubleMap’s overall design was leveraging Google Maps APIs, so most of the users experienced resemblance with google maps quite useful. The following comments were noted:

“App is good. Especially real-time bus tracking”
“Bus Tracking feature is accurate”
“Toggle buttons for selecting/unselecting routes are clear”
“Changing routes was easy”

System Usability Score(SUS)

Based on the calculations, the SUS for the application was determined as 81.25 where individual scores varied from 75 to 87.5. Overall, this score is a good usability score. DoubleMap is usable but has definite scope for improvement.

RECOMMENDATION STAGE – SCOPE FOR DESIGN IMPROVEMENTS

GPS enable prompt

Existing System Design