REBECCASCAFEMOBILEUSABILITYTESTING

EXECUTIVESUMMARY
This study was conducted to find out what people think of the design and functionality of our
Rebeccas Cafe app. The app allows users to check the open hours and order and pay for a
sandwich, without having to physically go there. We tested three Northeastern University
student that are representative ofourtargetpopulation.Verbal consentwasobtainedfromeach
participant before each session.Wechosenottodoademo oftheappatthebeginningofeach
sessionbecausetheappshouldbesimpleandintuitiveenoughtousebasedonprevioususeof
other ordering apps or websites. Each participant completed three tasks, the first involved
finding out if Rebeccas is open at a certain time, and the other two were different paths users
could take when ordering and paying for a sandwich. The study took place in the apartment of
one of the researchers at between 6pm and 10pm, as researchers had to accommodate to
participants schedules. One of the researchers was the facilitator and the other two were the
note takers. All participants successfully completedalltaskswithveryfewerrorsand reporteda
highsatisfactionandeaseofuse.

METHODOLOGY
Whowetested
Threeparticipants,havingthefollowingcharacteristics,evaluatedRebeccasCafeMobile.

Age

Major

24
1
HealthSciences
1
21
1
ComputerScience
1
20
1
ProjectManagement 1
Total(participants) 3
Total(participants) 3

SchoolYear

Gender

Middler
2
Female
2
Grad
1
Male
1
Total(participants) 3
Total(participants)
3

PhoneUse

OrderingAppUse

Daily
3
Onceamonth
2
Total(participants) 3
Weekly
1

Total(participants)
3
GotoRebecca's

Rarely
1

Onceaweek
1

Twiceaweek
1

Total(participants) 3

Whatparticipantsdid
Participants met with the researchers for about 10 minutes. They completed three tasks using
the app. The link to the app was emailed to them, so they could open it on their phones, and
complete the tasks there. After completing all the tasks, the facilitator asked the participants
sociodemographic,satisfaction,andopenendedquestions.

Thefollowingbriefingwasreadtoparticipantsatthebeginningofeachsession:

Hi,we'redesigninganappforRebecca'sCafe.Thisappwillallowyoutocheckthehoursthe
cafeisopen,aswellasorderandpayforasandwichwithouthavingtophysicallygothere.
Informedconsent:
Weareconductingastudytofindoutwhatpeoplethinkaboutthis.Wewill
notrecordorpublishanyinformationwithyourname.Thisisforacourseweretakingin
HumanComputerInteractionfromProf.BickmoreintheCollegeofComputerandInformation
Science.Yourparticipationisvoluntaryandyoucanstopanytimeandaskthatyourdatanotbe
used.Itshouldtakeabout10minutesandwewillcompensateyouwitheternalgratitudeanda
smile.Canyouhelpusoutwiththis?
Great.Thepurposeoftodayssessionisforyoutohelpusfigureouthowuserfriendlythis
mobileappis,sowecanfixanyissuesaswefinishdevelopingit.
Wellgiveyousometasksthatwethinkarerepresentativeofwhatpeoplemightdoinreallife.
Yourjobistotelluswhatmakessense,whatsconfusing,whetheritworksthewayyoud
expectitto,andanyquestionsthatcometomind.Yourquestionsareespeciallyvaluable,butI
maynotanswerthemrightawaybecauseourgoalistochangetheinterfacesoitanswers
them.
Keepinmindthatweretestingtheinterfacewerenottestingyousoifyourunintoany
problemsitsnotyourfaultanditmeansthattheressomethingweneedtochange.
Areyoureadytostart?
OK,heresthefirstthingwedlikeyoutodo.Takeaminutetoreadthisandletmeknowifit
makessense.Ifso,thenwheneveryourereadypleaseshowuswhatyouwoulddofirst.

Whatdatawecollected
During each task we collected number of errors, time to complete task, and think aloud notes.
Afterthethreetaskshadbeencompletedwecollectedthefollowinginformation:
Sociodemographic: Age, gender, year, major, phone use, online ordering app use, how
oftentheygotoRebeccas
Interview questions: Satisfaction rating (scale from 17, 1 being very satisfied and 7
being not satisfied), ease of use rating (scale from 17, 1 being very easy and 7 being
very difficult), anything that the user found confusing, anything the user would change,
would the user prefer the app or going to Rebeccas, what else the user would like to

see on the app. These questions were followed up with other questions whenever the
testersthoughtitnecessarytodoso.

TASK#1:
You have class until 2pm this coming Wednesday. Youre trying to figure out if you have time
forRebeccasafter.HowlateisRebeccasopenonWednesdays?

Numberofparticipants:
3
Percentsuccessful:
100%
Numberoferrors:
Mean:0
StandardDeviation:0
Timetocomplete:
Mean:3.67sec
StandardDeviation:2.31

Findings:
Allparticipantswereabletosuccessfullycompletethetaskinlessthan5seconds.

Recommendations:
None

TASK#2:
You get out of class soon and you want a sandwich. Order a BBQ pork sandwich with no
cheese. Add lettuce. Youwanttopickitupassoonasyoucan,andyoull paythere. Youllbe
theonepickingitup.

Numberofparticipants:
3
Percentsuccessful:
100%
Numberoferrors:
Mean:0.67
StandardDeviation:1.15
Timetocomplete:
Mean:69.67sec
StandardDeviation:39.12

Findings:
1 participant added the sandwich to her cart multiple times because she thought that
clicking Add to Cart hadnt worked. This is a major problem (lack of sufficient
feedback).

Recommendations:
Show a popup when an item has been added to the cart, or have a cart icon on the
navbar that shows how many items are in the cart. The latterwouldbemoreexternally
consistent,sothisistheoptionwewouldimplement.

TASK#3:
Its8:00ambutyouknowyouonlyhavealittletimeforlunchafteryour10:30class.Youorder
aCapresesandwichasis,tobepickedupatnoon
(byyou).Youwantchipsandcoffeewithit.
Youwillpaywithacreditcardonline(thenameisyourown):
CardNumber:5555555555555555Expdate:10/2017SecurityCode:555

Numberofparticipants:
3
Percentsuccessful:
100%
Numberoferrors:
Mean:0.33
StandardDeviation:0.58
Timetocomplete:
Mean:65sec
StandardDeviation:17.32

Findings:
1 user had filled out the payment section when she realized she had forgotten to adda
drinkandside.Thisisaproblemofrememberingthetask,ratherthanofthewaytheapp
is designed, since she knew where to find the drinks/sides once she realized her
mistake.

Recommendations:
None

EXITQUESTIONSANDSAMPLERESPONSES

Wasthereanythingthatwasconfusing?
1,2)No
3)Knowingwhensomethinghasbeenaddedtothecart.

Ratetheeaseofuseoftheappfrom17,1beingveryeasy,7beingverydifficult.
1,2)1
3)2

Rateyoursatisfactionwiththeappfrom17,1beingverysatisfied,7beingnotsatisfied.
1)1
2)3
3)2

WouldyouchoosetogotoRebeccasorusetheapp?
1,2,3)Usetheapp

Whatelsewouldyouliketoseeontheapp?
Addfavorites
GettingRebeccasdeliveredwouldbeadream