University of Sunderland

Assignment

CDM300

Human-Computer Interaction
Assignment2

Mwangi Moses Mwangi

ID: 059-057-796

Usability Evaluation of Walmart.com by use of

Contextual Inquiry
January 2008
Table of Contents

1. Abstract ........................................................................................................3

2. Introduction .................................................................................................3

3. Justification of Evaluation Approach .......................................................4

4. Evaluation Consideration ...........................................................................5

5. Evaluation Process.......................................................................................6

6. Reporting......................................................................................................6

7. Conclusion ...................................................................................................10

8. References ....................................................................................................11

Appendix
A: User profile ........................................................................................12
B: Tasks ..................................................................................................14
C: Questionnaire......................................................................................15
D: Observation Check list........................................................................17

The report contains 1992 words excluding references and appendix.

2
Abstract
This paper examines human-centred evaluation and critically evaluates the various user-
centred evaluation methodologies, and applies the contextual design methodology to
evaluate the effectiveness of walmart.com in achieving user acceptability and usability of
the website. Walmart.com is a subsidiary of Wal-Mart stores than enables customers shop
online and sells a wide variety of merchandise and provides shipping of the goods to
specific regions such as the Unites states. Contextual inquiry technique has been applied
in because of its efficiency. In this specific evaluation it was applied because it brought
real issues faced by customer while shopping online since the users are given an
opportunity to explore Walmart.com and shop online. From the evaluation it was clear
that website has a very good user interface and was easy to navigate through the website
given the well structured menus however most users felt that the website was a bit too
crowded and took time to open. This could have been influenced by the poor reception
and weak signal at the shopping mal. This could be improved by choosing an alternative
internet technology to provide a faster and reliable internet technology. The results of this
evaluation could be used in by Walmart.com to productively to improve the website
thereby improving the effectiveness of the website in terms of access and utilization.

Introduction
The internet has revolutionised and greatly influenced business activities making it
possible to purchase and sell goods online. The internet is a global information system
that is logically linked and is able to support communication, linking people, computers
and information (Henning, 2007). Through the use of website, a collection of web pages
accessible through the internet, Walmart.com enables shoppers shop online. Great
benefits are achieved through electronic commerce such time saving, great discounts,
endless choice as customers been able to orders goods from their home and pay through
credit cards. The usability of a website greatly influences customer’s retention as well as
the user’s perception of the company trading online thus making usability an essential
component of profit and return on Investment (Frank, 2007).
Human centred usability valuation tests the usability, functionality and acceptability of an
information system and may be conducted in a controlled environment such as a lab or in

3
the field. Several Usability evaluation methods exist such as user testing, heuristic
evaluation, cognitive walkthrough, action analysis, think aloud and field observation,
questionnaire among others. Each of the method has strengths and weakness based on
several factors such as stage of system under evaluation i.e. coding or implementation,
and the objective of the evaluation. Various evaluation methods were identified and their
appropriateness accessed based on the strengths and weakness expressed in each method.
In this report contextual inquiry was selected due to the extensive user involvement
among other strengths. Contextual Inquiry is a user centred approach aimed at collecting
information on how a customer interacts with an information system through the user
interface.

Justification of Evaluation Approach

The evaluation conducted was aimed at assessing user experience while purchasing
goods through walmart.com and identifying problems experienced by the user and thus
evaluating the general usability, acceptability and accessibility of the website.
The factors below were used as a bench mark in selecting contextual inquiry as an
appropriate evaluation method (Alan et al. 2004, p.357).
 Development stage of website- The website site already exist and this made it
possible to involve users who participated in the actual purchase of merchandise
The need for observation - User testing can identify very specific areas that need
improvement or change (Methods: User Testing, 2005).When testing usability
problems nothing is more convincing than watching the user experience difficulties
while performing tasks such as purchasing goods on the internet.
Need for objectivity – Contextual inquiry through use of a tasks scenarios ensures
objectivity in the evaluation thus providing a direct means of assessing and
evaluating users opinion and preferences. Data is obtained quickly easily and
inexpensively (Subjective vs. objective, 2000).
Cost implication – The resources needed to perform the contextual inquiry using task
scenarios approach were easily available as compared to other methods such as
protocol analysis and cognitive walkthrough which required expertise as well as
more finances as compared to the use of user testing approach

4
Non Biased results - Being non intrusive contextual inquiry reduced the probability of
influencing user behaviour and getting biased results (Alan et al. 2004, p.359). The
approach allowed the user to interact with the Walmart.com in completing the tasks
given.
Quality of data - Through use of a task scenario with each task having goals enabled the
collection of qualitative information such as assessing the time taken to purchase an
item from Walmart.com.
Quantity of data - A number of people can be included in contextual inquiry providing
quantitative data.

Evaluation Considerations
A group of 15 users were sampled from a population of 60 aged between 24 and 40 years
based on the level of education and economic power. Those who had tertiary education of
college level and above and were working were included in the study. The users were
both male and female with diverse literacy levels and had a working knowledge of
English and experience in using the internet.
User testing was conducted on a Core 2 dual 2.16GHz Toshiba Laptop with 1GB of
RAM. Windows XP professional was running on the Laptop and Internet explorer was
used as the web browser. A Nokia N70 was used as a modem to connect the Laptop to
the Internet using Celtel Kenya a GSM carrier as the Internet provider. The user testing
took place at a shopping mall (Village Market) with each user taking approximately 1 and
half hours to shop online and fill in the questionnaire. The entire exercise took a total of 8
days and was carried out between 9 am to 8pm.
The users were required to perform five tasks below on the website www.walmart.com
1. Register as a new user on Walmart.com
2. Select a specific item form the website, a brown men’s leather jacket.
3. Try to purchasing the selected item from task 2
4. Edit their profile information such as username and password
5. Get online help

5
Evaluation Process
The study began by sampling from a group of 60 people, to arrive at a representative
sample of 15 people following the criteria outlined above. A pilot test was conducted to
check the effectiveness of the tools developed i.e. questionnaire and the observation
checklist in meeting the goal of the evaluation. The selected sample of users were then
given access to the Laptop and requested to perform the documented tasks. Each task was
careful observed with difficulties faced by the user being noted down. After performing
the tasks defined, a post-test questionnaire was issued to the users who were then were
requested to fill in a questionnaire to assist in gathering more information not noted in the
observation. The Questionnaire was pre-tested on 3 users, and appropriate revision made
before finally issuing it to the test sample user to find if it was appropriate for the
evaluation. Observation was also used in the study to bring out the usability problems
faced by the user in the test.
A usability report was generated based on the study. The report includes strengths,
weaknesses as well as recommendations to the problems experienced and expressed by
the users. Common problems faced by the users were accessed and raised as issues that
needed consideration in each of the performed task.

Reporting
The results of the usability evaluation of Walmart.com were compiled and are
represented as a summary of common and general concerns raised by the user and as
observations and recommendation based on the questionnaire filled by the users as well
as the observation checklist filled as the users were shopping online. Below are the
observations and recommendations made from the evaluation.
Registration of a new user on Walmart.com
Most users were able to create a new account without hesitation with two of the users
asking for help in creation of new accounts this was attributed to the small and unclear
fonts used in creating a new account. Three of the user’s related new account with
financial account rather then a user account which would be seen as misleading
information.

6
Majority of the users felt a better word such as “new user” should be used instead of
“new account”. The suggestion seems reasonable. Alternatively “sign up” can be used as
it is commonly used in email accounts.
Selection of a specific item form the website, a brown men’s leather jacket
In the selection of the specified item most users found it easy to navigate through the
website by the use of the well designed drop down menus however most users were
unfamiliar with the term “apparel”. One of the users used the find text box to look for the
item that took less time as compared to the others. Two entries on the drop down menu
apparel indicating men (men and men’s Big & small) made it confusing for some users.
This could lead to a user spending more time and could lead to poor customer
satisfaction.

The users suggested “men” and “men’s big & tall” be put under the one menu name and
suggested “men” be used to avoid confusing users. A few of the users suggested that an
alternative name be used instead of “apparel” such as “attire” or “clothing”.

7
Purchasing the selected item
A few users had a problem in adding the selected items to the shopping cart largely
because they overlooked some aspects such as choosing the size of leather jacket. The
website provided an error message as seen below preventing the users from going to the
next stage. On placing the item on the cart it was straight forward on the next step
(proceed to checkout), however a few were confused by the term continue shopping

To avoid confusing the users with the button labelled “continue shopping” an alternative
should be used such as “Add to shopping” or “more shopping”. The button “add to cart”
should be disabled until all 2 steps of selecting size and quantity are complete.

Editing of user profile information such as username and password

8
Most of the users had a problem with identifying the link to changing their user profile.
This was attributed by the small and unclear fonts and the some of the user’s inability to
interpret the term “my account". One of the users questioned the appearance of the link
that seemed greyed out.

To make it clear to the users the fonts used for “my account” should be enlarged and the
name change to “user details” to make the language easier. The link should not be greyed
out so as not to look inactive.
Getting online help
None of the users was able to get online help and caused a lot of frustration when the
users were requested to enter the correct telephone number even when they did key in
their phone number.

An input mask should be used to guide the user in keying the right digits and give
allowance to other countries telephone number or alternatively make phone number
optional.

Conclusion

9
User testing identified both specific and common usability problems identified
throughout the evaluation. Including users during evaluation of website usability brought
out real issues experienced by users out in the market while through Walmart.com.
User involvement during evaluation of information systems such as websites on a
constant basis is important since the user has needs and expectation that keep changing. If
these needs are not meet the customer will soon get an alternative website that meets their
need.
One draw back of the evaluation was the use of one Laptop, using the same operating
system and same browser which does not represent real life scenario with the many
available options available. While this meant there was consistency with the platform
used diversity that the users’ world wide enjoys is compromised.

References

10
Alan Dix, Janet Finlay, Gregory D. Abowed, Russell Beale (2004). Human-Computer
Interaction, Third edition. New York: Prentice Hall

Frank Spillers, 2007 ‘How exactly is website usability, customer retention and brand
perception linked?’ Retrieved December 14, 2007 from
http://www.experiencedynamics.com/articles/articles/0603/

Henning Schulzrinne, 2005.’ Internet Definitions’ Retrieved December 14, 2007 from
http://www.cs.columbia.edu/~hgs/internet/definition.html

Marilyn Watson, 2005. Principles & Issues of Human-Centred Evaluation, University of

Sunderland. Computing & Technology. Session 9, p.10

Methods: User Testing, 2005. Retrieved December 10, 2007 from

http://www.usabilityfirst.com/methods/testing.txl

Subjective vs. objective, 2000 Retrieved December 10, 2007 from

http://herkules.oulu.fi/isbn9514259378/html/x743.html

Appendix A: User profile

11
 Educational Knowledge of
User Profile Gender Age
Level Walmart.com
An employee at a College and
part time diploma student. Has
very good Information
1 Diploma Female 30 Yes
Technology (IT) knowledge and
constant user of the computer and
internet.
An Employee at the shopping
mall. Has basic windows
2 knowledge and Microsoft office Diploma Male 26 Yes
suite but casual user of the
computers and internet.
A Pharmacist with good
3 computer literacy and frequent Graduate Female 24 Yes
user of the computer and internet.
A lecturer at an international
university. Very good
4 understanding of IT and a very Doctorate Male 36 Yes
frequent user of the Internet and
computer
A Cashier with an International
Non-Government Organisation
(NGO). Good understanding of
5 Graduate Female 33 Yes
computer application programs
and constant user of the computer
and internet.
A Software developer in a
software development
Higher
6 organisation. Excellent computer Male 29 Yes
Diploma
skill and constant user of the
computer and internet
IT consultant and free lancer.
Good understanding of
7 Information systems. Excellent Masters Male 36 Yes
computer skills. Constantly on
the internet via laptop and phone.
Project Officer in an International
NGO. Good knowledge of
8 computer application software Masters Female 27 Yes
and frequent user of the computer
and internet
A secretary with a law firm.
Knowledgeable in Microsoft
9 Office suite. Frequent user of the Diploma Female 25 No
computer but occasional internet
user.
Self employed. Low computer No
1
literacy and casual user of the Diploma Male 35
0
internet and computer. 12
A Landlord and part time masters
1 student. Good computer skills
Appendix B: Tasks

Task 1
Register as a new user

Purpose
a) Test the user’s ability to navigate through the Walmart.com home page
b) Test the user’s ability and eases of create a new account

Task 2
Select an item of choice form the website, a brown men’s leather jacket.

Purpose
a) Evaluate how easy it is for the user to navigate through the website and shop for
the product they need.
b) Assess the user’s understanding of the language used.
c) Evaluate the ease of reading the web content
d) Assessment of how easy the user interface can be learned quickly and effectively
e) Assess the ease of selecting items from the website

Task 3
Try to purchase the selected item from task 2

Purpose
a) Evaluate efficiency and effectiveness of Walmart.com by checking the time taken
when purchasing an item.
b) Evaluate error messages if any and their usefulness to the user.
c) Assess the ease of purchasing an item
d) .

Task 4
Edit their profile information such as username and password

13
Purpose

a) Evaluate if the user is able to make meaning of the web content and understand
links or menus.

b) Assess the ease of changing customer account information

Task 5
Get online help

Purpose
Evaluate the effectiveness of the website content in directing a customer to obtain help
Appendix C: Questionnaire

Usability Evaluation of www.walmart.com

1) How easy did you find it navigating through the website?

a) Very easy
b) Easy
c) Difficult
d) Very difficult
2) Which of the tasks below did you find difficult?
a) Registering as a user
b) Selecting an item
c) Purchasing an item
d) Editing your account
e) Getting online help
3) How would you rate the ability to read the web content?
a) Very easy
b) Easy
c) Difficult
d) Very difficult
4) Were you able to use informational images for alternative text?
a) Yes
b) No

14
5) What aspects of Multimedia did you use such as video, audio?
a) Video
b) Audio
c) Images
d) Text
e) Animation

6) Does the website have offensive content?

a) Yes
b) No
7) Would you visit the website for another purchase?
a) Yes
b) No
8) Would you encourage your friend to shop online?
a) Yes
b) No
9) Rate your understanding of the language used
a) Very easy
b) Easy
c) Difficult
d) Very difficult
10) How would you rate the speed of task execution such as loading of web page?
a) Very Slow
b) Slow
c) Moderate
d) Fast
e) Very Fast

15
Appendix D: Observation Check list

• Users ability to create a new user account at Walmart.com

• Easy of selecting the specific item form Walmart.com

• User’s ability to purchase the selected item

• Users ability to edit their profile information such as username and password

• Website ability to provide help

• Ease and speed of opening the website and loading pages.

• Ease of navigating through the website.

• User’s ability to complete a task.

• Tasks performed by the user to completion.

16
• Ability to place the goods selected in a shopping cart.

• Ease of checking out.

• Completion of purchase.

• Errors notification obtained.

17