Professional Documents
Culture Documents
Article information:
To cite this document:
Rian van der MerweJames Bekker, (2003),"A framework and methodology for evaluating e-commerce Web sites", Internet
Research, Vol. 13 Iss 5 pp. 330 - 341
Permanent link to this document:
http://dx.doi.org/10.1108/10662240310501612
Access to this document was granted through an Emerald subscription provided by emerald-srm:499410 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service
information about how to choose which publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.
Introduction
A framework and
methodology for
evaluating e-commerce
Web sites
Rian van der Merwe and
James Bekker
The authors
Rian van der Merwe is Research Assistant and Student,
School of Marketing, Curtin University of Technology, Perth,
Australia.
James Bekker is a Lecturer in Information Systems and
Computer Simulation, Department of Industrial Engineering,
University of Stellenbosch, South Africa.
Keywords
Electronic commerce, Internet, Function evaluation
Abstract
The corporate Web site is an indispensable part of any ecommerce venture. If users find it difficult to do electronic
transactions, they will visit another online source, or go to a
physical source instead. Evaluating the performance of ecommerce Web sites has received some attention in the past
few years, but there is still a pressing need for a more
comprehensive approach. This paper attempts to address
this need by providing an e-commerce Web site evaluation
framework and method built on solid business principles,
and using multidimensional scaling to analyse evaluation
data and present the results in a graphic manner for easy
interpretation.
Electronic access
The Emerald Research Register for this journal is available at
http://www.emeraldinsight.com/researchregister
The current issue and full text archive of this journal is
available at
http://www.emeraldinsight.com/1066-2243.htm
Internet Research: Electronic Networking Applications and Policy
Volume 13 . Number 5 . 2003 . pp. 330-341
# MCB UP Limited . ISSN 1066-2243
DOI 10.1108/10662240310501612
330
331
332
Level 2
With the first level of the evaluation framework
completed, the second level to be developed is
the criteria groups within each category. These
groups were derived from the literature on Web
site evaluation using popular criteria listed by
several authors (e.g. Berthon et al. 1996; Abels
et al. 1999; Cunliffe, 2000; Bauer and Scharl,
2000; Kim et al., 2003). These criteria were
then classified according to the researchers'
judgement under the most appropriate category
defined on level 1. Four groups per criteria
category were identified, as shown in Figure 2.
Each of these groups is explained in Table I.
Level 3
The third level of the evaluation framework
consists of specific criteria within each criteria
group. Although it cannot be completely
eliminated, the purpose of the criteria on this
level is to minimize subjectivity during the
evaluation process. These criteria are proposed
by the researchers and are debatable. Five
criteria were developed within each of the 20
criteria groups described above, resulting in a
total of 100 criteria. The criteria are shown in
Table II-VI. Note that language is not part of
the criteria, as it is assumed that e-commerce is
333
Criteria group
Interface
The effective use of colour, text, backgrounds, and other general graphic
design principles
The effectiveness of the graphics and multimedia used on the site
Whether or not the text is concise and relevant, and the style good
The degree to which the interface is designed to handle exceptions, for
example, text-only versions of pages
Logical structure
Ease of use
Search engine
Navigational necessities
Content
Product/service-related
information
Company and contact
information
Information quality
Interactivity
Reliability
The registering process and how the company uses the stored customer
profile
The effectiveness and ease of use of the online order process
The company's actions from order placement until the order is delivered
How the company communicates and helps its online customers
Order process
After-order to order receipt
Customer service
Technical
Speed
Security
Software and database
System design
Evaluation concept
Evaluation data for a given Web site are
gathered by going through the list of evaluation
criteria and awarding a value to each item. This
value will be an indication of how well the Web
site adheres to the specific criterion. All the
individual values for the criteria in a specific
group are then summed to a unique score for
that group. The group scores within each
category are then summed, which generates a
scale of criteria category scores indicating how
334
1 Interface
3 Content
Group 1:
Graphic design
principles
Group 1:
Product/servicerelated content
Group 2:
Graphics and
multimedia
Group 3:
Style and text
Group 4:
Flexibility and
compatibility
Group 3:
Content current and updated
Information quality Content relevant to purpose of site
Content concise and non-repetitive
High perceived quality of product/service
Quality of advertisements high
Group 4:
Interactivity
Group 1:
Logical structure
Group 2:
Ease of use
Easy
Easy
Easy
Easy
Easy
Group 3:
Search engine and
help function
Group 4:
Navigational
necessities
No broken links
No ``under-construction'' pages
Links clearly discernible, well labelled and defined
Clear label of current position on site
Effective use of frames, non-frames version available
Group 4:
Customer service
to
to
to
to
to
find site
explore specific idea or subject
return to main page
find specific information
access complete product/service range
Group 1:
Stored customer
profile
Group 2:
Order process
335
4 Technical
Group 1:
Speed
Group 2:
Security
Group 3:
Software and
database
Group 4:
System design
Correspondence analysis
In order to quantify, interpret and assess the
evaluation data, it is imperative to find a
suitable technique to analyse data that are
presented in this form. The purpose of
multidimensional scaling techniques is to
condense large amounts of data to as few
dimensions as possible, while still describing it
adequately. Formally, multidimensional scaling
can be defined as ``[a set of mathematical
techniques] concerned primarily with the
Table VII Example of a two-way contingency matrix for Web site evaluation
Criteria category Criteria category Criteria category Criteria category Criteria category
1
2
3
4
5
Web
Web
Web
Web
site
site
site
site
A
B
C
D
67
87
78
37
78
74
64
57
63
48
49
85
336
78
93
56
73
23
67
83
65
337
Site address
Amazon.com Books
Barnes & Noble
Exclusive Books
Kalahari.net
PC Books, Inc.
VS Online
www.amazon.com/books
www.bn.com
www.exclusivebooks.com
www.kalahari.net
www.pcbooks.co.za
www.vsonline.co.za
Interface Navigation
17
15
14
15
19
6
35
29
35
11
4
14
Content
30
25
18
13
1
9
Reliability Technical
31
27
27
24
16
6
28
19
23
12
22
11
Total
141
115
117
45
16
28
338
Conclusions, recommendations,
managerial and research implications
At present, firms engaged in e-commerce are
limited in their ability to gauge their online
performance based on objective criteria. Their
ability to compare their performance with other
companies in the same industry is also
restricted. The technique presented in this
paper is structured and repeatable, and gives
managers the opportunity to evaluate the as-is
situation of their Web sites (assuming that they
wish to do so) with respect to those of other
companies. They can use the approach to
identify where they perform well, and where
there is room for improvement. Managers can
determine which sites within their industry
perform better than they do in particular areas,
and why. This objective information can be
used to make strategic decisions on how to
improve the efficiency and performance of their
Web sites.
Practising and academic marketing
researchers can also apply the technique.
Practitioners can identify who the leaders are in
a specific industry, as well as why they do so
well. Academics can conduct cross-industry
research, which may reveal aspects of Web sites
within one industry that may have been
neglected in others. They will be able to
quantify the performance of Web sites and use
this as a dependent or independent variable in
research directed at determining the
consequences of Web site performance.
There are, however, a few reservations about
the individual criteria:
.
Although the objectivity of criteria was a
major driving force behind the
development process, the criteria still entail
a certain measure of human subjectivity, as
mentioned above. Although the evaluation
framework and criteria groups seem to be
reasonably inclusive, it may be necessary to
modify or even replace some of the criteria
to ensure greater objectivity in the process.
The criteria of level 3 requires further
evaluation, and should perhaps be
339
340
References
Abels, E.G., White, M.D. and Hahn, K. (1999), ``A user-based
design process for Web sites'', OCLC Systems &
Services, Vol. 15 No. 1, pp. 35-44.
Alexander, J. and Tate, M. (1996), ``Evaluation criteria'',
available at: http://hoku.lcc.hawaii.edu/kaz/100/
evalCriteria.htm
Bamber, M., Hill, M.C. and Watson, R.T. (1995), ``Group
support systems can improve your meetings'', The CPA
Journal, Vol. 65 No. 3, pp. 32-7.
Bauer, C. and Scharl, A. (2000), ``Quantitative evaluation of
Web site content and structure'', Internet Research:
Electronic Networking Applications and Policy, Vol. 10
No. 1, pp. 31-43.
Berthon, P.R., Pitt, L.F. and Watson, R.T. (1996), ``The World
Wide Web as an advertising medium: toward
understanding of conversion efficiency'', Journal of
Advertising Research, Vol. 36 No. 1, pp. 43-53.
Churchill, G.A. Jr, Ford, N.M. and Walker, O.C. Jr (1993),
Sales Force Management, 4th ed., Richard D. Irwin,
Homewood, IL.
Cunliffe, D. (2000), ``Developing usable Web sites a review
and model'', Internet Research: Electronic Networking
Applications and Policy, Vol. 10 No. 4, pp. 295-307.
D'Angelo, J. and Little, S.K. (1998), ``Successful Web pages:
what are they and do they exist?'', Information
Technology and Libraries, Vol. 17 No. 2, pp. 71-81.
Day, A. (1997), ``A model for monitoring Web site
effectiveness'', Internet Research: Electronic
Networking Applications and Policy, Vol. 7 No. 2,
pp. 109-15.
Green, P.E., Carmone, F.J. and Smith, S.M. (1989),
Multidimensional Scaling: Concepts and Applications,
Allyn and Bacon, Needham Heights, MA.
Greenacre, M.J. (1984), Theory and Applications of
Correspondence Analysis, Academic Press, London.
341
25. GLIN BYKZKAN, DA RUAN. 2007. EVALUATING GOVERNMENT WEBSITES BASED ON A FUZZY
MULTIPLE CRITERIA DECISION-MAKING APPROACH. International Journal of Uncertainty, Fuzziness and KnowledgeBased Systems 15, 321-343. [CrossRef]
26. Young Namkung, Seo-Young Shin, Il-Sun Yang. 2007. A Grounded Theory Approach to Understanding the Website Experiences
of Restaurant Customers. Journal of Foodservice Business Research 10, 77-99. [CrossRef]
27. Glin Bykzkan, Da Ruan, Orhan Feyziolu. 2007. Evaluating e-learning web site quality in a fuzzy environment. International
Journal of Intelligent Systems 22:10.1002/int.v22:5, 567-586. [CrossRef]
28. Ray Welling, Lesley White. 2006. Web site performance measurement: promise and reality. Managing Service Quality: An
International Journal 16:6, 654-670. [Abstract] [Full Text] [PDF]
29. R. Ufuk Bilsel, Glin Bykzkan, Da Ruan. 2006. A fuzzy preference-ranking model for a quality evaluation of hospital web
sites. International Journal of Intelligent Systems 21:10.1002/int.v21:11, 1181-1197. [CrossRef]
30. Ray Welling, Lesley White. 2006. Measuring the Value of Corporate Web Sites. Journal of Internet Commerce 5, 127-145.
[CrossRef]
31. Talha Harcar, John E. Spillan. 2006. Exploring Latin American family decision-making using correspondence analysis. Journal
of World Business 41, 221-232. [CrossRef]
32. COSTAS ZAFIROPOULOS, VASILIKI VRANA, DIMITRIOS PASCHALOUDIS. 2006. Patterns of Information Services
on Hotel Websites: The Case of Greece. Anatolia 17, 55-74. [CrossRef]
33. Nikos Manouselis, Constantina Costopoulou. 2006. Quality in metadata: a schema for ecommerce. Online Information Review
30:3, 217-237. [Abstract] [Full Text] [PDF]
34. Charles Oppenheim, Louise Ward. 2006. Evaluation of web sites for B2C ecommerce. Aslib Proceedings 58:3, 237-260. [Abstract]
[Full Text] [PDF]
35. Wayne Huang, Taowen Le, X. Li, S. Gandha. 2006. Categorizing web features and functions to evaluate commercial web sites.
Industrial Management & Data Systems 106:4, 523-539. [Abstract] [Full Text] [PDF]
36. Costas Zafiropoulos, Vasiliki Vrana, Dimitrios Paschaloudis. 2006. The internet practices of hotel companies: an analysis from
Greece. International Journal of Contemporary Hospitality Management 18:2, 156-163. [Abstract] [Full Text] [PDF]
37. Irene Pollach. 2005. Corporate selfpresentation on the WWW. Corporate Communications: An International Journal 10:4,
285-301. [Abstract] [Full Text] [PDF]
38. Jrgen KaiUwe Brock, Yu (Josephine) Zhou. 2005. Organizational use of the internet. Internet Research 15:1, 67-87. [Abstract]
[Full Text] [PDF]
39. Muneesh Kumar, Mamta SareenEvaluating Web Sites of Municipal Corporations 143-157. [CrossRef]