Professional Documents
Culture Documents
WINTER 2008
Executive Summary
Customer demand for the superior web experiences that Web 2.0 applications can provide
-- including increased user interactivity, relevant real-time content and rich media, content cre-
ated by social networks -- is driving leading web companies to design complex new end-user
facing web applications. But building these applications often requires moving functionality
from the well-understood server environment to inside the user’s browser itself. For the major-
ity of companies accustomed to developing applications in the traditional way, the Web 2.0
paradigm represents an unprecedented loss of control. Many continue to develop applications
behind the firewall, test with conventional tools, and launch into production with minimal
regard for the actual customer experience on the desktop.
Along with these new end-user requirements comes the need for an integrated platform of
tools to painlessly support those tasked with designing, developing and deploying those new
applications. The ExperienceFirst platform from Gomez described in this white paper provides
the industry’s first end-to-end solution for web application experience management through-
out the application lifecycle. This technology promises to help business and IT managers to:
Ensure end-users have quality Web experiences;
Set performance and web experience goals early in the design phase and test against those
goals throughout development and quality assurance (QA) phases;
Monitor and manage application performance levels and the impact of third-party content
across multiple browser types and operating systems;
Measure the relationship of performance and customer satisfaction with analysis of yield and
conversion rates of marketing funnels; and
Understand the impact of changes – in everything from partner relations, vendor sources,
capacity, addition of new applications and site modifications – on end user experience.
It’s a composite world today, so much of the web experience a company provides hinges on content, applications and
infrastructure delivered by third parties. As we move toward rich Internet technologies such as Ajax, even less of the
customer experience is under our direct control. Yet the customer still holds the host brand accountable for
the web experience and expects much richer services delivered instantaneously.”
Steve Trimbo
Director of Online Operations Best Buy.com
W hite Pa p e r Managing User Experience and the Organization
Contents
Introduction.......................................................................................................................................................................................3
A Customer Example..................................................................................................................................................................... 10
The information contained in this document represents the current view of Gomez, Inc. on the issues discussed as of the
date of publication. Because Gomez must respond to changing market conditions, it should not be interpreted to be a
commitment on the part of Gomez, and Gomez or its respective suppliers cannot guarantee the accuracy of any informa-
tion presented after the date of publication.
This white paper is for informational purposes only. GOMEZ AND ITS RESPECTIVE SUPPLIERS MAKE NO WARRANTIES,
EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright,
no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form
or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express
written permission of Gomez, Inc.
Gomez may have patents, patent applications, trademarks, copyrights or other intellectual property rights covering subject
matter in this document. Except as expressly provided in any written license agreement from Gomez, the furnishing of this
document does not give you any license to these patents, trademarks, copyrights or other intellectual property.
©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and ExperienceFirst™, ExperienceFirst
SCoE™, ExperienceFirst Network™, Active Network™ XF, Private Network™ XF, Active Last Mile™ XF, Private Locations™
XF, Actual Experience™ XF, Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web
Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc. All other trademarks and
service marks are the property of their respective owners.
This document contains information of a proprietary nature. All information contained herein shall be kept in confidence
and shall be for the original recipient’s use only. Any unauthorized reproduction by any other party shall constitute an
infringement of copyright.
2
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
W hite Pa p e r Managing User Experience and the Organization
Introduction
Website development has completely transformed in a very short amount of time. First-
generation sites circa 1997 – 2003 were architected in a fairly straightforward manner, Site
control resided neatly on the server side. Tens of thousands of sites were built following a basic
three-tiered layered approach where each layer was neatly segmented by function and all the
underlying site components resided in the data center. The back-end mix of databases, applica-
tion servers and web servers were all under the control of developers, and by extension, the
business and brand behind the site.
Year 2000: IT Controls Web Experience Year 2007: Web Application Lifecycle Distributed
thin/
dumb client smart clients
Fast-forward to today’s Web 2.0 world and the difference is astonishing. Top web applications
today have leaped from yesterday’s simple setup to a more complex architecture focused on
providing superior online customer experiences. More user interactivity, relevant content and
rich media is what leading web companies want to deliver in order to achieve a superior user
experience. Outsourcing this new content and functionality saves time and money and makes
good business sense. Implementing these technologies almost always requires moving more
functionality from the well-understood server environment to inside the browser itself. A big
challenge today is that only a few companies know how to do this well.
Unlike ten years ago when companies clearly maintained ownership of the entire web experi-
ence by controlling it from the data center, today there is no single owner of the user experi-
ence. Modern web applications are typically comprised of code and content coming from
numerous parties, only one of which is the company whose logo and brand are displayed
prominently on the page. This mosaic of content and services that comes together in the
browser relies far more on local processing than ever before. With today’s multitude of browser
types, computer operating systems, and versions, there is a real need to ensure that web appli-
cations work well for all end-users.
Outside of a select few developers and business managers in the savviest web companies, few
people realize how much of a site’s content and functionality depends on outside sources. Yet
building sites that use an increasing number of outside services is increasingly the norm.
3
W hite Pa p e r Managing User Experience and the Organization
Unbeknownst to most users, adding layer upon layer of third party content to web applications
is becoming increasingly common. For example, users may think they’re just paying bills on
their bank’s web site. In reality, they are interacting with a third-party bill-paying application
that is enlisting another provider’s hidden web analytics service. Similarly, comparison shopping
services are often outsourced to providers who add features such as local shopping to web-
only national businesses. In the new paradigm, it’s not unusual for sites to have 10 or more
service providers contributing the website content and functionality that end-users rely on.
Meanwhile, businesses increasingly rely on
service-oriented architectures (SOA) to expand
What Rich Internet Applications (RIAs) have you deployed or are planning to deploy in the next year?
(Percentages are based on the total number of responses) the functionality of their web applications. These
applications leverage blocks of external applica-
tion code or web services to add a wide range
of features and functions to a site. In many cases,
these applets are supplied not only by the com-
pany itself but by partners invisible to end-users.
With all this added complexity, few companies
today really understand how their applications
are performing from the end-user perspective.
The fact is, in today’s composite world, it’s dif-
ficult to tell when some combination of external
content and the user’s browser or operating sys-
tem breaks down and spoils the user experience,
particularly if the organization is using conven-
tional testing tools.
Ajax Flash/Flex Streaming Silverlight Adobe Air Other
63.1% 57.1% 28.9% 4.7% 2.7% 14.1%
The user experience trumps all. When done
properly, pushing more Ajax and Flash code to
the browser enhances the user experience and
makes the application more rich and respon-
sive. However, as more content and code goes
Why did you choose to use these Rich Internet Applications (RIA’s)?
(Percentages are based on the total number of responses) directly to the browser, it’s harder for developers
to predict performance. Often, design prob-
lems don’t emerge until after deployment in a
live customer-facing environment. When web
applications fail outright or perform poorly, the
danger for the business is lost revenue to com-
petitors and long-term damage to the brand. For
developers, it means spending time on rework
rather than focusing on future application devel-
opment.
Worryingly, many organizations still aren’t think-
ing about the increasingly important role of
extended web architectures. Following traditional
User Demand Inadequacy of existing Keeping up with the Desire of internal Executive Mandate Other development procedures, they continue to test
29.5% technologies to meet market/competitive developers to use new 7.5% 11.6%
design goals
39.7%
pressure
56.2%
technologies
36.3%
what’s inside their zone of control and to see the
effects of external content and code only after
their application is launched into production.
With increasingly sophisticated user expectations,
those tasked with developing, deploying, and managing applications can no longer afford to
wait to test, tweak, and improve web applications and practices. The stakes for their companies
are too high. Rather, they must start early on and set their targets to define success criteria for
the user experience in the Web 2.0 world. What’s needed is a new, more fully integrated devel-
opment and management platform that starts with the end-user experience in mind, builds on
and combines the time-tested processes that have traditionally been favored.
4
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
W hite Pa p e r Managing User Experience and the Organization
5
W hite Pa p e r Managing User Experience and the Organization
Preview the Customer’s Web Experience. Once development teams are sure their applica-
tion will look and function properly for end-users across different browsers and OS’, it’s time to
transition the application to the Operations team and take testing one step closer to the cus-
tomer. Using Gomez’ active web application testing and monitoring products, companies can
test their applications in a pre-production environment to understand how they will perform for
customers around the world, without actually having to expose the applications to customers.
Private Locations XF
Validate the speed and reliability of your
partners and corporate end users
WEB APP
SERVER SERVER
LOCAL ISP
MAJOR
DATABASE PROVIDER CONSUMER
DESKTOPS
3RD PARTY
CONTENT
Active Last Mile XF
Extend performance insights down
to the desktop
Private Network XF Active Network XF
Behind-the-firewall network monitoring to Real-time monitoring from
quickly determine performance problems the Internet Backbone
MAJOR
PROVIDER
LOCAL ISP
ACTUAL
CUSTOMER
Actual Experience XF
Measure your online customer’s
experience from their browser
6
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
W hite Pa p e r Managing User Experience and the Organization
Ensure Quality Web Experiences in Production. Because revenue is not generated in your
test environments or inside your firewall, Gomez provides the same active monitoring services
that companies can use for pre-launch testing to monitor and troubleshoot the performance
of those applications once they are released and customer facing. Additional monitoring tools
that leverage these agents, including Gomez ExperienceFirst Alerts, ExperienceFirst Dashboards,
and ExperienceFirst Screen Capture on Error (SCoE), and Performance Analyzer also serve to help
companies identify and resolve problems with pre- and post-production web applications more
quickly.
ExperienceFirst Dashboard
Measuring Experience from the Customer’s Browser. Using Actual Experience XF, compa-
nies gain an authentic picture of customer web experiences, individually and collectively, by mea-
suring performance directly from the end-users’ browsers as they visit web pages. As such, Actual
Experience XF is a real-user monitoring service that expands the ExperienceFirst platform to
address the new challenges facing both business line owners and Web 2.0 developers concerned
with the performance impact on customer web experience.
Using a few lines of proprietary code embedded on each page (similar to web analytics technol-
ogy), Actual Experience XF reports back on the end-user experience with the portion of the appli-
cation delivered by the servers and on the portion running locally in end-users’ browser (e.g. Web
2.0 technologies like Ajax and Flash).
What it measures. From its vantage point in the browser, Actual Experience XF measures all
aspects of the end user experience, including the perceived performance of a web application,
download times, abandonment rates, service consistency, geographic disparities, browser win-
dow size, browser type, cache utilization and first time/repeat visitor status. Armed with this infor-
mation, a company can:
Analyze the actual experience of specific customers and segments, and understand which
buyer segments have poor performance and why;
Know which pieces of a company’s website contribute to (or detract from) a favorable cus-
tomer experience, including third party content providers, use of multiple browsers, and rich
media types; and
Make better decisions as to what Web components to monitor and improve by analyzing
performance and how they impact business performance.
Actual Experience XF dives deeply into many factors influencing customer satisfaction and far
beyond what’s found in standard web analytics. For example, the tool can uniquely show the
impact of application performance on conversion by tracking both page level abandonment rates
(e.g., when a user clicks on the stop button of a browser to stop the page from downloading) and
business process level abandonment (e.g., when a user abandons a multi-step process before
reaching the last step, such as putting an item in the shopping cart but never checking out).
7
W hite Pa p e r Managing User Experience and the Organization
TM
Develop
& Test Monitor
Design
8
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
W hite Pa p e r Managing User Experience and the Organization
9
W hite Pa p e r Managing User Experience and the Organization
A Retailer doing significant business in the United States is releasing a new Web 2.0 shopping
cart application, but wants to make sure that the release will not negatively impact the cus-
tomer web experience – and therefore the businesses revenue. Based on information captured
with Actual Experience XF, business managers have benchmarked the current end-user satisfac-
tion levels and also determined that the large majority of top customers in the US are accessing
the company’s Website using a variety of browser/OS combinations, but are primarily on one
ISP using low-broadband and dial-up connections around the Western US.
Armed with this information, the company’s developers went to work with the goals of meet-
ing or exceeding the current end-user performance and experience levels. Using Reality View
XF and Reality Check XF, the developers are able to test the code as they build it. To make sure
the new application both functions and appears as intended from all the different browser and
operating system combinations, including those of the high value customers.
Once the application had been developed, the quality assurance team decides that it’s ready to
put the new application to the test. Using information gathered with Actual Experience XF, the
company uses Gomez RealityLoad XF to create a load testing population scenarios that are rep-
resentative of their customer base. They then test the pre-production application with the real
traffic levels that it expects to see in the upcoming busy season.
Assured that scalability is not a concern, the operations team is now able to use Active Network
XF and Active Last Mile XF to test the beta application and measure how it responds from the
perspective of the low-broadband and dial-up customers. Fortunately, this testing exposes a
period issue with their third party content acceleration vendor that was causing a 50% increase
in response time. After sharing the data with the vendor, the company then validates that the
discussed fix has been implemented and has resolved the problem.
Satisfied with the results of the load test and the operations team’s initial testing, they share the
results and expected user experience levels internally to review that the initial goals have been
met. After getting approval from all the involved units, the operations team sets up continuous
monitoring to catch any new customer or application problems that might arise and launches
the new application into production with confidence that the new application will exceed the
quality of web experiences delivered by the old application.
The business managers, eager to measure the impact of the new release, quickly correlates the
new Actual Experience XF measurements with the old measurements to determine that they
have indeed successfully launched the new application, and improved performance for that key
customer segment more than anticipated, and achieved the web application experience goals
they set at the beginning of the process.
Gomez, Incorporated ©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and
ExperienceFirst™, ExperienceFirst SCoE™, ExperienceFirst Network™, Active Network™ XF,
10 Maguire Rd, Bldg 3, Suite 330 Lexington, MA 02421 Private Network™ XF, Active Last Mile™ XF, Private Locations™ XF, Actual Experience™ XF,
General: +1 781-778-2700 Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web
Sales: USA +1 781-778-2760; UK +44 (0)207-554-5154; Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc.
All other trademarks are the property of their respective owners.
Germany +49 (0)40-53299-167.
www.gomez.com Specifications may change. Consult the technical documentation for the most current information.
Some features described are extra cost options. Ask your sales representative for details.
wp2008003