Building A Survey Question Conjoint Cbcwebtut

CBC/Web Tutorial and Example
Introduction
This tutorial leads you step-by-step through a simple CBC study, from thinking about a
marketing problem, planning a questionnaire and creating a new study in SSI Web, to entering a
list of the attributes and levels, generating the questionnaire and analyzing the results.
SSI Web is a powerful program, and this tutorial only touches on the basics. Youll become
aware of different or more sophisticated approaches by reading other sections in this manual,
by viewing on-line help and by experimenting with SSI Web.
We also suggest the following two articles, available within our Technical Papers Library at
www.sawtoothsoftware.com/techpap.shtml : CBC Technical Paper and Getting the Most from
CBC.
(Note: this example and the data presented are purely fictional. The suggested steps represent
one approach to this type of marketing problem, but may not necessarily reflect the single best
way to conduct such a research study using the SSI Web system.)
The Marketing Problem
You have been hired as a consultant to a little-known startup company called Performance Plus.
The brilliant design engineers at Performance Plus have developed a golf ball that flies farther
than average balls. The proposed name for the ball is Long Shot. Because Performance Plus
is not well known among golfers, your client has considered approaching a well-known maker of
golf clubs and balls (Golfers, Inc.) to market the ball using their existing brand name for balls:
Eclipse. Sales for Eclipse have been declining, and they welcome the opportunity to benefit
from the new technology. A line extension brand called Eclipse+ has been proposed.
However, the royalty Golfers, Inc. is proposing seems high.
Two other manufacturers already market high performance balls: High-Flyer Pro (by Smith and
Forester), and Magnum Force (by Durango). High performance balls command a price
premium of between 20% to 50% over traditional balls, and the market is growing rapidly.
One day between rounds of golf with the president of Performance Plus (you gracefully lost),
you jointly draft up the following questions on the back of your score card.
1) How well could Long Shot by Performance Plus hope to compete with existing
competitors Smith and Forester and Durango in the high performance ball market?
2) Should Performance Plus form an alliance with Golfers, Inc.? How much incremental
value does the Golfers, Inc. brand name Eclipse add?
3) How do golfers trade off performance vs. price for high performance balls?
4) How should Long Shot be manufactured and positioned in terms of performance and
price to maximize profitability?
1
5) Do novice or experienced golfers differ in their preferences for high performance golf
balls? Could this lead to a target market strategy?
Defining a List of Attributes and Levels
One of the first steps in using any Conjoint technique is to define the problem in terms of
attributes and levels. Back at the office, you review the points written on the back of the score
card, and you write down the following:
Brand
High-Flyer Pro, by Smith and Forester
Magnum Force, by Durango
Eclipse+, by Golfers, Inc.
Long Shot, by Performance Plus
Performance
Drives 5 yards farther than the average ball
Price
$4.99 for package of 3 balls
Each of these levels is mutually exclusive; golf balls are described using a single level from
each attribute. The range of the attributes is based on technical specifications that Performance
Plus has given you. As for prices, your research suggests that the two competitors are currently
selling packages of three balls for between $7.99 to $10.99. You select a wider range to cover
the price levels you are interested in, but dont consider any prices below $4.99, having
determined that such a low price cannot cover the manufacturing and marketing costs and
result in a profit.
A Study Plan and Sample Scripted Questionnaire
Now that you have formulated some attributes and levels, you begin to think about the rest of
the questionnaire, and plan a design and field strategy.
You decide to invite golfing enthusiasts (by email) to visit a site on the web and take a CBC
survey. Youve purchased an opt-in list of 2000 email addresses from a widely-read golfing
magazine. Each of the participants who complete the survey will be entered into a drawing for a
free set of golf clubs from a leading manufacturer. You expect a completion rate of between
15% to 25%, resulting in a final sample size of between 300 to 500.
You have scripted the following introductory text screens and questions in your word processing
package:
2
Introduction:
Thank you for your willingness to participate in this study.
You are part of a special group of golfers weve selected to ask for feedback regarding
golf equipment. By completing this survey, youll be eligible for a drawing to receive a
free set of PrimoDriver clubs.
(Click the Next button to continue)
Frequency:
First, wed like to ask you about how often you golf. Would you say that you golf...
5 times or fewer per year
from 6 to 25 times per year
more than 50 times per year
Skill:
We'd like to get a feel for your skill level. Over the last year, what was your best score
on a par-72, 18-hole course?
120 strokes or more
Between 100 and 120 strokes
Between 85 to 100 strokes
Above par, but lower than 85
At or below par (72 or lower)
3
Venue:
Do you usually golf at...
Public courses
Private courses
IntroCBC:
In the next section, well be asking you specifically about high performance golf balls.
These balls are premium balls specially engineered to travel farther than the typical ball
with good accuracy.
Wed like you to imagine that you are considering purchasing golf balls for your next golf
outing. Well show you some different high performance golf balls, and ask which one
you would purchase.
Some of the golf balls you are going to see are not currently available on the market, but
wed like you to imagine that they were available today. It is important that you
answer in the way you would if you were actually buying golf balls.
If you wouldnt purchase any of the balls well show you, you can indicate that by
choosing None. By choosing none, you indicate that youd buy another brand, or
would continue using existing balls in your golf bag.
At this point in the questionnaire, you plan to ask the Choice-Based Conjoint questions (tasks).
You want to measure preference for the different levels of the attributes and their impact on
choice for high performance golf balls. To accomplish this, you decide to specify 15 randomized
CBC tasks (below is an example of one such task). The features will combine freely to form up
to 48 (4 x 3 x 4) different product offerings. Even though there are only 48 total possible product
concepts in this relatively small design, there are thousands of different ways to display multiple
product concepts at a time in choice sets.
Choice Task (one of 15 randomly constructed tasks):
If you were considering buying golf balls for your next outing and these were the only
alternatives, which would you choose?
High-Flyer Pro, Magnum Force, Eclipse+, by

by Smith and by Durango Golfers, Inc.
Forester None: I
Wouldnt
Drives 10 yards Drives 15 yards Drives 5 yards Purchase Any of
farther than the farther than the farther than the These
average ball average ball average ball
$10.99 for package $6.99 for package of $8.99 for package

of 3 balls 3 balls of 3 balls
4
The term random tasks is easily misunderstood. Some people take it to mean that the
composition of the questions is randomly (haphazardly) selected. However, the questions are
carefully chosen according to design principles of balance and independence (orthogonality).
We call the tasks random because respondents are randomly selected to receive one of many
carefully constructed, unique versions of the CBC questionnaire.
In addition to the random choice tasks, you also plan to ask a few fixed holdout tasks that will
be constant across all respondents. You plan to do this for three main reasons:
1) By doing some research on the Internet and at local golf stores, you have learned the
performance specifications and average prices for the two existing balls on the market
(High-Flyer Pro and Magnum Force). Your client has also given you a good feel for the
product specifications they think could match up well with these two brands. Therefore,
you and your client can imagine a specific potential product scenario youd like to directly
assess.
2) Your client has no experience with conjoint methods and seems a bit uncertain
regarding the reliability and accuracy of the market simulator. Demonstrating that the
market simulator can accurately predict responses to some holdout fixed scenarios
should boost his confidence in the method.
3) In addition to logit, you are considering using an advanced technique such as Latent
Class analysis and Hierarchical Bayes (HB) to analyze the results. Youd like to have a
couple of holdout observations by which to compare the predictive accuracy of
alternative simulation models you might develop.
These fixed tasks will not be constructed randomly, but are to be displayed exactly with the
specific levels you use to specify them. These two tasks will be asked as the 7th and 9th tasks.
In total, youll ask 15 random + 2 fixed = 17 tasks. In the first fixed holdout, you plan to show
the relevant competition versus your clients offering with your clients brand name. In the
second fixed task, you plan to show that same relevant competition versus the Eclipse+, by
Golfers, Inc.
Here are those two fixed holdout concepts:
Fixed Holdout Task #1 (7th overall choice task):
High-Flyer Pro, Long Shot, by Magnum Force,

by Smith and Performance Plus by Durango
Forester None: I
Wouldnt
Drives 15 yards Drives 10 yards Drives 5 yards Purchase Any
farther than the farther than the farther than the of These
$10.99 for package $6.99 for package of $8.99 for package of

of 3 balls 3 balls 3 balls
5
Fixed Holdout Task #2 (9th overall choice task):
Magnum Force, Eclipse+, by High-Flyer Pro,

by Durango Golfers, Inc. by Smith and
Forester None: I
Wouldnt
Drives 5 yards Drives 10 yards Drives 15 yards Purchase Any
farther than the farther than the farther than the of These
$8.99 for package of $6.99 for package of $10.99 for package

3 balls 3 balls of 3 balls
Notice that your clients offering is in the center position for both tasks, but the position of its
competitors is rotated. The two fixed holdout tasks are separated by a random choice task so
that respondents will generally not recognize that it is a repeated task with just one small
difference (the brand name attached to your clients ball).
Lastly, you decide to ask about gender and income. These might prove useful as a respondent
filter in analysis, for bench-marking versus future waves of research, or for future
target/database marketing efforts.
Gender:
Are you..
Male
Female
Refused
Income:
This is the last question in the survey. What is your total household income for the last
calendar year, before taxes?
Less than $30,000
Between $30,000 and $60,000
Between $60,000 and $100,000
More than $100,000
Refused
6
Signoff:
That completes our survey. Thank you very much for your input.
Now that you have developed a list of attributes and levels, and scripted a rough-draft of your
questionnaire and the design of the choice tasks, you are ready to create the study and
compose the questionnaire within the SSI Web system.
Using SSI Web to Write the Questionnaire
Start by opening the Sawtooth Software SSI Web program. If you have a standard installation,
you start it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web.
SSI stands for Sawtooth Software Inc. and Web conveys the idea that this software is for
developing Web-based surveys (although SSI Web can also be used for standalone computer
interviewing with its CAPI interviewing module.) CBC/Web is one component within that
software program. The next step is to open a new study, and define a study name.
After starting up SSI Web, choose File | New Study. The New SSI Web Study dialog
appears, with your cursor active in the studyname field.
The Location field lists the folder (directory) that SSI Web currently is using to store studies.
You can use any folder you like for this tutorial project. In general we suggest placing each new
SSI Web study you create in a separate study folder. You can browse to and create new
folders by clicking the button to the right of the Location field, and then (after browsing to the
folder in which you want to create a new sub-folder to contain this study) by clicking the new
folder icon. Each studyname in SSI Web has a maximum number of four characters (either
letters or numbers). For this tutorial, you might create a name such as cbc1. From the New
SSI Web Study dialog, specify cbc1 as the studyname, and make sure that the Survey Type
field indicates CBC and CiW.
Click the OK button. You are returned to the main menu, and a new Study Navigator window
is displayed along the left-hand side of the screen.
7
As you work with this study, items appear in the Study Navigator window, listing the functional
areas you have been using or that are now available to you. This list of items provides a
Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas
by clicking icon buttons on the toolbar or by using the pull-down menus.
Entering the List of Attributes and Levels
When composing conjoint analysis studies, one typically begins by specifying the list of
attributes and levels in the software.
To enter the list of attributes and levels you developed, click the A button (A for Attributes)
on the toolbar, or choose Compose | Conjoint Settings | Add/Edit Attributes. The Specify
Attributes and Levels dialog opens.
To add the first attribute, click the Add button (at the bottom left of the Attributes panel). Your
cursor is now active in the Attribute 1 Name field.
8
Type in the first attribute name: Brand. The attribute name is a label that youll use for
organizing information within SSI Web. Click OK to accept this information and close the dialog.
Now that at least one attribute name is in the list, the Add button under the Levels panel
becomes active. Also note that the Brand attribute is highlighted in the Attributes panel. With
the Brand attribute highlighted in the Attributes panel, click the Add button under the Levels
panel to add levels within the Brand attribute. The Level Text dialog is displayed. Type High-
Flyer Pro, by Smith and Forester. To add the next level of Brand, press the ENTER key
twice. Type in the next level: Magnum Force, by Durango. Repeat the process for the
remaining two levels of brand.
When you are ready to add the next attribute (Performance), click the Add button under the left
Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute
list. With that new attribute highlighted on the attributes list, click the Add button under the
Levels panel to add the three levels of that attribute.
Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list
of attributes below. Note that you can copy-and-paste attribute level text from this document
(and other text documents) into the text fields within SSI Web. After highlighting the words to
copy with your mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired
field.
Brand
High-Flyer Pro, by Smith and Forester
Magnum Force, by Durango
Eclipse+, by Golfers, Inc.
Long Shot, by Performance Plus
Performance
Price
To make changes to existing attributes or levels, select the attribute or level to edit by clicking
the attribute or level in the Attribute or Levels lists within the Specify Attributes and Levels
9
dialog and by clicking the corresponding Edit button. Then, edit the item in the Level or
Attribute text fields.
When you are finished entering the list of attributes and levels, close the Specify Attributes
and Levels dialog by clicking OK.
Additional Study Parameters
After you have specified your list of attributes and levels, you can specify other study
parameters to govern your CBC questionnaire. Click Compose | Conjoint Settings | CBC
Settings (or click the icon) and a message appears, stating that if you continue, CBC
questions will be added to your survey. Click OK to accept and close the warning box. The
Specify CBC Interview Parameters dialog is shown:
Depending on your license, you will see various tabs for this dialog (if you have the Advanced
Design Module, the tabs are available as shown).
On the General tab, you can specify how many Random (experimentally designed) choice tasks
and how many fixed (user specified) tasks you want. For this golf ball study, we want 15
random and 2 fixed tasks. Change the Number of Random Choice Tasks field to 15.
As planned, the questionnaire shows three products on the screen, plus a None, for a total of
four columns in the task (each available choice alternative can be fit into a column). So, the
default settings for Number of Concepts per Choice Task and Number of Columns are already
correct and need no modification.
Recall that we want to include a None option in this CBC questionnaire. This gives
respondents the option to indicate that they would choose none of the products displayed on the
screen. The Include None Option box is checked by default, but the text is not quite correct.
Our planned questionnaire shows the following None text:
None: I Wouldnt Purchase Any of These
10
You can directly edit the Text for None Option field to change this, or you can click the Pencil
icon on the toolbar to access a larger edit dialog.
Click the Format tab, to review the following settings:
The Alternating Concept Colors are used in alternating concepts in the choice task to help make
the choice task more readable, and provide a visual indicator from one choice task to another
that the question has changed (sometimes choice tasks can look so similar that respondents
might not notice from, say, the first task to the next that anything is different.)
The Format Options on the right-hand side of this dialog control how large the choice tasks will
appear on the respondents monitor, how much white space is included between concepts and
attribute levels, and whether an additional separation is added between concepts.
To preview how the CBC task will look when it runs on the respondents computer using
Windows Explorer, click the Preview button. The question is displayed in Preview mode (no
data are saved, and no special error checking or validation is performed for the question).
11
Youll note that SSI Web has used default heading text for the CBC question (and it isnt quite
what we need for this questionnaire). In the next section, well show you the dialog for
modifying that text. Click OK to close the preview window. At this point, you might try modifying
some of the settings on the General and Format windows. After each change you make, you
can click Preview to see the effect of the changes.
Generating the Experimental Design
In CBC studies, we use a carefully chosen experimental design plan. The experiment involves
observing how respondents react to different golf ball designs. The design reflects the
attribute combinations that make up the golf balls, and how these combinations are placed into
choice sets. Ideally, each respondent would receive a unique questionnaire version, with
different combinations of golf balls arranged within sets in unique ways. There are only a finite
number of possible combinations, so there potentially can be some identical choice tasks across
respondents, but the idea is to improve measurement of the effects of the attribute levels
(including reducing order and learning effects) by ensuring a high degree of variability in the
choice tasks across individuals.
CBC/Web lets you generate up to 999 unique versions of the questionnaire to be uploaded to
the web server. By pre-specifying the design plans in a file residing on the server, we can
simply assign each new respondent to the next subsequent design version within the file, which
places minimal demands on the web server. Even if you have more than 999 respondents,
12
once a reasonably large number of designs are allocated across respondents, the statistical
gains of using even more design versions are very minimal. The default in the software is to
use 300 design versions. The additional gains in precision for going beyond that are probably
negligible.
If you arent already at the Specify CBC Interview Parameters dialog, go there by clicking
Compose | Conjoint Settings | CBC Settings. Click the Design tab, and the following is
displayed:
Many of the features of this dialog are beyond the scope of this tutorial. You can read additional
details by pressing F1. Well cover some of the basics here.
Under Experimental Design Creation Settings, we can specify the Random Task Generation
Method. This indicates the strategy that CBC/Web uses to generate the random choice tasks
in our study. Complete Enumeration is the default strategy, and it should work well for our
study.
Number of Questionnaire Versions refers to how many unique versions of the CBC questions
that SSI Web will save into a file. When respondents enter the questionnaire, they are assigned
the next subsequent questionnaire version. Once a respondent is assigned the 300th
questionnaire, the next respondent begins again with version #1, and so forth.
Randomize Attribute Position within Concepts lets you randomize the order in which the
attributes appear in the choice task. We specified the attribute list in the order Brand,
13
Performance and Price. If we use the default (which is not to randomize the order), Brand will
always appear in the top and Price always at the bottom. This seems like a natural presentation
order for this study, so well retain the Do not Randomize Attribute Order setting.
Concept Sorting within Tasks controls how the concepts are arranged within the choice task.
By default, the order of presentation is randomized. But, if we wanted the first brand always to
appear in the first concept position, etc., we could specify to sort concepts based on the Natural
Order for Brand. You can investigate details regarding Concept Sorting by clicking F1. For this
tutorial, well retain the defaults.
Once we are comfortable with our settings, we click Generate / Test Design. The following
report is displayed:
Design Efficiency for Main Effects

Copyright 1993-2004 Sawtooth Software
A Priori Estimates of Standard Errors for Attribute Levels

Task Generation Method: Complete Enumeration
Design seed: 1
Number of versions: 300
Total Choice Tasks: 4500
Att/Lev Freq. Actual Ideal Effic.

1 1 3375 (this level has been deleted) High-Flyer Pro, by Smith and Forester
1 2 3375 0.0258 0.0258 1.0000 Magnum Force, by Durango
1 3 3375 0.0258 0.0258 1.0000 Eclipse+, by Golfers, Inc.
1 4 3375 0.0258 0.0258 1.0000 Long Shot, by Performance Plus
2 1 4500 (this level has been deleted) Drives 5 yards farther than
2 2 4500 0.0211 0.0211 1.0000 Drives 10 yards farther than
2 3 4500 0.0211 0.0211 1.0000 Drives 15 yards farther than
3 1 3375 (this level has been deleted) $4.99 for package of 3 balls
3 2 3375 0.0258 0.0258 1.0000 $6.99 for package of 3 balls
The report indicates that 300 versions of the CBC questionnaire were created. We know that
each version of the questionnaire had 15 random choice tasks, so a total of 300 x 15 = 4500
choice tasks were generated. Some of the details of the report are beyond the scope of the
unit, but you can obtain more information within the Help text. The Freq. column indicates how
many times each level is represented in the design. Within each attribute, you can see that
CBC has perfectly balanced the presentation of levels. The Effic. Column indicates the
relative efficiency of the design. Design efficiency runs from the worst (0.0) to the best (1.0),
and we can see that this particular design is quite efficient with respect to main effects.
If you include prohibitions in your design, sometimes designs can be quite inefficient. In some
cases, you will receive a warning regarding that the design is deficient or will notice asterisks
rather than data in the test design report. You should re-evaluate your design setup should this
occur. Failure to correct the problem can result in unusable data.
Click Close to close the test design report, and click OK to close the Specify CBC Interview
Parameters dialog.
14
Lets now examine the questionnaire weve built to this point. Click Compose | Write
Questionnaire or click the pencil icon on the toolbar, and the following dialog is
displayed:
Youll note that SSI Web has added 15 Random (CBCRAN1 through CBCRAN15) and 2 Fixed
CBC tasks (CBCFIX1, CBCFIX2) to the survey. These questions are based on your settings
and the default template within SSI Web. The CBCFIX tasks arent yet in the proper positions
(we wanted them in the 7th and 9th task positions in the survey). You also havent yet specified
the attribute combinations used for your user-defined Fixed tasks, so SSI Web has inserted the
questions with default levels that well modify shortly.
Before we fine tune those CBC questions, lets turn our attention to the issue of assigning
passwords for respondents to enter at the opening screen of an SSI Web survey.
Assigning Passwords and the Password Screen
Placing a survey on the web makes it convenient for a geographically dispersed population to
take surveys. However, the danger is that the survey may become available to people that
have not been invited to take the survey. Also, some respondents might try to take the survey
multiple times. Assigning respondent passwords is a way to deal with both of these issues.
Password assignment is beyond the scope of this tutorial, so to make things simple well
assume that no passwords are to be used.
To turn off passwords for this project, click Compose | Passwords | Set Passwords and in
the upper-right hand side of the Enter Passwords dialog, un-check both the User Names and
Passwords boxes. SSI Web will ask you to confirm that you want to delete User Names and
Passwords from your study. Finally, youll be asked to specify the maximum number of
respondents for this study. Since we plan to interview about 300 to 450 people in this example,
15
lets specify a comfortable limit such as 1000. After you have typed 1000 click Save and
Close.
Specifying Explanatory Text and Select Questions
(If you are already familiar with adding Text/HTML Filler and Select questions within SSI Web
surveys, you may choose to skip forward to the section entitled Fine-Tuning the CBC
Questions).
Lets enter the introductory text and initial questions using the Write Questionnaire dialog. You
can access the Write Questionnaire dialog by selecting Compose | Write Questionnaire or
by clicking the pencil icon on the toolbar.
The introductory text and survey questions would most likely be initially developed within a word
processing document. Assuming you really had such a document, you might use the Ctrl-C to
copy, and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply
copy-and-paste the text within this document into your SSI Web questions rather than re-type
the text (if viewing this document with Acrobat Reader, you can use the text select icon
from the Acrobat Reader toolbar.)
The first question to compose is the introductory screen:
Introduction:
You are part of a special group of golfers weve selected to ask for feedback regarding
golf equipment. By completing this survey, youll be eligible for a drawing to receive a
free set of PrimoDriver clubs.
In the terminology of SSI Web, this could be a Text/HTML Filler question (it requires no
specific response). But, every SSI Web questionnaire begins with an introductory
<Password> question, even though there may not be passwords used in every project.
The <Password> question is always the first question in the List of Questions.
To edit the <Password> question (which now serves as our text-only introduction
question), highlight <Password> on the List of Questions and click the Edit button.
The Password Settings dialog is displayed:
16
Most question types in SSI Web have a Header 1, Header 2 and Footer section.
These are text areas in which you can insert any text (or HTML, if you know HTML).
When the question is viewed with a web browser, the sections are organized roughly as
follows:
For this introductory screen (that includes three paragraphs of information), it seems to
make sense to place the first paragraph in the Header 1 area, the second paragraph
in the Header 2 area, and the third paragraph in the Footer area.
Type (or cut and paste) the following text for the Introduction question into the text
17
areas in the <Password> question. Put the first paragraph in Header 1, the second
paragraph in Header 2, and the third paragraph in Footer.
You are part of a special group of golfers weve selected to ask for
feedback regarding golf equipment. By completing this survey, youll be
eligible for a drawing to receive a free set of PrimoDriver clubs.
The main dialog only shows a portion of the text, but you can click the pencil icon
adjacent to each text area to expand the text editing window.
To preview how the question will look when it runs on the respondents computer using
Windows Explorer, click the Preview button. The question is displayed in Preview
mode (no data are saved, and no special error checking or validation is performed for
the question).
SSI Web automatically places paragraph breaks (extra blank lines) between the Header
1, Header 2, and Footer sections. If you put all the text in a single section, you may
see that when the web browser interprets the text, it all runs together without any blank
lines between the paragraphs (unless you insert some HTML instructions to force blank
lines between paragraphs). Well talk about using HTML within your text to take greater
control over the layout, font, and style later in this unit.
Click the X icon in the upper-right hand portion of the Preview window to close that
window and return to the previous SSI Web dialog.
After viewing the text in preview mode, you might decide that it is really too small, or that
you want to make the text bold. You can change the size and styles (bold, italic,
underline) or the text justification for the three major text sections by clicking the controls
on the Password Settings dialog. After changing any setting, click Preview again.
18
After you are happy with the layout of the introductory screen, click OK to return to the
Write Questionnaire dialog.
Now you are ready to specify the first Select-Type question:
Frequency:
First, wed like to ask you about how often you golf. Would you say that you golf...
5 times or fewer per year
more than 50 times per year
First, make sure that you highlight the <Password> question on the List of Questions
(when you add a new question, it is placed directly below the highlighted question on the
list). At the Write Questionnaire dialog, click Add, choose Select as the question type
(click CiW, and then choose Select from the available drop-down list), and type
Frequency for the question name. Click OK and the Select Question dialog is shown.
Place the heading text for the question in the Header 1 field. To specify the response
options, click the Response Options button. From the Specify Response Options
dialog, use the Add button to add the four response options for this question. When you
are finished, the question should look something like:
Now that you have seen how to specify Single Select question types in SSI Web, you
have the tools you need to specify the remaining four select-type questions for the golf
ball questionnaire (Skill, Venue, Gender and Incomeplease refer to the questionnaire
text for these questions as presented near the front of this unit).
19
Remember that when you add a new question to the List of Questions, it is inserted
directly below the question that was highlighted when you clicked the Add button. If you
need to move a question once it has been added to the List of Questions, simply
highlight the question to be moved and click the or buttons to move the
questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut
or copy and Ctrl-V to paste questions within the current questionnaire, or even to
another SSI Web study.)
Formatting Text in SSI Web Questions
In addition to the Select questions we added to the questionnaire previously, there are
two text-only, instructional screens to add. Referring to the questionnaire we outlined
earlier, these questions are named IntroCBC and Signoff. Even though the
respondents arent asked to provide specific inputs to these, we refer to these as
questions in the sense that we add them to the questionnaire as if they were standard
questions, and they are listed separately by their question name in the List of
Questions.
You may recall that we placed the very first introduction screen within the <Password>
question. Following that were Select-type questions Skill and Venue. After the Venue
question follows the text-only instructions:
IntroCBC:
In the next section, well be asking you specifically about high performance golf balls.
These balls are premium balls specially engineered to travel farther than the typical ball
with good accuracy.
Wed like you to imagine that you are considering purchasing golf balls for your next golf
outing. Well show you some different high performance golf balls, and ask which one
you would purchase.
Some of the golf balls you are going to see are not currently available on the market, but
wed like you to imagine that they were available today. It is important that you
answer in the way you would if you were actually buying golf balls.
If you wouldnt purchase any of the balls well show you, you can indicate that by
choosing None. By choosing none, you indicate that youd buy another brand, or
would continue using existing balls in your golf bag.
This text layout is a bit more challenging than we dealt with before (when we put three
paragraphs of introductory text into the <Password> question). There are more than
three separate paragraphs here, and well need to deal with the additional element of
bolding selected text.
Add a Text/HTML Filler question directly following the Venue question in the
questionnaire. To do so, highlight the Venue question, click Add, and specify the
Question Name as IntroCBC and the question type as Text/HTML Filler.
20
The Text/HTML Filler dialog is displayed:
If the text-only instruction question was the last question in our survey (or if it were one
of many possible ending points in a survey), we would indicate it was a terminating
question by checking the Terminating Question box. This instructional section is an
introduction to the CBC questions, so well not check that box.
Earlier in this tutorial, when we used the <Password> question to format the opening
page of the survey, we placed each paragraph of text in a separate Header 1, Header
2, or Footer section. We saw that SSI Web automatically places blank lines between
text in these sections. However, with the text in the IntroCBC question, there are many
more paragraph breaks. Well take the opportunity here to introduce the concept of
using a few simple HTML instructions within our survey text.
HTML stands for HyperText Markup Language and provides simple ways for you to
enhance the look of your surveys, such as by bolding or underlining words, or making
paragraph breaks within text. Browsers know how to interpret HTML instructions when
displaying the results to the screen. If you know HTML, you can use it whenever you
wish within SSI Web surveys to accomplish your aims. If you dont know HTML, it isnt
very difficult to learn a few HTML tricks, or you can use the toolbar in SSI Web available
within the editor that appears when you click Pencil icons in SSI Web:
Either type or cut-and-paste the text for the IntroCBC question into the Header 1 text
21
field (Any of the other three fields could be used also, though the Body field is indented).
Click the Pencil icon to display the larger text editing window. Initially, the text appears
something like this:
Even though it appears that there are extra blank lines between the paragraphs, web
browsers ignore these hard returns (as well as more than one consecutive space
characters), so youll need to provide HTML instructions to insert these paragraph
breaks (hard return plus blank line). HTML instructions are placed within <> brackets,
called tags. For example, the HTML instruction to create a paragraph break begins
with an open paragraph tag written as and optionally ends with a close
paragraph tag written as . The text to be formatted as a separate paragraph is
enclosed within these tags. You can either directly type HTML within your document, or
you can highlight text to be modified and click the icons on the HTML toolbar in the
editor. If you highlight the first paragraph with your mouse:
In the next section, well be asking you specifically about high performance golf
balls. These balls are premium balls specially engineered to travel farther than
the typical ball with good accuracy.
And then (with the text in the first paragraph highlighted), click the Paragraph icon
on the toolbar. This inserts a prior to the text and a after the text:
22
 In the next section, well be asking you specifically about high performance
golf balls. These balls are premium balls specially engineered to travel farther
than the typical ball with good accuracy.
When the browser interprets this text, it doesnt display the tags but instead separates
the text enclosed within the tags as a separate paragraph. Repeat the same for each of
the paragraphs in the IntroCBC question.
Next, we need to bold certain words in the text. The HTML tags for bolding text are
 (with the text to be bolded placed between the open and close bold tags). You
can either directly type these tags within the document, or highlight the text to be bolded
with the mouse and click the Bold icon on the toolbar. After you finish separating
the paragraphs with tags and bolding the appropriate text with tags, it
should look something like:
Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web
browser displays this question. It should look like:
23
You should now have the tools necessary to add the other Text/Only HTML Filler
question (Signoff) at the end of the survey. Make sure when adding the Signoff
question at the end of the survey to mark this Text/HTML Filler question as a
Terminating Question. To make it a terminating question, edit the question from the
Write Questionnaire dialog and check the Terminating Question checkbox. SSI Web
may warn you at this point that a terminating question must stand alone on its own
page in the survey. This leads us into our next discussion on page layout.
Page Layout and Global Settings
It may be helpful at this point to review how SSI Web breaks the survey into separate
pages, and some basic global settings that affect the look and functionality of your SSI
Web questionnaire. From the Write Questionnaire dialog, click the Layout button.
24
The Layout dialog shows how the various questions weve specified (or that SSI Web
has automatically added to the list) are arranged across different pages. Notice that the
Select questions (Frequency, Skill and Venue) and IntroCBC weve added to the
questionnaire are all currently arranged on page 2. Lets assume we wanted to break
these up, one question per page. You set a new page break after the Frequency
question by highlighting Skill on the list and clicking the Set Page Break button. Skill is
now placed on page 3. Repeat this operation to place Venue and IntroCBC on
separate pages (pages 4 and 5). Make sure to place the final text screen Signoff on its
own (last) page.
Click OK to return to the Write Questionnaire dialog. While we had been setting page
breaks for our questionnaire, you may have noted that the two fixed CBC tasks
(CBCFIX1 and CBCFIX2) are not yet in the 7th and 9th CBC task positions. You can
move a question on the list by highlighting the question in the List of Questions and
clicking the or buttons to move the question to another point in the
questionnaire. Move CBCFIX1 directly after CBCRAN6 and move CBCFIX2 directly
after CBCRAN7.
25
Specifying Fixed Tasks
A fixed choice task looks just like typical Random (experimentally designed) tasks.
However, rather than let the design algorithm determine the combination of attribute
levels to be shown for each respondent, you specify the codes for the levels to be
displayed in each fixed product concept. Recall that the purpose of specifying fixed
holdout tasks for this example is to achieve a controlled and direct measure of the
existing competitors offerings versus our clients proposed offerings. Youll also be able
to use the results to check the ability of the market simulator you develop using the 15
random choice tasks to predict the responses to the 2 fixed holdout tasks.
Recall that our list of questions already includes two fixed tasks named CBCFIX1 and
CBCFIX2. Lets preview the first fixed task, CBCFIX1. From the Write Questionnaire
dialog, edit CBCFIX1. From the CBC Random and Fixed Questions dialog, click
Preview. Youll notice that the CBCFIX1 question looks exactly like the other CBC
choice tasks weve previewed to this point, however the same attribute levels (level 1)
are used across all three product concepts. This is an initial starting point that SSI Web
creates, which you need to modify for each fixed choice task in your study.
Click OK to return to the CBC Random and Fixed Questions dialog.
To modify the levels displayed in the fixed choice task, click the Fixed Task Designs
button. The following is displayed:
From this one dialog, you can specify the attribute combinations to be used in each of
your fixed tasks. To change from one fixed task to another, use the Fixed Choice Task
drop-down box at the upper-left hand corner of the dialog. Well first modify CBCFIX1.
26
Recall that the first fixed holdout was to display the following three products:

by Smith and Performance Plus by Durango
Forester
Drives 15 yards Drives 10 yards Drives 5 yards

farther than the farther than the farther than the

Use the drop-down controls provided to specify the three product concepts as shown.
Click the OK button when finished. Preview the fixed holdout task to make sure it looks
as you expect. Make any necessary changes.
Next, you need to modify the second fixed holdout task. From the CBC Fixed Choice
Task Settings, used the Fixed Choice Task drop-down control to select CBCFIX2.
Modify CBCFIX2 to have the appropriate specifications for our second fixed choice task:

by Durango Golfers, Inc. by Smith and
Forester
Drives 5 yards Drives 10 yards Drives 15 yards


Again, preview this task to make sure it looks as expected.
Fine-Tuning the CBC Questions
After you have finished these steps, your questionnaire should be nearly complete,
though you havent yet customized the text to display directly above each choice task.
At this point, we suggest you view your survey by clicking Preview Survey under the
Compose | Write Questionnaire area, or by editing/previewing each question one-by-
one. Depending on how picky you are about the look of your survey, you could spend a
good deal of time adjusting each question and adding HTML elements to achieve just
the right look.
Lets turn to editing the CBC tasks. Under Compose | Write Questionnaire, highlight
one of the CBC choice tasks (perhaps CBCRAN1) and click Edit. The following
dialog is displayed:
27
You can modify the text place directly above the CBC task in the Header field. Note that
the changes you make on this dialog apply universally to all CBC choice tasks (Random
plus Fixed tasks). Therefore, if you want to somehow customize the text shown directly
above or below the CBC tasks on a task-by-task basis, you may wish to delete the text
from the Header or Footer field and instead add a Text/HTML Filler question directly
before or after each choice task.
Note that you can control the Size and Style for the Header and the Level Text used in
the CBC task from this dialog.
Click Preview to see the effect of any of your changes.
(The in the Footer field is an HTML-special string that indicates to place a space
character in the Footer field, which is needed for some browsers to add the default
space beneath the choice task. You can change or delete this if you like. Click Preview
to see the effect.)
Testing Your Questionnaire
To test run your questionnaire, we use a tool within SSI Web called Local Web Server.
Local Web Server uses Apache Server technology to make your computer act like a
web server, showing the questionnaire and saving the respondents answers to a data
file. To test run your survey, click Field | Local Web Server (or click the globe icon
). The Local Web Server dialog is displayed:
28
We suggest you use the default Survey Subdirectory Name (your four-character study
name) for this example. When you click Upload Survey, SSI Web copies the files
needed to run the survey into a local folder on your hard drive where the Local Web
Server can use them. A message appears stating that the upload is finished. It is not
necessary for you to know the details of where this folder is or how Local Web Server
works, but if you are interested, you can investigate details in the online Help.
Whenever you make a change to your survey setup and want to test run the survey,
youll need to make sure to click Upload Survey so that the new survey project is copied
into the subfolder used by Local Web Server. If you dont do this, youll end up test
running the old survey, prior to your changes. Whenever you click Upload Survey, any
data that you have entered in your test surveys are deleted (after an initial warning
message).
Click Run Survey, and a browser window opens with your questionnaire. When you are
finished taking the survey, you can close that window by clicking the in the upper-
right hand corner of the browser window.
If you took the survey under Local Web Server mode, you may have noted that there are
pre-programmed error messages that are displayed if, for example, respondents forget
to answer a question on a page, or type a letter into a numeric field. You can access
and edit these error messages from the Global Settings dialog.
Warning: We do not suggest you use Local Web Server for collecting real respondent
data, as there is potential for data loss should someone accidentally click Upload
Survey. There is a free software application for running SSI Web in CAPI mode (stand
alone computer interviewing) called SSI Web CAPI. We suggest using this instead if
you need to conduct real interviews locally (rather than on a Web server).
After you have edited the survey to the point that you are pleased with the look, content,
and functionality of your survey, you should examine the test data youve collected using
Local Web Server (this process is described in a separate tutorial called Getting Started
with SSI Web: A 45-minute Hands-On Tour available on the Help menu), then at least
29
examine the CBC data using Counting analysis (described later). The results should
generally reflect your preferences. If they dont, this suggests something may be wrong
with your study setup.
After you are convinced that the survey is functioning properly, you should also pretest
your survey among your colleagues. They can give you feedback regarding the usability
of the survey, and you can examine the resulting Counts data or perhaps even the part
worths (assuming you have enough data) to make sure the data at least have face
validity.
Pretesting and Fielding
Lets now return to the marketing problem and story we began at the beginning of this
document.
After you have tested the survey in Local Web mode, you post the survey to the Web
using an account set up for you by an ISP (Internet Service Provider). (Setting up your
survey on the web is beyond the scope of this unit, but is described in detail in the SSI
Web help documentation. Fielding options also include hosting on your own companys
server, or using Sawtooth Softwares hosting services).
The next day, you send an email to your client, with a link to take the survey. The
president of Performance Plus first takes a survey and suggests some minor wording
changes. After you make those changes, you invite six other individuals at Performance
Plus take the survey. You download the results and analyze the resulting data. Due to
the small sample size, the results are a bit noisy (and there is an obvious bias toward
your clients balls), but the data seem to feel right.
Next, you recruit six golf enthusiasts within your city to come to a central site to take the
survey. You watch silently as they take the survey. At the end of each session, you ask
each respondent follow-up questions to ensure that there werent any parts that were
difficult to understand or just didnt make sense. You ask them specifically about the
choice tasks, making sure that there wasnt too much information on the screen at once,
and that they didnt feel overwhelmed with the task. After debriefing the test
respondents and analyzing their data to ensure that the results looked reasonable, you
make a few small adjustments and proceed to field.
Using a bulk email program, you send an email to the list of 2000 golfing enthusiasts
from the opt-in list mentioned at the beginning of this unit. After a few weeks, and a
reminder email sent to those who had not yet completed the survey, you achieve 300
total completes.
The completed data reside on the ISPs server where you uploaded the SSI Web survey.
The details for viewing and downloading the data from the server are provided in other
areas of the SSI Web documentation and are beyond the scope of this CBC/Web unit.
Even so, it is useful to at least describe the steps you would perform to access and
download the data from the Web, estimate the part worth utilities, and begin analysis
using the market simulator.
30
Data Management, Utility Estimation, and Moving the Data into SMRT
SSI Web includes an Online Data Management Module so that you can monitor or
access your project from any computer connected to the Web. The Online Data
Management Module is password protected, and your passwords for access are
specified by clicking File | Settings and editing/viewing the Administrative Access
fields. SSI Web generates random administrative access passwords whenever you
create a new study, but you can modify them to suit your needs.
To download the data from the 300 respondents to the golf ball study, you would browse
to the administrative module for your study on the web site (again, we are speaking
hypothetically, as for this tutorial study there is no such site set up). Once at the
administrative module, you would accumulate and download your data, making sure to
save your data (named STUDYNAMEdat.dat, where STUDYNAME is your four-
character study name) to the same folder on your hard drive in which you developed the
SSI Web project.
Once you have downloaded the data, you are ready to export the CBC data to a .CHO
format file (with accompanying attribute labels in a file with extension .ATT), and
estimate part worths in preparation for running market simulations. Assuming you had
downloaded the data as described above, you would click File | Export Data | Prepare
CBC Data Files (*.cho and *.att). If you try this with this tutorial study, you will receive
an error stating that you have no data within your project folder to export. However,
hypothetical data for this project are stored in a tutorial folder within the accompanying
SMRT software that you received together with your SSI Web system.
SMRT stands for Sawtooth Software Market Research Tools and contains the market
simulation tool used to analyze data resulting from any of Sawtooth Softwares conjoint
analysis systems. SMRT is the platform that (given the proper license with its user
identification codes) runs CBC for Windows studies and includes the tutorial data for this
golf ball study. Well discuss how you would move data from your CBC/Web project into
SMRT (hypothetically, since the data arent actually in your CBC/Web project) but then
open the SMRT software system and continue using the golf ball tutorial data that were
installed there.
CBC researchers employ a variety of techniques for analyzing their data. One common
approach for initially understanding the basic, summary preferences for the market is
called Counting analysis. This produces proportions from 0.0 to 1.0 for each level in
your study, reflecting how often this level was chosen, when available in a choice set.
The higher the proportion, the higher preference for the level. While this method of
analysis has good intuitive appeal, there are more powerful ways to analyze the data
based on estimation of part worth utilities and subsequent market simulations.
There are three part worth estimation routines that Sawtooth Software provides. The
first technique is called multinomial logit, which pools respondent data in a single
aggregate model. This technique was the first part worth estimation technique that
Sawtooth Software used for analyzing CBC data. It is the standard base estimation
module provided with the SMRT software that accompanies your CBC/Web software
license. Its main weakness is its susceptibility to the IIA (red-bus/blue-bus) problem,
which is discussed in many technical papers and in the CBC/Web help documentation.
31
One strength of logit is that by pooling respondent data, it can often obtain stable
estimates of interaction terms. Two other estimation techniques are commonly used
today with CBC: Latent Class and Hierarchical Bayes (HB). These techniques model
respondent heterogeneity (recognize differences between respondents in terms of
preferences) and in most cases provide more useful part worths for the purpose of
market simulations.
It is beyond the scope of this tutorial to discuss these part worth estimation methods (or
a fourth method called ICE). These part worth estimation programs all work seamlessly
with the .cho and .att files exported by SSI Web. If you use the logit estimation routine
built into the SMRT software system, you are immediately ready to perform additional
market simulations and analysis within SMRT. If you use either Latent Class or HB
estimation, these routines produce a text-only format files containing case IDs and part
worth estimates that can easily be imported by SMRT.
For simplicity, well assume that you plan to use logit estimation provided by SMRT. To
move the STUDYNAME.cho file into SMRT for analysis by logit, you would (again, this is
for discussion purposes onlyyou should not actually perform these steps for this golf
ball tutorial):
1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software
| Sawtooth Software SMRT.
2. Within SMRT, create a new study for analyzing the results (select File | New and
choose a folder and a studyname). You can choose any folder or studyname
that you want, as this new study functions independently of your original SSI
Web study.
3. Import the STUDYNAME.CHO file into your SMRT study by clicking (from the
SMRT software menu) File | Import, selecting the file type as Choice Data
(*.cho) and browsing to your .CHO file. Once you import the CBC data from the
.CHO file, you can analyze the data.
Analyzing the CBC Data Using Counts
SMRT is a companion software system to SSI Web that can be used for analyzing the
results of CBC/Web studies. To start SMRT, click Start | Programs | Sawtooth
Software | Sawtooth Software SMRT. Open the golf ball study by clicking File | Open
and then browsing to find the Tutor2.smt file located in the ...Program Files\Sawtooth
Software\SMRT\Tutorial folder.
Hypothetical data are provided to be used for the remainder of this tutorial. To access
the data, please select the Tutor2 study (select File | Open, and double-click the
Tutor2.smt file located in your Tutorial directory).
If you used a randomized design (which you did for this project), usually the first step in
analyzing the choice results is to conduct a Counting analysis. CBCs Counts program
reports the percent of times each attribute level was chosen when it was available on the
screen. Counts provides an intuitive measure of the impact of each attribute level on
overall choice for golf balls. Counts are proportions ranging from 0 to 1. For example, a
32
Count of 0.31 for an attribute level would mean that when a golf ball was displayed
including that particular level, respondents chose it 31% of the time.
To access the Counts program, return to the CBC main menu, and select Analysis |
Counts.
By default, the Counts program analyzes all one-way and two-way Count proportions.
Notice also by default that the Counts program uses only responses to the 15
randomized choice tasks you fielded (under Choice Tasks to Include, All Random is
checked). The two fixed holdout tasks are not included. That is because results from
Counts assume random designs where each attribute level appears an equal number of
times with each level of the other attributes. Randomized designs (with no prohibited
level combinations) make it possible to analyze the effect of each attribute level
independent of all other levels. That characteristic does not usually hold if fixed holdout
tasks are included in the analysis.
Click Compute! and the following report is displayed in the report window:
Brand
Total
Total Respondents 300
High-Flyer Pro, by Smith and Forester 0.367
Magnum Force, by Durango 0.336
Eclipse+, by Golfers, Inc. 0.189
Long Shot, by Performance Plus 0.159
Within Att. Chi-Square 416.495
D.F. 3
Significance p < .01
Performance
Total
Drives 5 yards farther than the average ball 0.168

D.F. 2
Price
Total
$4.99 for package of 3 balls 0.397

D.F. 3
Brand x Performance
Total
High-Flyer Pro... Drives 5 yards farther... 0.263
33
High-Flyer Pro... Drives 10 yards farthe... 0.401
High-Flyer Pro... Drives 15 yards farthe... 0.436
Magnum Force, ... Drives 5 yards farther... 0.234
Magnum Force, ... Drives 10 yards farthe... 0.340
Magnum Force, ... Drives 15 yards farthe... 0.431
Eclipse+, by G... Drives 5 yards farther... 0.091
Eclipse+, by G... Drives 10 yards farthe... 0.206
Eclipse+, by G... Drives 15 yards farthe... 0.266
Long Shot, by ... Drives 5 yards farther... 0.083
Long Shot, by ... Drives 10 yards farthe... 0.180
Long Shot, by ... Drives 15 yards farthe... 0.216
Interaction Chi-Square 25.644

D.F. 6
Brand x Price
Total
High-Flyer Pro... $4.99 for package of 3... 0.549
High-Flyer Pro... $10.99 for package of ... 0.204
Magnum Force, ... $4.99 for package of 3... 0.477
Magnum Force, ... $10.99 for package of ... 0.216
Eclipse+, by G... $4.99 for package of 3... 0.292
Eclipse+, by G... $10.99 for package of ... 0.080
Long Shot, by ... $4.99 for package of 3... 0.284
Long Shot, by ... $10.99 for package of ... 0.039

D.F. 9
Performance x Price
Total
Drives 5 yards... $4.99 for package of 3... 0.281
Drives 5 yards... $10.99 for package of ... 0.079
Drives 10 yard... $4.99 for package of 3... 0.400
Drives 10 yard... $10.99 for package of ... 0.130
Drives 15 yard... $10.99 for package of ... 0.185

D.F. 6
34
None
Total
None chosen: 0.211
At first, this report may seem overwhelming, so well break it up and discuss it in pieces.
First, Counts reports that 300 respondents were used. Then, the count proportions for
brand name are displayed:

Eclipse+, by Golfers, Inc. 0.189
High-Flyer Pro was the most preferred ball name and brand on average, being chosen
36.7% of the times that is was presented and available for choice. Your clients ball
name and brand, Long Shot, by Performance Plus, was the least preferred at 15.9%. If
your client markets their ball under the Eclipse+ name with the Golfers, Inc. brand,
choice probability improves from 15.9% to 18.9%. These are ratio quality data, so one
might infer from the counts that using the Golfers, Inc. name and brand increases the
probability of choice by 19% (18.9/15.9 - 1).
Next comes the counts for Performance:

Respondents on average chose balls that flew 15 yards farther more than twice as often
as those that flew 5 yards farther. There appears to be a non-linear effect of
performance on choice probability. Likelihood of choice nearly doubles as performance
increases from +5 yards farther to +10 yards farther than the average ball. Much less is
gained in terms of choice probability by increasing flight from +10 yards to +15 yards. A
preliminary conclusion might be that your client should make sure their ball is rated to
travel at least 10 yards farther than the average ball. But improving the performance to
travel 15 yards farther than the average ball might not be worth the extra manufacturing
cost, if that cost is significant.
Last comes price:

As expected, respondents prefer lower prices over higher ones. Probability of choice
decreases monotonically for each step increase in price.
To this point, youve only analyzed one-way (main effect) effects of attribute levels on
choice. You can further consider the probabilities of choice when a combination of two
attribute levels are available for choice. There are three tables of two-way probabilities
in our study. Rather than look at all three, lets examine the table (brand x price) that
appears (by the Chi-Square statistic) to show the most promise of being interesting (and
35
reflect a potentially significant interaction effect) for our study:
Eclipse+, by G... $10.99 for package of ... 0.080
Long Shot, by ... $10.99 for package of ... 0.039
This table shows the probability of choice for each brand when it was shown at each
price. This table is more involved than the simpler one-way tables you examined.
Therefore, it may be helpful to plot the results with a graphics or spreadsheet package.
If you plot price on the x-axis and probability of choice on the y-axis, it would appear like
the familiar demand curves one learns about in economics. We should note, however,
that counting analysis has some drawbacks and inaccuracies associated with it. There
are more accurate ways to generate demand curves with CBC using the market
simulator. Also, demand curves from CBC assume perfect information, equal
distribution, and other assumptions mentioned in the online Help.
The pseudo demand curves seem to suggest that the Eclipse+ name is preferred over
all levels of price to the Long Shot name. Also, the gap appears to widen slightly as
prices increasethough we cannot tell from this chart whether that is a significant or a
chance occurrence.
After you have spent some time looking at the choice results for the randomized tasks,
you might consider also looking at the results for the fixed holdout tasks. Recall that we
asked these fixed scenarios to gauge preference for what may play out as actual future
market scenarios. We also asked these fixed questions so that we could see how well
the market simulator predicts the preferences for those two questions.
Recall that the first holdout task was asked as the seventh task, and the second holdout
as the ninth. To analyze responses for the first holdout, you need to use the Counts
program again, this time isolating only the seventh choice task. To do that, while in
Counts, in the Choice Tasks to Include Area, un-check the All Random option, then click
the Filter... button. Un-check all of the random choice tasks (by clicking the boxes in
front of the task labels), and select just the first fixed holdout choice task, called
Fixed_T1. Click OK to close the dialog. You will only be interested in one-way (Main
Effect) counts, so in the Level of Analysis area, un-check the 2-Way Interactions
selection (leaving only Main Effect checked). Click Compute!. The Counts for brand
are displayed:
36
Eclipse+, by Golfers, Inc. *
Recall that we did not display Eclipse+ in the first holdout task, so it is marked with an
asterisk. For review, here is the configuration for the first fixed holdout task, with the
choice probabilities listed:
30.7% 23.7% 21.3% 24.3%

by Smith and Performance Plus by Durango None: I
Forester Wouldnt
Purchase Any
Drives 15 yards Drives 10 yards Drives 5 yards of These

If you isolate only the second fixed holdout task (by selecting only Fixed_T2 in the
Choice Tasks to Include dialog) and click Compute!, youll get the following probabilities
for the second holdout task:
18.0% 28.7% 29.6% 23.7%

by Durango Golfers, Inc. by Smith and None: I
Forester Wouldnt
Purchase Any
Drives 5 yards Drives 10 yards Drives 15 yards of These

Each of these choice probabilities are based only on 1 task x 300 respondents = 300
total choice tasks, whereas the previous counting data was based on a total of 15 tasks
x 300 respondents = 4,500 total tasks.
In accordance with the previous conclusion we made based on the random choice tasks,
the holdout data suggest that there is marginal benefit from using the Eclipse+, by
Golfers, Inc. name.
As a final note on counting analysis, one should not put too much stock in the None
percentage. We shouldnt conclude that since the None percentage is running at about
25% that roughly three-quarters of these respondents would be expected to purchase a
37
premium ball for their next golf outing. It is our experience that respondents tend to
exaggerate their likelihood to purchase or choose products in survey research. The
actual proportion of buyers that purchase premium balls would probably be significantly
less.
Now that you have spent some time learning about the Choice results using the Counts
program, you are ready to run logit analysis and construct a market simulation.
Analyzing the Choice Data Using Logit
Logit is the method provided in the base CBC/Web license for analyzing choice results.
There are great advantages to Latent Class and Hierarchical Bayes analysis programs,
but for simplicity well focus on logit for this tutorial.
Logit analysis estimates an effect, or logit utility for each level of each attribute. It also
can be used to estimate interaction effects. A utility refers to a degree of worth or
preference for a product feature. As with any complex analytical tool, we suggest you
learn as much about the technique as you can, to ensure that you interpret the results
wisely. Much more information about logit analysis and logit utilities can be found in the
online help.
To compute logit, select Analysis | Compute Utilities. First click the Settings button.
You should provide the following settings:
Respondents to Include (All)
Respondent Weights (Equal)
Choice Tasks to Include (All Random)
Effects Coding (Main)
Output Precision (5 decimal places)
After you have checked these settings and closed the Logit Settings dialog, click
Compute!. The following report is displayed:
Main Effects
Choice Tasks Included: All Random
Total number of choices in each response category:

1 1135 25.22%
2 1377 30.60%
3 1039 23.09%
NONE 949 21.09%
Files built for 300 respondents.

There are data for 4500 choice tasks.
Iter 1 log-likelihood = -5517.00384 rlh = 0.29335

Converged.
Log-likelihood for this model = -5498.66372

Log-likelihood for null model = -6238.32463
------------
38
Difference = 739.66091 Chi Square =
1479.322
Effect Std Err t Ratio Attribute Level

1 0.52865 0.03219 16.42166 1 1 High-Flyer Pro...
2 0.37167 0.03213 11.56654 1 2 Magnum Force, ...
3 -0.37734 0.03728 -10.12212 1 3 Eclipse+, by G...
4 -0.52298 0.03878 -13.48669 1 4 Long Shot, by ...
5 -0.49336 0.03006 -16.41310 2 1 Drives 5 yards...

6 0.12924 0.02652 4.87399 2 2 Drives 10 yard...
7 0.36411 0.02589 14.06139 2 3 Drives 15 yard...
8 0.66815 0.03219 20.75429 3 1 $4.99 for pack...

9 0.16510 0.03350 4.92893 3 2 $6.99 for pack...
10 -0.09740 0.03527 -2.76111 3 3 $8.99 for pack...
11 -0.73585 0.04092 -17.98437 3 4 $10.99 for pac...
12 -0.02819 0.03825 -0.73693 NONE
The column labeled Effect contains the utilities for each level of each attribute. The
larger the utility, the more preferred the level. The utilities sum to 0 within each attribute
(they are zero-centered). These utilities are used within CBCs market simulator to
compute interest (share of choice) among products in competitive scenarios.
The next column displays the standard errors for each logit effect. Earlier under
Counting analysis, we saw that Eclipse+ was preferred to Long Shot, but we could not
tell at that point if the result was statistically significant. Here, we note that the utility for
Eclipse+ is 0.14564 utility points higher than Long Shot (-0.37734 - -0.52298). We also
note the standard errors for each of these utilities. The pooled standard error for the
difference between these two utilities is equal to the square root of the sum of the
squared standard errors, or sqrt (0.037282 + 0.038782) = 0.0537912. To get the t-value,
we divide the difference in the two utilities by the pooled standard error 0.14564 /
0.0537912 = 2.708. If we look this up on a standard t-table, we see that this difference
is significant at roughly the 99% confidence interval. Therefore, we conclude that the
Golfers, Inc. name is more preferred than the Performance Plus name.
The logit run above reflects main effects only (no interactions). The results from Counts
suggested that interaction effects might be significant. Adding interaction terms to the
logit model (and the resulting simulator) might significantly improve our ability to predict
respondent choices. We will not go into detail in this tutorial regarding how to test for
significance of interaction terms in logit. To briefly summarize the process, since we
believe that changes in price may act differently on different brands, we decide to rerun
the logit model specifying the brand x price interaction. We compare the overall fit (log
likelihood) of the model before and after the inclusion of the interaction terms. With this
data set, we find that the interaction term improves the fit.
To include the interaction between brand and price in your logit run, click Analysis |
Compute Utilities. Click the Settings... button. Make sure your settings are the same
as previously outlined in this section. However, instead of checking Main Effects, un-
check it and then click Effects... and a grid is displayed.
Checks are displayed along the diagonal of the grid, representing main effects only (no
interactions). To specify the interaction between brand and price, additionally check the
39
box in the lowest cell of the left-most column. Click OK. You are returned to the Logit
Settings dialog. Click OK again to close the dialog. Then, click Compute!. A new
solution is computed including main effects and interactions (it may take a few minutes).
To save the run for use in the Market Simulator, choose Save Run. Specify a name by
which to reference that run, such as Main Effects + A1xA3.
Running Market Simulations
Once you have computed logit utilities, you can use those within a market simulator to
predict choices for any combination of products defined using the attributes and levels
you measured. You can test what-if scenarios, conduct sensitivity analysis, and predict
interest in different product concepts in competitive scenarios.
To open the CBC Market Simulator, click Analysis | Market Simulator or double-click
Market Simulator from the study Navigator window.
The logit runs that you have saved are displayed in the Utility Runs list. If you saved a
run (or multiple runs) while working through the last section of the tutorial, these should
appear on the list.
The first time you enter the Market Simulator for a study, the list of Simulation Scenarios
is empty. Create a simulation scenario by clicking Add....
When you click the Add... button, the Scenario Specification dialog is displayed.
Recall that one of the reasons for including fixed holdout tasks in your survey was to
check the accuracy of the market simulator in predicting an outcome for a choice task
that wasnt included in the collection of tasks used to help generate the attribute utilities.
To check this, you decide to specify the first fixed product scenario.
The first step is to type a scenario name into the Name field, in the upper left-hand
corner of the dialog. Type Fixed Holdout Task 1.
Next, youll specify the three products that were included in that choice task. The area
you type the products into looks like the grid of a spreadsheet. The first column is where
you type the product label. As you click the various columns associated with the
attribute levels, the level codes will appear in the window below to remind you what
codes are associated with each attributes levels.
The attribute level codes for the three products that were shown in the first holdout fixed
task were:
Product Name Brand Performance Price
High-Flyer Pro 1 3 4
Long Shot 4 2 2
Magnum Force 2 1 3
40
Specify these three products in the product entry grid. To add new rows to the grid (for
additional products) click the Insert Product button.
After you have specified these three products, there are a few other things we should do
before running the simulation. First, we need to specify a simulation method. For the
purposes of this tutorial, the method well select is called Share of Preference. This
method works well when the products that are specified are basically unique, with little or
no overlap in attribute levels used to define those products. The three products we
entered are unique in terms of brand, performance and price, so our situation seems to
fit this well. If some products shared a level specification (e.g. having the same
performance), we would probably favor using the default Randomized First Choice
method.
Next, we need to tell the simulator to estimate a share of preference associated with the
None product. (We generally suggest not estimating share of preference for the None
product; but in this case, where we want to predict responses to a holdout task that
included a None choice, it makes good sense). To do so, click the Advanced
Settings... button. (The settings on this dialog are described in more detail in online
Help).
Specify a None Weight of 1, and click OK to close the dialog.
Click OK again to close the Scenario Specification dialog and return to the main
Market Simulator dialog.
Choose a utility run to use for your simulation. You should have saved a run that
computed main effects plus the interaction between brand and price. Select that run by
highlighting it in the list of Utility Runs.
To simulate shares for a scenario you have specified, place a check mark in the box
next to the scenario name in the Simulation Scenarios list, and then click Compute!.
The following report is displayed (weve inserted commentary within brackets):

Scenario: Fixed Holdout Task 1
Utility Run: Main Effects + A1xA3
Average Utility Values

Rescaling Method: Diffs
Total
Eclipse+, by Golfers, Inc. -32.49
Long Shot, by Performance Plus -51.56
Drives 5 yards farther than the average ball -41.06


$8.99 for package of 3 balls -7.17
$10.99 for package of 3 balls -70.19
41
High-Flyer Pro... $4.99 for package of 3... -4.60
High-Flyer Pro... $6.99 for package of 3... -6.50
Magnum Force, ... $4.99 for package of 3... -11.64
Magnum Force, ... $6.99 for package of 3... -20.31
Eclipse+, by G... $4.99 for package of 3... -2.01
Eclipse+, by G... $8.99 for package of 3... -5.78
Eclipse+, by G... $10.99 for package of ... -6.40
Long Shot, by ... $10.99 for package of ... -34.82
None -0.38
<<The above average utilities are rescaled logit utilities using the zero-centered diffs
method. The diffs method rescales utilities so that the total sum of the utility differences
between the worst and best levels of each attribute across attributes (main effects) is
equal to the number of attributes times 100. Note: the attribute utilities are influenced by
the number of respondents in the simulation and respondent weighting, but are not
affected by the product specifications you enter. After you have seen these once for a
particular group of respondents, you may choose to omit them in subsequent simulations
by un-checking the Display Utilities box in the Scenario Specification dialog.>>
Product Simulation Settings
Simulation Mode: Simulation
Model: Share of Preference
None Weight: 1
Exponent: 1
Product Specifications
Brand Perf... Price
High-Flyer Pro 1 3 4
Long Shot 4 2 2
Magnum Force 2 1 3
<<Above are the product level codes you specified for the three products in this
simulation. Below are the simulated shares of preference (choice) for these products.>>
Shares of Preference for Products
Total
High-Flyer Pro 30.99
Long Shot 22.16
Magnum Force 22.00
None 24.85
42
In the table below, weve summarized the actual and predicted choices shares for the
first holdout task. Notice that the simulated shares of preference are very similar to the
actual share of choices respondents gave to fixed holdout task number one (which we
analyzed earlier using Counts):
Product Name Actual Choice Simulated Choice Absolute Error

Shares Shares
High-Flyer Pro 30.67 30.99 0.32
Long Shot 23.67 22.16 1.51
Magnum Force 21.33 22.00 0.67
None 24.33 24.85 0.52
To quantify how closely the simulated shares match the actual choice shares, weve
computed the absolute value of the difference between the actual and simulated shares
in the last column. If we average the errors in the last column, we find the Mean
Average Error (MAE) is 0.76. (We should note that the MAE for this example is much
lower than is generally observed when comparing simulation predictions versus actual
holdout shares).
We wont take the space here to show the MAE computation for holdout task #2, though
you may decide to specify another simulation scenario and compute predicted shares for
that task.
You can use MAE (or Mean Squared Error or Chi Square) to compare the results of
different simulation models (i.e. Logit versus Latent Class or HB). You can also use it to
tune the scaling parameter (exponent) or other advanced settings covered in more depth
in the online help.
The Market Simulator is a powerful tool for testing nearly an unlimited number of
possible market scenarios. You can use the simulator to answer the strategic questions
related to the fictitious golf ball situation posed at the beginning of this unit.
For example, the main question facing Performance Plus was whether they could hope
to compete with the two existing performance balls in this market. The simulation results
suggest that, if they were able to provide the specified performance at the simulated
price and could achieve the level of distribution and awareness of the existing brands
(whose specifications and prices didnt change), they could expect a market share
roughly equal to Magnum Force, and lower than the leader, High-Flyer Pro.
43
A secondary question was whether they should market their new golf ball under the
Eclipse+, by Golfers Inc. name. If you specify the second holdout task as a new
simulation scenario, the following shares result (displayed next to the previous
simulation for fixed holdout task #1):
Simulated Shares of Choice
Fixed Task 1 Fixed Task 2

High-Flyer Pro 30.99 High-Flyer Pro 29.20
Long Shot 22.16 Eclipse+ 26.65
Magnum Force 22.00 Magnum Force 20.73
None 24.85 None 23.42
By marketing their ball under the Eclipse+, by Golfers, Inc. name instead of Long Shot,
by Performance Plus, share of choice increases from 22.16 to 26.65, representing a
20% relative increase. If you have a good memory, youll note that this conclusion is
similar to what we inferred from Counts data, and by comparing the actual choices of the
two fixed holdouts. The simulated shares, however, should be both more reliable and
more accurate. They are not subject to some inherent weaknesses of counting analysis,
and they leverage much more data (15 randomized tasks) than isolating only the choices
given to the two fixed holdout tasks.
It is important to remember that the shares of preference resulting from conjoint

predictions are not equivalent to actual market shares, and often look quite different.
Many other factors in the real world factor into market shares and cannot be measured
and reflected solely by conjoint data. Conjoint assumes perfect information, equal
distribution and availability, and that each respondent is in the market and able to
purchase. Conjoint results reflect the potential market acceptance, given proper
promotion, distribution and time.
44

Building A Survey Question Conjoint Cbcwebtut

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Building A Survey Question Conjoint Cbcwebtut

Uploaded by

Copyright:

Available Formats

CBC/Web Tutorial and Example

The Marketing Problem

Defining a List of Attributes and Levels

A Study Plan and Sample Scripted Questionnaire

Thank you for your willingness to participate in this study.

(Click the Next button to continue)

5 times or fewer per year

from 6 to 25 times per year

from 26 to 50 times per year

more than 50 times per year

120 strokes or more

Between 100 and 120 strokes

Between 85 to 100 strokes

Above par, but lower than 85

At or below par (72 or lower)

Do you usually golf at...

Choice Task (one of 15 randomly constructed tasks):

High-Flyer Pro, Magnum Force, Eclipse+, by

$10.99 for package $6.99 for package of $8.99 for package

Here are those two fixed holdout concepts:

Fixed Holdout Task #1 (7th overall choice task):

High-Flyer Pro, Long Shot, by Magnum Force,

$10.99 for package $6.99 for package of $8.99 for package of

Magnum Force, Eclipse+, by High-Flyer Pro,

$8.99 for package of $6.99 for package of $10.99 for package

Less than $30,000

Between $30,000 and $60,000

Between $60,000 and $100,000

More than $100,000

Using SSI Web to Write the Questionnaire

Entering the List of Attributes and Levels

Additional Study Parameters

None: I Wouldnt Purchase Any of These

icon on the toolbar to access a larger edit dialog.

Click the Format tab, to review the following settings:

Generating the Experimental Design

Design Efficiency for Main Effects

A Priori Estimates of Standard Errors for Attribute Levels

Att/Lev Freq. Actual Ideal Effic.

Assigning Passwords and the Password Screen

Specifying Explanatory Text and Select Questions

by clicking the pencil icon on the toolbar.

The first question to compose is the introductory screen:

Thank you for your willingness to participate in this study.

(Click the Next button to continue)

Thank you for your willingness to participate in this study.

(Click the Next button to continue)

Now you are ready to specify the first Select-Type question:

5 times or fewer per year

from 6 to 25 times per year

from 26 to 50 times per year

more than 50 times per year

Formatting Text in SSI Web Questions

Page Layout and Global Settings

Click OK to return to the CBC Random and Fixed Questions dialog.

High-Flyer Pro, Long Shot, by Magnum Force,

Drives 15 yards Drives 10 yards Drives 5 yards

$10.99 for package $6.99 for package of $8.99 for package of

Magnum Force, Eclipse+, by High-Flyer Pro,

Drives 5 yards Drives 10 yards Drives 15 yards

$8.99 for package of $6.99 for package of $10.99 for package

Again, preview this task to make sure it looks as expected.

Fine-Tuning the CBC Questions