Survey Research

OUTLINE Part 1
Types of Surveys Types of Questionnaires/Interviews Conducting a Survey What is asked in a Survey?

IssuesQuestion, Content, Bias

OUTLINE Part 2
Constructing the Questionnaire Types of Structured Questions

Question Content/Wording Dealing with Bias Question Placement

OUTLINE Part 3
Advantages/Disadvantages of Surveys Interviews: Preparation The Interview

Types of surveys
Surveys can be divided into two broad categories: the questionnaire and the interview. Questionnaires usually paper-and-pencil instruments that the respondent completes. Interviews completed by the interviewer based on what the respondent says.

Trochim, 2006

Serial surveys

Serial surveys are those which repeat the same questions at different points in time, producing time-series data. They typically fall into two types: 1. Cross-sectional surveys which draw a new sample each time. In a sense any one-off survey will also be cross-sectional. 2. Longitudinal surveys where the sample from the initial survey is recontacted at a later date to be asked the same questions.
WIkipedia

Questionnaires
Mail survey Relatively inexpensive to administer. Allow the respondents to fill it out at their own convenience. Response rates are often very low. Not the best vehicles for asking for detailed written responses.

Trochim, 2006

Questionnaires
Group administered questionnaire Sample of respondents is brought together and asked to respond to a structured sequence of questions. Each respondent is handed an instrument and asked to complete it while in the room. If respondents are unclear about the meaning of a question they could ask for clarification.

Trochim, 2006

Questionnaires
Household drop-off Researcher goes to the respondent's home or business and hands the respondent the instrument. Respondent is asked to mail it back or the interview returns to pick it up. Respondent can work on the instrument in private. Respondent can ask questions about the study and get clarification on what is to be done.
Trochim, 2006

Interviews
Personal interview Interviewer has the opportunity to probe or ask follow-up questions. Generally easier for the respondent, especially if what is sought is opinions or impressions. Can be very time consuming; resource intensive.

Trochim, 2006

Interviews
Telephone interview Enables a researcher to gather information rapidly. Favorite method of major public opinion pollsters (in the US). Allows for some personal contact between the interviewer and the respondent. Many people don't have publicly-listed telephone numbers. Some don't have telephones. Phone calls are intrusive.
Trochim, 2006

Conducting a survey
Step 1 Develop hypotheses Decide on type of survey (mail, telephone, interview) Write survey questions Decide on response categories Design layout Step 2 Plan how to record data Pilot test survey instrument
Neuman, 2006

Conducting a survey
Step 3 Decide on target population Get sampling frame Decide on sample size Select sample Step 4 Locate respondents Conduct interviews Carefully record data
Neuman, 2006

Conducting a survey
Step 5 Enter data into computers Recheck all data Perform statistical analyses on data Step 6 Describe methods and findings in research report Present findings to others for critique and evaluation.
Neuman, 2006

What is asked in a survey

Behavior How frequently do you go to the mall? Did you vote in the last election? Attitudes/beliefs/opinions What kind of job do you think the President is doing? What is the biggest problem facing the nation? Characteristics Are you a scholar? IS UA&P your 1st, 2nd, 3rd choice? Expectations Do you plan to buy an iPad in the next few months? Self-classification Do you consider yourself to be liberal, moderate or conservative? Knowledge Who was elected President in the last election?
Neuman, 2006

Question issues
What types of questions can be asked? Are you going to ask personal questions, or need lots of details in the responses? Can you anticipate the most frequent or important types of responses and develop reasonable closed-ended questions? How complex will the questions be? The questions may have multiple parts. You may need to branch to sub-questions.
Trochim, 2006

Question issues
Will screening questions be needed? Is the respondent qualified to answer your question of interest. You wouldn't want to ask someone her opinion about a specific computer program without first "screening" her to find out whether she has any experience using the program. Sometimes you have to screen on several variables (e.g., age, gender, experience). The more complicated the screening, the less likely it is that you can rely on paper-and-pencil instruments without confusing the respondent.
Trochim, 2006

Question issues
Can question sequence be controlled? Can you construct in advance a reasonable order of questions? Or, will you ask follow-up questions that you can't easily anticipate? Will lengthy questions be asked? If your subject matter is complicated, you may need to give the respondent some detailed background for a question. Will long response scales be used? If you are asking about different computer equipment people use, have a lengthy response list (CD-ROM drive, floppy drive, mouse, modem, network connection, etc.).
Trochim, 2006

Content issues
Can the respondents be expected to know about the issue? If the respondent does not keep up with the news (e.g., by reading the newspaper, watching television news, or talking with others), he may not even know about the issue you want to ask about. Will respondent need to consult records? Even if the respondent understands what you're asking about, she may need to consult records in order to get an accurate answer. For instance, if you ask how much is spent on shopping in the past month, she may need to dig up receipts and credit card bills. Trochim, 2006

Bias issues
Can social desirability be avoided? Respondents generally want to "look good" in the eyes of others. No one likes to look like he/she don't know the answer, or say anything embarrassing. If you ask people about information that may put them in this kind of position, they may "spin" the response.

Trochim, 2006

Bias issues
Can interviewer distortion and subversion be controlled? Interviewers may distort an interview as well. They may not ask questions that make them uncomfortable. They may not listen carefully to respondents on topics for which they have strong opinions. They may make the judgments that they already know what the respondent would say to a question based on their prior responses, even though that may not be true.
Trochim, 2006

Bias issues
Can false respondents be avoided? With mail surveys it may be difficult to know who actually responded. Did the head of household complete the survey or someone else? Did the CEO actually give the responses or instead pass the task off to a subordinate? Is the person you're speaking with on the phone actually who she says she is?

Trochim, 2006

Constructing the questionnaire

Writing the questions Determine the question content, scope and purpose Choose the response format for collecting information from the respondent Figure out how to word the question to get at the issue of interest Once you have your questions written, determine how best to place them in your survey instrument.

Trochim, 2006

Types of structured questions

Questions based on level of measurement Nominal - the number next to each response has no meaning except as a placeholder for that response.

The choice of "2" for a lawyer and "1" for a truck driver is arbitrary -- from the numbering system used we can't infer that a lawyer is "twice" something that a truck driver is.
Trochim, 2006

Types of structured questions

Questions based on level of measurement Ordinal respondents rank order their preferences. Rank the presidentiables from best to worst _____ Loren Legarda _____ Noli de Castro _____ Mar Roxas _____ Manny Villar
We want the respondent to put a 1, 2, 3 or 4 next to the candidate, where 1 is the respondent's first choice.
Trochim, 2006

Types of structured questions

Questions based on level of measurement Interval most common is a Likert response scale Example: asking an opinion question on a 1-to-5 bipolar scale theres a neutral point and the two ends of the scale are at opposite positions of the opinion)

Trochim, 2006

Types of structured questions

Questions based on level of measurement Semantic differential - an object is assessed by the respondent on a set of bipolar adjective pairs (using 5-point rating scale)

Trochim, 2006

Types of structured questions

Questions based on level of measurement Cumulative or Guttman scale - the respondent checks each item with which he agrees.

The items are constructed so that they are cumulative -if you agree to one, you probably agree to all of the ones above it in the list.
Trochim, 2006

Types of structured questions

Filter or contingency questions
Sometimes you have to ask the respondents one question in order to determine if they are qualified or experienced enough to answer a subsequent one.

Trochim, 2006

Types of structured questions

Filter or contingency questions There are a couple of conventions you should keep in mind when using filters: 1. Try to avoid having more than 3 levels. Too many jumps will confuse the respondent. 2. For only two levels, use graphic to jump (e.g., arrow and box) to help direct the respondent to the correct subsequent question.

Trochim, 2006

Question content
Is the question necessary/useful? Examine each question to see if you need to ask it at all and at the level of detail you currently have. Examples: Do you need the age of each child or just the number of children under 16? Do you need to ask size of dwelling or can you estimate?

Trochim, 2006

Question content
Are several questions needed? Double-barreled questions What are your feelings towards Muslims and lumads? What do you think of proposed changes in the constitution and the organic law creating the ARMM? Rule: Split each of the questions into two separate ones.

Trochim, 2006

Question content
Are several questions needed? Question does not cover all possibilities. For instance, if you ask about earnings, the respondent might not mention all income (e.g., dividends, gifts). Question does not give enough context to interpret the answer. For instance, if you ask about attitudes towards Muslims, can you interpret this without finding out about their attitudes towards religion in general, or non-Christian religious groups?
Trochim, 2006

Question content
Are several questions needed? Question does not determine the intensity of the respondent's attitude or belief. For example, if respondents say they support Gawad Kalinga, you should also ask them whether they would be willing to part with some of their income for it. The intensity of that response is greater if they are willing to back their sentiment of support with their behavior.

Trochim, 2006

Question content
Do respondents have the needed information? For example, let's say you want to ask the question: Do you think the MILF acted correctly when the Supreme Court temporarily restrained the signing of the GRP-MNLF agreement? The respondent won't be able to answer this question if he has no idea what the MILF is or what the GRP-MNLF agreement was. In surveys of television viewing, dont expect the respondents to answer questions about shows they have never watched. Ask a filter question first (e.g., Have you ever watched Pinoy Idol?)
Trochim, 2006

Question content
Does the question need to be more specific? For example, let's say you want to find out respondent's opinions about a specific book. You could ask How well did you like the book? on some scale ranging from "Not At All" to "Extremely Well." But what does it mean to say you liked a book very well? Instead, you might ask: Would you recommend the book to others? Or Have you read other books by that author? Trochim, 2006

Question content
Is the question biased or loaded? Example 1: You might generally be in favor of tax cuts. If you ask a question like: What do you see as the benefits of a tax cut? You're only asking about one side of the issue. You should also ask about the disadvantages of tax cuts. Example 2: You are in favor of feeding programs in public schools and you ask: What do you see as the disadvantages of eliminating the feeding programs? without also asking about the potential benefits.
Trochim, 2006

Question content
Will respondent answer truthfully? For instance, some people are sensitive about answering questions about their exact age or income. In this case, give them response brackets to choose from (e.g., between 30 and 40 years old, between $50,000 and $100,000 annual income).
Sometimes even bracketed responses won't be enough. Some people do not like to share how much money they give to charitable causes (they may be afraid of being solicited even more). Ask the respondent how much money "people you know" typically give in a year to charitable causes.
Trochim, 2006

Question content
Will respondent answer truthfully? You can dispense with asking a question at all if you can obtain the answer unobtrusively If you are interested in finding out what magazines the respondents read, tell them you are collecting magazines for a recycling drive and ask if they have any old ones to donate. You have to consider the ethical implications of such deception!

Trochim, 2006

Question content
Will respondent answer truthfully? Use a sleeper question questions about nonexistent people or events to check whether respondents are being truthful. Example: In a study to determine which new sports should be included in the Olympics, add a fictitious sport such as Space Football, or Arcade Game.

Neuman, 2006

Question wording
Can the question be misunderstood? For instance, if you ask a person for her nationality, it might not be clear what you want (Do you want someone from Malaysia to say Malaysian, Asian, or Pacific Islander?). If you ask for marital status, do you want someone to say simply that hes either married or not married? Or, do you want more detail (like divorced, widow/widower, etc.)? Some terms are just too vague to be useful. For instance, if you ask a question about the "mass media," what do you mean? The newspapers? Radio? Television?
Trochim, 2006

Question wording
Can the question be misunderstood? Another example: What kind of headache remedy do you use? Do you want to know what brand name medicine is taken? Do you want to know about "home" remedies? Are you asking whether the respondent prefers a pill, capsule or caplet?

Trochim, 2006

Question wording
Is the time frame specified? For instance, you might ask: Do you think Congress will amend the constitution by making itself up into a constituent assembly? or something like Do you think Congress could successfully resist amending the constitution? Neither of these questions specifies a time frame (During GMAs remaining term? After 2010?)
Trochim, 2006

Question wording
How personal is the wording? With a change of just a few words, a question can go from being relatively impersonal to probing into your private perspectives. Consider the following 3 questions, each of which asks about the respondent's satisfaction with working conditions:
Are working conditions satisfactory or not satisfactory in the plant where you work? Do you feel that working conditions are satisfactory or not satisfactory in the plant where you work? Are you personally satisfied with working conditions in the plant where you work?
Trochim, 2006

Question wording
How personal is the wording? The first question is stated from a fairly detached, objective viewpoint. The second asks how you "feel." The last asks whether you are "personally satisfied." Be sure the questions in your survey are at an appropriate level for your context. And, be sure there is consistency in this across questions in your survey.

Trochim, 2006

Question wording
Is the wording too direct? Asking a question too directly may be too threatening or disturbing for respondents. For instance, consider a study where you want to discuss battlefield experiences with former soldiers who experienced trauma. Examine the following three start-up question options: How did you feel about being in the war? How well did the equipment hold up in the field? How well were new recruits trained?

Trochim, 2006

Question wording
Is the wording too direct? Question 1 - too direct. May elicit powerful negative emotions based on their recollections. Question 2 - less direct. It asks about equipment in the field, but, for this population, may also lead the discussion toward more difficult issues to discuss directly. Question 3 - least direct and least threatening. Bashing the new recruits is standard protocol in almost any social context. The question is likely to get the respondent talking, recounting anecdotes, without eliciting much stress.
Trochim, 2006

Dealing with bias

Ambiguity. Questions should be specific, avoiding generalities. Example 1: "On a scale from 1 to 10, how popular is GMA at this time?" This begs the question, "popular with whom?" Popular with the respondents themselves, or with their peer? Others will respond in terms of their recollection of past opinion polls. Unbalanced question Example: If the elections were held today, which of the five prospective candidates would you vote for Mar Roxas or one of the others?
Garson, 2008

Dealing with bias

Ambiguity. Questions should be specific, avoiding generalities. Example 2: "Do you favor government policy toward population management?" leaves ambiguous the level of government (central, local) as well as just what dimension of law enforcement is at issue. Example 3: When value preferences are requested by ranking "above average," "average," "below average," etc., ambiguity is introduced unless the question is answered, average in comparison with what? In general all pronouns, nouns, and noun-phrases should have a clear referent. Garson, 2008

Dealing with bias

Mutual exclusivity. Overlapping categories (e.g., 5-10,10-20, 20-30; religion Catholic, Protestant, Christian) can be easily corrected. Non-exhaustive response set. Bias is introduced when the response alternatives available to the respondent leave out valid choices they would otherwise make. Most common example - leaving out such responses as "neutral" or "don't know" when, in fact, respondents may well be neutral or may actually not know, rather than be hiding their "true" responses which the researcher is trying to force out by omitting these categories.
Garson, 2008

Dealing with bias

Mutual exclusivity. Overlapping categories (e.g., 5-10,10-20, 20-30; religion Catholic, Protestant, Christian) can be easily corrected. Non-exhaustive response set. Bias is introduced when the response alternatives available to the respondent leave out valid choices they would otherwise make. Ambiguous terms, such as "parents," can also lead to non-exhaustive response sets, as by omitting the distinction between birth parents and adoptive parents. Likewise, "single" may omit the distinction between "never married," "widowed," and "divorced."

Garson, 2008

Dealing with bias

Residual categories. "Don't know," "Don't care," "Not applicable," and "Did not respond" are separate types of responses. It is better for the instrument to keep them separate so they can be analyzed for patterns, even if they are combined for other purposes in statistical processing. Rank lists. Ranking can be a challenging task. Respondents should not be asked to rank more than four or five items. Beyond that, respondents may give arbitrary rankings just to get past the item.
Garson, 2008

Dealing with bias

Social favorability. Certain topics deal with actions or beliefs for which there is a socially favored position (people are supposed to enjoy parenting, to vote, not to approve of marijuana use, etc.). Such items should be buffered with phrasing such as "Some people favor x, some people favor y, which do you favor" or "There are many reasons why someone might do x instead of y, or do y instead of x. Did you happen to do x or y the last time the opportunity for either came up?"

Garson, 2008

Dealing with bias

Loaded terms Example. "Do you lean more toward the pro-life or toward the pro-abortion position on issue of termination of late-term pregnancies where the health of the mother is threatened?" Biased - one position is labeled with its most favorable label (pro-life, rather than antiabortion), while the other position is labeled with its less favorable label (pro-abortion, rather than pro-choice). As another example, more people will "not allow" something than will "forbid" the same thing.
Garson, 2008

Dealing with bias

Loaded terms Implicit endorsements is another form of loading to be avoided. "Doctors recommend drinking eight glasses of water a day. What do you think is the correct amount for you?" is an example with an endorsement by a prestigious class. Sensitive topics questions about sexual behavior, drug or alcohol use, deviant behavior. Example: Have you shoplifted? has an accusatory tone and implies an illegal act. Instead, ask Have you ever taken anything from a store without paying for it?
Garson, 2008

Dealing with bias

Leading questions Example: "Do you favor an increase in the minimum wage to P500?" This example is not as leading as an item of the type, "You favor X, don't you?" but it is still leading because it does not legitimize both affirmative and negative responses. A negative response may appear mean-spirited toward the poor, and this may bias the survey toward too many affirmative responses. A better method is to phrase such policy options in the form, "Some people favor X, while other people are opposed to X. What is your opinion?"
Garson, 2008

Dealing with bias

Unfamiliar terms and jargon Example: "Do you think it is timely for government to adopt measures that will mitigate global warming or climate change?" Terms such as global warming or climate change" are apt to not be well understood by typical survey populations. The interviewer must precede the item with a brief explanation. Wherever possible, familiar terms should be substituted for unfamiliar terms.

Garson, 2008

Dealing with bias

Requiring inaccessible information. An item may use familiar terms but require information most respondents would not know. For instance, a question about the GRP-MNLF agreement might have been acceptable 10 years ago, but today's population of respondents is apt to know little about this subject and perhaps not even recognize MNLF" The item "Is your family income above, about equal to, or below the official poverty rate?" is similarly flawed.
Garson, 2008

Dealing with bias

Multidimensionality Example 1: "On a scale of 1 to 10, please rank the performance of the president?"
The respondent may be torn between multiple dimensions: personal vs. official performance, or domestic vs. foreign policy performance, for instance.

Example 2: "Do you believe Romulo Neri lied under oath in the ZTE scandal, impairing his effectiveness as a major government official?"
This item calls for two opinions, not one: one on lying under oath and one on subsequent effectiveness. It should be broken into two separate items.
Garson, 2008

Dealing with bias

Compound items Example: "Do you have or have you ever had a physical, mental, or other health condition which has lasted over six months and which has limited the kind of work you could do on your job?" Better broken into two items: "Do you have or have you every had a physical, mental, or other health condition which has lasted over six months?" and the follow-up item, "If you answered yes to the previous question, did this condition limit the kind of work you could do on your job?"
Garson, 2008

Dealing with bias

Recall items. People's ability to recall the past is limited. The more current and specific the question reference, the better. If recall is necessary, the time frame should be as recent as possible and not over six months unless the reference is to major events (ex., marriage, changing jobs, buying a car). Telescoping when respondents compress time when answering about past events, overreporting recent events, and underreporting distant past ones. May be corrected thru landmark anchoring ask whether something occurred before or after a major event. Garson, 2008

Dealing with bias

Complexity and memory overload. It is possible to overtax the respondent by requiring an excessive memory burden due to complexity. The explanation should not demand the respondent to learn complex new material. Double negatives: Do you disagree with the statement No graduate students in the Asian Center should be allowed not to take a comprehensive exam. If there are over 5 alternatives, a show card should be used to allow the respondent to view the alternatives, not simply hear them orally in an interview situation.
Garson, 2008

Dealing with bias

Poor grammatical format. Example: "Would you say that you approve very strongly, strongly, ..." presents "dangling alternatives" which the respondent must memorize before even knowing what the question is. This format is frustrating and may bias responses toward the first-presented response, or toward negativity. Putting the subject first is the preferred order.

Garson, 2008

Dealing with bias

Hypothetical items (ex., "What would you do if ...") creates a difficult challenge for respondents. People tend to base responses to such items on their most-related actual experiences, and it may be better to ask about such experiences directly.

Garson, 2008

Dealing with bias

Inappropriate assumptions. Items should not contain false or arguable premises. Example: "How much improvement in your life would passage of the reproductive health bill make? A great deal, some, very little, or none?" is an item which assumes that the effect of the bills passage could not be negative on a respondent, forcing respondents who believe the effect would be negative to either skip the item or give a response which does not represent their views.

Garson, 2008

Dealing with bias

Gender bias. Avoid gender-specific wording in items.
A reference to "man," for example, may refer to humans while others will interpret the reference to refer to males, thereby creating an unwanted response ambiguity. It may also be offensive.

Mismatched item and response set. The response categories should be appropriate to the dimension probed by the item.
Example: "How unusual do you think it is for a person to accept P500 to vote for a certain candidate? A great deal, some, very little, or none?" A great deal unusual," "some, unusual," etc., are not grammatically acceptable responses and will Garson, 2008 confuse respondents.

Dealing with bias

Language differences. One must assure that items have the same meaning when the questionnaire is to be administered to populations speaking different languages. One must consider three dimensions of the problem: semantic equivalence, conceptual equivalence, and normative equivalence of items. Semantic equivalence is sought through the translation/back translation method, having independent translators translate from one language to another, and then back again, to see if the original and re-translated item remain the same. Garson, 2008

Dealing with bias

Language differences. One must assure that items have the same meaning when the questionnaire is to be administered to populations speaking different languages. Conceptual equivalence may be sought by such methods as factor analysis, demonstrating that items in one language version have the same factor structure (load similarly on underlying factors) as in another language version.

Garson, 2008

Dealing with bias

Language differences. One must assure that items have the same meaning when the questionnaire is to be administered to populations speaking different languages. Normative equivalence requires such strategies as developing close relations with respondents, using trusted agents for interviewing, providing assurance of anonymity or confidentiality, and pilot testing extensively.

Garson, 2008

Question placement
Decisions about placement Which topics should be introduced early in the survey, and which later? If you leave the most important questions until the end, the respondents may be too tired to give them the kind of attention you would like. If you introduce them too early, the respondents may not yet be ready to address the topic, especially if it is a difficult or disturbing one.

Trochim, 2006

Question placement
Decisions about placement Consider the following questions: Is the answer influenced by prior questions? Does question come too early or too late to arouse interest? Will the question receive sufficient attention?

Trochim, 2006

Question placement
Opening questions First impressions are important in survey work. The first few questions you ask will determine the tone for the survey, and can help put your respondent at ease. The opening few questions should, in general, be easy to answer. Start with some simple descriptive questions that will get the respondent rolling. Never begin your survey with sensitive or threatening questions.

Trochim, 2006

Question placement
Sensitive questions Before asking difficult or uncomfortable questions, attempt to develop some trust or rapport with the respondent. Make sure that the sensitive material does not come up abruptly or appear unconnected with the rest of the survey. It is often helpful to have a transition sentence that will give the respondent some idea of the kinds of questions that are coming.
Example: In this next section of the survey, we'd like to ask you about your personal relationships. Remember, we do not want you to answer any questions if you are uncomfortable doing so.

Trochim, 2006

Question placement
Checklist
Start with easy, nonthreatening questions Put more difficult, threatening questions near end Never start a mail survey with an open-ended question For historical demographics, follow chronological order Ask about one topic at a time When switching topics, use a transition Reduce response set (the tendency of respondent to just keep checking the same response) For filter or contingency questions, make a flowchart
Trochim, 2006

Question placement
Keep the following in mind: Thank the respondent at the beginning for allowing you to conduct your study Keep your survey as short as possible -- only include what is absolutely necessary Be sensitive to the needs of the respondent Be alert for any sign that the respondent is uncomfortable Thank the respondent at the end for participating Assure the respondent that you will send a copy of the final results
Trochim, 2006

Advantages and disadvantages of surveys

Advantages It is an efficient way of collecting information from a large number of respondents. Statistical techniques can be used to determine validity, reliability, and statistical significance. Surveys are flexible in the sense that a wide range of information can be collected. They can be used to study attitudes, values, beliefs, and past behaviours. Because they are standardized, they are relatively free from several types of errors.

WIkipedia

Advantages and disadvantages of surveys

Advantages They are relatively easy to administer. There is an economy in data collection due to the focus provided by standardized questions. Only questions of interest to the researcher are asked, recorded, codified, and analyzed. Time and money is not spent on tangential questions.

WIkipedia

Advantages and disadvantages of surveys

Disadvantages They depend on subjects motivation, honesty, memory, and ability to respond. Subjects may not be aware of their reasons for any given action. They may have forgotten their reasons. They may not be motivated to give accurate answers, in fact, they may be motivated to give answers that present themselves in a favorable light. Structured surveys, particularly those with closed ended questions, may have low validity when researching affective variables.
WIkipedia

Advantages and disadvantages of surveys

Disadvantages Although the chosen survey individuals are often a random sample, errors due to non-response may exist. That is, people who choose to respond on the survey may be different from those who do not respond, thus biasing the estimates.

WIkipedia

Advantages and disadvantages of surveys

Disadvantages Survey question answer-choices could lead to vague data sets because at times they are relative only to a personal abstract notion concerning "strength of choice". For instance the choice "moderately agree" may mean different things to different subjects, and to anyone interpreting the data for correlation. Even yes or no answers are problematic because subjects may for instance put "no" if the choice "only once" is not available.

WIkipedia

Interviews: preparation
Role of the interviewer Locate and enlist cooperation of respondents. In door-to-door surveys, this means being able to locate specific addresses and finding times when respondents are most readily available. Motivate respondents to do good job The interviewer has to be motivated and has to be able to communicate that motivation to the respondent. Often, this means that the interviewer has to be convinced of the importance of the research.
Trochim, 2006

Interviews: preparation
Role of the interviewer Clarify any confusion/concerns The interviewer has to be able to respond candidly and informatively when respondents raise objections or concerns that were not anticipated. Observe quality of responses Whether the interview is personal or over the phone, the interviewer is in the best position to judge the quality of the information that is being received. Conduct a good interview
Trochim, 2006

Interviews: preparation
Role of the interviewer Conduct a good interview! Some respondents are motivated and attentive, others are distracted or disinterested. The interviewer also has good or bad days. Assuring a consistently high-quality interview is a challenge that requires constant effort.

Trochim, 2006

Interviews: preparation
Training the interviewers The major topics that should be included in interviewer training: 1. Describe the entire study Interviewers should learn about the background for the study, previous work that has been done, and why the study is important. 2. State who is sponsor of research Interviewers need to know who they are working for. They -- and their respondents -- have a right to know not just what agency or company is conducting the research, but also, who is paying for the research.
Trochim, 2006

Interviews: preparation
Training the interviewers 3. Teach enough about survey research Sometimes it may not be apparent why a question or set of questions was asked in a particular way. The interviewers will need to understand the rationale for how the instrument was constructed. 4. Explain the sampling logic and process Naive interviewers may wonder why you go through all the difficulties of selecting the sample so carefully. Explain that sampling is the basis for the conclusions that will be reached.
Trochim, 2006

Interviews: preparation
Training the interviewers 5. Explain interviewer bias This is especially a problem when investigating political or moral issues on which people have strongly held convictions. While the interviewers may think they are doing good for society by slanting results in favor of what they believe, they need to recognize that doing so could jeopardize the entire study in the eyes of others. 6. "Walk through" the interview It's a good idea to walk through the entire protocol so the interviewers can get an idea of the various phases and how they interrelate.
Trochim, 2006

Interviews: preparation
Training the interviewers 7. Explain respondent selection procedures, including Reading maps - In personal interviews, the interviewer may need to locate respondents who are spread over a wide geographic area. And, they often have to navigate in neighborhoods they're not familiar with. Teaching basic map reading skills and confirming that the interviewers can follow maps is essential.

Trochim, 2006

Interviews: preparation
Training the interviewers 7. Explain respondent selection procedures, including Identifying households - In your study, you may want to interview only people who live in single family homes. It may be impossible to distinguish townhouses and apartment buildings in your sampling frame. The interviewer must know how to identify the appropriate target household.

Trochim, 2006

Interviews: preparation
Training the interviewers 7. Explain respondent selection procedures, including Identifying respondents For instance, your study may require that you speak with a female head-of-household between the ages of 30 and 40 who has children under 18 living in the same household. It may be impossible to obtain statistics in advance to target such respondents. The interviewer may have to ask a series of filtering questions before determining whether the respondent meets the sampling needs.
Trochim, 2006

Interviews: preparation
Training the interviewers 8. Rehearse interview The interviewers should be very familiar with the entire interview before ever facing a respondent. 9. Explain supervision In order to assure the quality of the responses, the supervisor may have to observe a subsample of interviews, listen in on phone interviews, or conduct follow-up assessments of interviews with the respondents. This can be very threatening to the interviewers. You need to develop an atmosphere where interviewers and supervisors feel like they're working together Trochim, 2006 towards a common end.

Interviews: preparation
Training the interviewers 10.Explain scheduling The interviewers have to understand the demands being made on their schedules and why these are important to the study. In some studies it will be imperative to conduct the entire set of interviews within a certain time period. It's important to have the interviewers available when it's convenient for the respondents, not necessarily the interviewer.

Trochim, 2006

Interviews: preparation
Interviewer's kit A "professional-looking" notebook (having the logo of the company or organization conducting the interviews) Maps Sufficient copies of the survey instrument Official identification (preferable a picture ID) A cover letter from the Principal Investigator or Sponsor A phone number the respondent can call to verify the interviewer's authenticity
Trochim, 2006

The interview
To the outsider, an interview looks like a fairly standard, simple, prosaic effort.
To the interviewer, it can be filled with nuances and interpretations that aren't often immediately apparent. Every interview includes 1. The opening, where the interviewer gains entry and establishes the rapport and tone for what follows. 2. The middle game, the heart of the process, that consists of the protocol of questions and the improvisations of the probe. 3. The endgame, the wrap-up, where the interviewer and respondent establish a sense of closure.
Trochim, 2006

The interview
Opening remarks 1. Gaining entry The way the interviewer appears initially to the respondent has to communicate some simple messages -- that you're trustworthy, honest, and non-threatening. Cultivating a manner of professional confidence, the sense that the respondent has nothing to worry about because you know what you're doing -- is difficult to teach but an indispensable skill for achieving initial entry.

Trochim, 2006

The interview
Opening remarks 2. Doorstep technique You need to be brief. State why you are there and suggest what you would like the respondent to do. Instead of saying "May I come in to do an interview?", you might try a more imperative approach like " I'd like to take a few minutes of your time to interview you for a very important study."

Trochim, 2006

The interview
Asking the questions 1. Use questionnaire carefully, but informally Establish a rapport with the respondent. If you have your face in the instrument and you read the questions, you'll appear unprofessional and disinterested. Use eye contact and a confident manner to set the tone for the interview and help the respondent get comfortable.

Trochim, 2006

The interview
Asking the questions 2. Ask questions exactly as written Sometimes an interviewer will think that he/she could improve on the tone of a question by altering a few words to make it simpler or more "friendly." If you had a problem with a question, the time to raise it was during the training and rehearsals. It is important that the interview be as standardized as possible across respondents. You may think the change you made was inconsequential when, in fact, it may change the entire meaning of the question or response.
Trochim, 2006

The interview
Asking the questions 3. Follow the order given Sometimes the respondent brings up a topic that you know will come up later in the interview. You may be tempted to jump to that section of the interview while you're on the topic. You are more likely to lose your place. You may omit questions that build a foundation for later questions.

Trochim, 2006

The interview
Asking the questions 4. Ask every question Sometimes you'll be tempted to omit a question because you thought you already heard what the respondent will say. For example, let's say you were conducting an interview with college age women about the topic of date rape. In an earlier question, the respondent mentioned that she knew of a woman who had been raped on a date within the past year.
Trochim, 2006

The interview
Asking the questions 4. Ask every question A few questions later, you are supposed to ask "Do you know of anyone personally who was raped on a date?" You figure you already know that the answer is yes, so you decide to skip the question.

Trochim, 2006

The interview
Asking the questions 4. Ask every question Instead, you might say something like "I know you have already mentioned this, but do you know of anyone personally who was raped on a date?" The respondent may say something like "Well, in addition to the woman who lived down the hall in my dorm, I know of a friend from high school who experienced date rape." If you hadn't asked the question, you would never have discovered this detail.

Trochim, 2006

The interview
Asking the questions 5. Don't finish sentences Once you know where a sentence seems to be heading, you may be aching to finish the sentence. You should practice the art of patience (and silence) before doing any interviewing.

Trochim, 2006

The interview
Obtaining adequate responses - the probe When the respondent gives a brief, cursory answer, how do you elicit a more thoughtful, thorough response? 1. Silent probe The most effective way to encourage someone to elaborate is to do nothing at all - just pause and wait. It suggests to the respondent that you are waiting, listening for what they will say next. 2. Overt encouragement Try to do so in a way that does not imply approval or disapproval of what the respondent said (that could bias their subsequent results).
Trochim, 2006

The interview
Obtaining adequate responses - the probe 3. Elaboration You can encourage more information by asking for elaboration. For instance, it is appropriate to ask questions like "Would you like to elaborate on that?" or "Is there anything else you would like to add?" 4. Ask for clarification Sometimes, you can elicit greater detail by asking the respondent to clarify something that was said earlier. You might say, "A minute ago you were talking about the experience you had in high school. Could you tell me more about Trochim, 2006 that?"

The interview
Recording the response Although we have the capability to record a respondent in audio and/or video, most interview methodologists don't think it's a good idea. Respondents are often uncomfortable when they know their remarks will be recorded word-forword. They may strain to only say things in a socially acceptable way. Although you would get a more detailed and accurate record, it is likely to be distorted by the very process of obtaining it. In general, personal interviews are still best when recorded by the interviewer using pen and paper. Trochim, 2006

The interview
Recording the response 1. Record responses immediately This conveys the idea that you are interested enough in what the respondent is saying to write it down. record certain key phrases or quotes verbatim. You need to develop a system for distinguishing what the respondent says verbatim from what you are characterizing (how about quotations, for instance!). 2. Include all probes You need to indicate every single probe that you use. Develop a shorthand for different standard probes. Use a clear form for writing them in (e.g., place probes in the left margin). Trochim, 2006

The interview
Recording the response 3. Use abbreviations where possible Abbreviations will help you to capture more of the discussion. Develop a standardized system (e.g., R=respondent; DK=don't know).

Trochim, 2006

The interview
Concluding the interview 1. Thank the respondent Even if the respondent was troublesome or uninformative, it is important for you to be polite. 2. Tell the respondent when you expect to send results You owe it to your respondent to show them what you learned. It's common practice to prepare a short, readable, jargon-free summary of interviews that you can send to the respondents.
Trochim, 2006

The interview
Concluding the interview 3. Don't be brusque or hasty Allow for a few minutes of winding down conversation. The respondent may want to know a little bit about you or how much you like doing this kind of work. They may be interested in how the results will be used. Use these kinds of interests as a way to wrap up the conversation. Some respondents may want to keep on talking long after the interview is over. You have to find a way to politely cut off the conversation and make your exit.
Trochim, 2006

The interview
Concluding the interview 3. Don't be brusque or hasty Immediately after leaving -- write down any notes about how the interview went. Go over your notes and make any other comments and observations.

Trochim, 2006

Issue

Questionnaire
Group Mail

Interview

DropPersonal Phone Off

Are Visual Presentations Possible? Yes

Are Long Response Categories Possible? Yes Is Privacy A Feature? No Is the Method Flexible? No Are Open-ended Questions Feasible? No Is Reading & Writing Needed? ??? Can You Judge Quality of Response? Yes Are High Response Rates Likely? Yes

Yes
Yes Yes No No Yes No No

Yes
Yes No No No Yes ??? Yes

Yes
??? Yes Yes Yes No Yes Yes

No
No ??? Yes Yes No ??? No

Can You Explain Study in Person? Yes

Is It Low Cost? Yes Are Staff & Facilities Needs Low? Yes

No
Yes Yes

Yes
No No

Yes
No No

???
No No

Does It Give Access to Dispersed Samples? No

Does Respondent Have Time to Formulate Answers? No Is There Personal Contact? Yes Is A Long Survey Feasible? No Trochim, 2006 Is There Quick Turnaround? No

Yes
Yes No No Yes

No
Yes Yes No No

No
No Yes

No
No No

Yes No Trochim, 2006 No Yes

Next: Analysis of quantitative data