Social Research Methods/Surveys

Survey Research

Certain topics are more suited for survey research than others. Surveys are mainly used in social research that has individual people as the unit of analysis. These individual people are also called respondents or a person who provides data for analysis by responding to survey questionnaire. Survey research is best used to gain information about large populations that the researcher can not exactly observe. Surveys are also used to gather information about attitudes and orientation of a large population. Well known survey groups like Gallup, Harris, Roper, and Yankelovich do this.

Survey Questions

Research should only pose questions to which they believe the respondents know the answers. For instance, questions pertaining to the respondent’s own behavior, beliefs, or demographic characteristics are considered appropriate. Questions that require the respondent to provide answers about other people should be avoided because it is often particularly difficult for respondents to provide fully unbiased information. Questions that ask respondents to assess issues of causality or require the respondents to make scientific or technical judgments should also be avoided. It is not uncommon for respondents to feel threatened by the questions that are posed. In these instances, researchers must take special measures to ensure that the subject feels comfortable responding truthfully. Questions that pertain to issues such as sexual behavior, drug abuse, and criminal behavior are often under- or over-reported. This is often a consequence of the social desirability bias, which is the tendency of respondents to reply in a fashion that they feel will be perceived favorably by others. The social desirability bias typically results in the over-reporting of “good” behavior and the under-reporting of “bad” behavior.

When it comes to asking questions in a questionnaire researcher can use one of two different methods: open-ended questions or closed-ended questions. Open-ended questions are questions for which the respondent is asked to provide his or her own answer. These survey questions need to be coded before the researcher can process them using computer analysis. The coding of open-ended questions entails that the researcher explain the meaning of the response. There is also the possibility that the respondent could give answers that are not relevant to the researcher. Close-ended questions are survey questions in which the respondent is asked to select an answer from among a list provided by the researcher. These survey questions tend to be more popular in social research because they give the researcher more uniformity in their answers and able to be easily processed. The format for closed-ended questions should follow the following two requirements. The categories that the respondents are given must be comprehensive or exhaustive. This means that they should include all of the possible responses. Next, the category of answers must be mutually exclusive. This means that the respondent should only want to select one of the possible answers given.

Different Survey Methods

Self Administered Questionnaires: questionnaires in which respondents are asked to complete without assistance.
Goal is to have a good response rate which would be higher than 30%
Avoid response bias where only highly motivated people return a survey
Mail in surveys are considered the best to use
Interview Surveys: type of survey where questionnaire is administered to the respondent by a face-to-face interviewer.
There are guidelines established for proper interviewing
Face to face interviews are commonly used and have high response rates.
Interview bias is usually observed during face to face interviews in which the respondent changes his or her answer because of certain characteristics of the interviewer.
Telephone Surveys: questionnaires administered over the phone.Telephone surveying has a number of advantages. First of all, 95% of all households have phones, making it a perfect method to reach a wide population of people. Secondly, Surveying over the phone is relatively cheap and time efficient. There are, of course some drawbacks of this method. Some people might not have phones at all or only have cell phones. Also, some numbers may be unlisted, thus that part of the population cannot be reached. Researchers use such tools as RDD (Random Digit Dialing), which insures the randomness of the sample, and CATI (Computer-Assisted Telephone Interviewing), which allows for a simpler and faster data entry.
Response rates are very high
Online Surveys: questionnaires administered online.
These must be a very easy for the respondent to complete.
It may be beneficial to offer incentives in order for respondent to become interested.
This type of survey should not be very long in order to maintain the attention of the respondent.

Questionnaire Construction

Researchers should do the following things when constructing a survey questionnaire: Researchers should make items clear, questionnaires should be unambiguous. Questionnaire questions should be so precise that the respondent knows what the researcher is asking. Researchers should avoid double-barreled questions. These are questions that a respondent could answer in both a positive and negative way or have multiple parts to an answer (e.g. do you support president obama and health care reform?). Respondents must be competent to answer the questions. This means that the researcher should make sure that the respondent can indeed give an accurate and reliable answer to the question being asked. The respondents must be willing to answer the questions given. The questions should be relevant to the life of the respondent. Short items listed in a survey are the best because it will hold the undivided attention and interest of the respondent. Researchers should avoid negative items. This means that the researcher should attempt to avoid using negation in survey questions. And finally researcher should avoid using biased items and terms. A bias is that quality of a measurement device that tends to result in a misrepresentation of what is being measured in a particular direction.

When constructing a survey, is important to consider the effect that a "no answer" option may have on participants. Thus there are generally three routes that are taken concerning this issue:

standard format questions- "no opinion" or "no answer" is not allowed to be a choice in the survey
quasi-filter questions- these offer a "no opinion" or "no answer" alternative to the other responses. These questions can therefore filter out participants who do not hold strong views about the issue being studied
full-filter questions- these remind participants that they are permitted to not have an opinion about the issue. These are often used in the form of screening questions to again filter out those who do not have strong opinions

It is also important to filter out respondents who are not accurately reporting their knowledge about the researched issue. Sometimes people may not know the answer to questions or do not have the resources necessary to answer them, but are not willing to admit they cannot answer. Thus sleep questions can be used to make sure that people are reporting what they know accurately. Things to avoid when developing a survey questionnaire:

biased and emotional language
leading questions
double barreled questions (ask multiple questions but only allow for one answer)
double negatives
questions about the future or making predictions
questions the respondent can not answer or will not be comfortable answering
ambiguity or vagueness in question wording
slang, abbreviations or jargon, do use standard English

Remember: It is important that the questions in a survey do not affect how the respondent answers them, leading questions or questions that can not be clearly answered may affect how accurately a subject answers a survey.

Survey Response

It is quite evident that the response or completion rates of surveys are affected by the method of survey that is used. For instance, it may be quite obvious that as the length of a questionnaire increases, the rate of response generally decreases, as people tend to lose patience during long questionnaires. Mail interviews over 4 pages long and phone interviews over 10 minutes long are known to have decreased response rates. On the other hand, it is not uncommon for face-to-face interviews to last as long as 45 minutes, presumably because it is much more difficult for respondents to attempt to prematurely terminate an ongoing interview while directly facing their interviewer than it is for them to simply not turn in a long paper questionnaire or hang up the phone during a long interview. Therefore, the best response rates can typically be obtained from surveys that involve face-to-face communication.

There are five groups who will not answer surveys. These include:

Nonlocation - the subject could not be located, even though they were included in the sampling frame. This is usually due to errors in the sampling frame.
Noncontact - the subject was never found at their home after repeated tries to contact them.
Ineligibility - the subject was found not to be eligible for the survey.
Refusal - the subject refused to partake in the survey.
Noncompletion- the subject did not answer all of the questions in the survey.

Strengths and Weaknesses in Survey Research

There are both major strengths and weaknesses to surveys. By using larger number of respondents, inquiries regarding general populations such as student bodies, cities or even nations can be referenced to survey finding. Next to objective facts such as birth records, election polls, etc., survey research is the best means for general observations/data. Also, surveys allow for operation definitions to be assumed from actual observations opposed to experimental research where speculations are usually done prior to the research. Although flexibility is granted with survey research, a universal definition to what the survey asks is needed for all participants, otherwise there can be biases or result in faulty findings. Furthering this, because an ideal questionnaire needs to have the same definitions for the questions asked, crucial data can be neglected due to vagueness. The most general questions must relate to the majority, if not all the participants, in some way to ensure a functional survey with little bias. When conducting research, a survey usually avoids contexts of social life as well as maintains unchanged throughout the research. This can be unfavorable when new findings are observed in the initial survey and very little can be done until the research is concluded. Generally survey research is not very valid but can be very reliable. Artificiality can lead the respondent to answer questions favoring how the researcher phrased the question. A second flaw with artificiality is having the respondent be indifferent to a question prior to the study but feels pressured into giving an opinion, an issue that can happen frequently.

Key Terms in Survey Research

Survey:predetermined set of questions posed to a predetermined population
Respondent: A person who provides data for analysis by responding to a survey questionnaire.
Questionnaire: A document containing questions and other types of items designed to solicit information appropriate for analysis. Questionnaires are used primarily in survey research but also in experiments, field research, and other modes of observation.
Open-ended Questions: Questions for which the respondent is asked to provide his or her own answers. In-depth, qualitative interviewing relies almost exclusively on open-ended questions.
Close-ended Questions: Survey questions in which the respondent is asked to select an answer from among a list provided by the researcher. Popular in survey research because they provide a greater uniformity of responses and are more easily processed than open-ended.
Bias: That quality of a measurement device that tends to result in a misrepresentation of what is being measures in a particular direction. For example, the questionnaire item "Don't you agree that the president is doing good job?" would be biased in that it would generally encourage more favorable responses.
Social Desirability Bias:Tendency for respondents to reply in a manner that they feel will be viewed favorably by others. (e.g. the reduction/denial of sexual behavior, drug abuse, criminal behavior)
Contingency Question: A survey question intended for only some respondents, determined by their responses to some other question. For example, all respondents might be asked whether they belong to the Cosa Nostra, and only those who said yes would be asked how often they go to company meetings. The latter would be a contingency question.
Response Rate: The number of people participating in a survey divided by the number selected in the sample, in the form of percentage. This is also called the completion rate or, in self-administered surveys, the return rate: the percentage of questionnaires that are returned.
Interview: A data-collection encounter in which one person (an interviewer) asks questions of another (a respondent). The interview may be conducted face to face or over the telephone.
Probe: A technique employed in interviewing to solicit a more complete answer to a question. It is a non-directive phrase or question used to encourage a respondent to elaborate on an answer.
Situational framing: Ask details about the time of a well-remembered event
Decomposition: Ask details about several specific periods, then add them up
Landmark anchoring: Ask details before/after some well-known benchmark
Random Digit Dialing (RDD): A sampling technique in which random numbers are selected from within the ranges of numbers assigned to all active telephones; erases the bias of using a phone book where unlisted numbers are missed.
Computer Assisted Telephone Interviewing (CATI): a technique that is used during telephone interviews; computers select random phone numbers to call, provide the interviewer with questions to ask and then are used to record data.
Secondary Analysis: A form of research where data previously collected by one researcher is reanalyzed by another researcher (often with a different purpose); especially appropriate in the case of survey data.
General Social Survey:A survey in which large online databases are used to aid the data collection process. These surveys are usually very large in size.
Screening question:First asked to determine whether to ask contingency questions

When it comes to asking questions in social research to acquire data for analysis and interpretation variables are operationalized. Surveys use questionnaires or document containing questions and other types of items designed to solicit information appropriate for analysis. There are a few different options when it comes to creating a questionnaire for a survey. When the researcher wants to figure out the magnitude to which a respondent holds a belief the researcher can produce a statement and ask the respondent whether or not they agree on it. An example of this would be the Likert scale. Using a combination of questions and statements in a questionnaire gives the researcher more flexibility in the questionnaire and can make it more interesting.

In addition to the previously mentioned items:

A census is a form of survey in which the entire target population is sampled. Censuses are usually territorial in nature. Censuses of agriculture and industry exist along with censuses of population. Early surveys were abstracts of the target population, but without scientific sampling. True scientific surveys using probability sampling techniques came to dominate after 1940. Surveys are most appropriate for gathering factual information that can be confidentially self-reported.

Tips in Surveying and Questionnaires

Respondents should be asked questions to which they should know the answers to
People generally cannot give unbiased information about others and cannot judge issues of causality (even for respondent's own behavior)
Basic foundations of survey writing are "avoid confusion" and "keep respondents' perspectives in mind"
Surveyors (verbal) should be friendly and make the participant feel comfortable without encouraging a particular response.
In specific surveys, the questions asked in a survey should be those in which the respondent would know the answers to. For example, questions about the respondent’s own behavior, attitudes, opinions, beliefs, their own demographic characteristics, or their own expectations for the future, are all appropriate. Sleeper questions are used to ensure that the respondent is accurately reporting their knowledge.
It is important for the researcher to consider how questions are phrased. This is necessary in order to avoid topics, such as drug use, being under/over reported.
In-person interviews have a high rate in overall questions being answer as well as offer clarification to questions to ensure better response rates/information. Interviewer can also monitor facial expressions and body language of the participant which can be beneficial to see if one is uncomfortable or not being completely honest.
Clearly stated questions avoids confusion, shorter surveys, and simplistic questioning are ideal for the participant to comprehend what is being asked of them and ensure a survey that can be completed in a timely fashion.
Do not ask negative questions (e.g. I am not opposed to abortion. circle yes or no) these questions can be confusing and can mislead respondents.
You should avoid using terms that can create a bias in your respondents, such as welfare, and use terms like public assistance that do not have a negative connotation.
The way in which you order your questions is also important. You should not lead your respondents to answer a certain way. For example if you have asked a few questions regarding the dangers of drug use you should not follow those questions with a question like "do you think drugs are dangerous" you should instead ask such a question in the beginning to avoid biasing the respondents answers.
If the response rates are also something all survey researchers must be concerned with. If your response rates are low your data could be bias.