How to Prevent Response Bias when Conducting Surveys

The adage, “good data in, good data out,” cannot be truer in today’s digital business landscape. Furthermore, if knowledge is power, then to quote Alexander Pope, “A little learning is a dangerous thing.”

Okay, enough with the quotes and clichés. But that’s the problem with surveying people: they’re imperfect and don’t always answer honestly. Human beings are known to fall into patterns that are not truthful expressions of their inner thoughts. This incomplete or misleading information is the “little learning” that can fool you into thinking you’ve got enough data to make bullet-proof business plans.

Is there a way to address this inherent bias when surveying individuals so that the data collected is uncontaminated by prejudice? The first step is identifying the cause of the problem: response bias.

What Is Response Bias?

The human prejudice that can compromise data during a survey is called response bias or survey bias. It speaks to the tendency of people to answer questions in a not wholly truthful way when they are administered a survey.

There are any number of reasons for this disconnect, from the person answering the question feeling as if there’s pressure to respond in a socially acceptable way, to the format of the questions being leading—creating an unwanted impact on how that person responds.

Response bias can ruin data and steer companies in the wrong direction. Huge business decisions and strategic plans are regularly influenced by market and internal surveys. Companies often survey customers, but they also survey employees to gather ideas for better business practices or to gauge their job satisfaction.

How Does Response Bias Happen?

The main cause of response bias is simply that people are more interested in portraying themselves in the best possible light, instead of revealing what they really think, especially if those thoughts are less defensible. It’s not uncommon. Psychology has experienced a great deal of response bias when it comes to patients describing their personal traits, their attitudes on race or sex and their behaviors pertaining to alcohol and drug use.

The truth is, the image we hold of ourselves is often in conflict with who we actually are. This is especially true when it comes to the identity we like to portray to others around us, such as when we respond to surveys. There is any number of psychological reasons for this behavior, and as interesting as they may be, they are off-topic.

Any organization that is conducting a survey of its customers is less concerned with the causes of dishonesty than with gathering the most accurately focused snapshot of those people. So, the bottom line is that you want to get the truth from your customers.

How Questions Can Mislead Answers

The questions in any survey are almost always a hook that will capture the sort of data you want. However, the best questions lead to the truth—not the conclusion the person asking the question expects. When creating a survey, it’s best to mimic the scientific method, which uses experimentation to reach conclusions. Bad surveys start with a preferred conclusion and work backward to support it, which leads to bad results and bad business.

Watch Out for Wording

Misleading questions are going to get responses that support bias. The wording of a sentence can point a respondent to an answer. A simple example of this would be a multiple-choice question in which most of the answers are positive, with only one being negative. The person is less likely to choose that one negative answer.

Who’s Taking the Survey?

Then there is the very act of the survey might lead to response bias. A study by Health Services Research shows that respondents who are more satisfied tend to respond to surveys. While it might seem as if those who have complaints would be more vocal, this doesn’t pan out with the research about those who are willing to take the time to fill out a survey. Therefore, the results of the survey, even if accurate, are misleading due to a biased survey pool.

Using dedicated survey software to conduct your surveys can make it simple for your intended audience to easily participate.

Unqualified Responses

If the questions deal with content that is unfamiliar to the respondent, then the results of those questions will be compromised. Without accurate background or knowledge, answers are not going to provide any useful data.

Then the physical condition of the respondent must be considered. If they are sick or tired, then they’re not going to give the survey their fullest attention, and their responses might suffer.

When the survey is asking the respondent to speak about events that have occurred a long time ago, there are many problems. They might not remember, or their memories are going to be impacted by other factors. Memory is not like a film strip that can be rewound and played back with accuracy.

There are ways to deal with these issues. The simplest solution would be to offer the person answering the questions the option of responding that they don’t know, are undecided or unsure.

Types of Response Bias

To fully address the issue of response bias takes an understanding of how it can impact surveys. By being able to see them before submitting the survey to your target audience, you can make sure that the data you receive will be more truthful. Here are some types of response bias to be aware of.

Demand Characteristics: This is when the respondent is influenced by being administered the survey. They will change their behavior and opinions because they want to answer in ways that support what they think are the aims of the survey. The setting or interviewer might have an undue influence on the respondent, or the questions might be biased. It’s possible that the respondent even has prior knowledge of the survey.
Social Desirability Bias: As noted, this is when there is a bias because the questions are of a sensitive nature, usually touching on personal subjects like sexual behavior, drinking, etc. The image people want to portray doesn’t always align with their actions behind closed doors.
Extreme Responses: These can be either extremely positive or extremely negative responses, which will poorly impact your overall data. You find these problems occurring when the questions are given with answers that have a scale of responses, say from one to five, one being very satisfied and five being very unsatisfied.
Neutral Responding: As opposed to the extremes of the previous type of response bias, there is the neutral respondent, who is always tagging their answer to the middle of the road. This is often because the respondent is uninterested or not engaged with the survey for some reason. This type of response is also a problem in that it skews data.
Acquiescence Bias: This is the type of response bias where the respondent agrees with all the questions, sometimes even if those questions contradict each other. If this happens, you might as well disregard their entire survey, as the data gathered from it will be useless. Whether the respondent is trying to please or they’re so uncritical as to see themselves in the question, the data collected is not helpful.
Dissent Bias: This is the opposite of acquiescence bias in that the answers always disagree with every question. These results are also equally problematic for gathering accurate and useful data results from the survey.

How to Avoid Response Bias

Knowing what response bias is and how it shows up in surveys is a good start to removing these inaccuracies from your resulting survey data. But it’s only a start; there are other things to take into consideration too.

Consider the Demographic

For starters, think about the demographic you’re surveying. Know who they are, what they have in common if they are going to be interested in responding to the survey and why.

Proofread Carefully

Also, don’t forget to edit your questions so that they are not exhibiting any wording bias as illustrated earlier in the post. You’ll also want to diversify your questions. That is getting deeper into their answers, starting more generally and moving into greater detail.

Provide an Escape Route

Don’t forget to give your respondents an out. Allowing the participants to say “no” or “undecided” or “I don’t know” gives them a more honest response than trying to fit their answer into something that doesn’t sit right. And those “don’t know’s” are a dataset that can prove helpful.

Administer Carefully

Then there is the administration of the survey, which is also important for gathering accurate detail. Participants can be influenced by those doing the research, so researchers must always be neutral and maintain survey integrity.

Keep it Clean

When giving the survey, avoid hot-button terms, such as ones dealing with religion and politics. If they are used in a general sense, then the responses will be triggered by prejudices. Negative words can also unduly influence answers. The best practice is to be as transparent as possible.

Surveys are data intensive. There’s a lot of information to track, from participants and questions to results. Once you have your survey massaged to reflect the data you want to collect, then you’ll need a robust tool that can manage and report on it. ProjectManageris a cloud-based project management software with online Gantt charts to schedule surveys and a real-time dashboard to track and report on them. See how it can help you get more accurate results today by taking this free 30-day trial.