4 Tips for More Effective Web Surveys

Conducting a web survey is not always as simple as it sounds. Sure, online survey services like SurveyMonkey have made it easy for folks to create their own surveys and send them out to their mailing lists, and even to the general public. But what good is a survey if it’s not asking the right questions? Or if the questions are confusing to respondents? Or if there is something in the survey that makes respondents click away from the survey before completing it? All of these issues can affect the quality of your survey and, in turn, the lessons you are able to take from your data.

Building on the things I mentioned in the video "Top 3 Tips for an Effective Web Survey,” here are a few suggestions to help you develop a web survey that will yield accurate, actionable results.

1. Ask the right questions.

It’s tempting to put together a scattershot survey — coming up with every question you can and tossing all of them onto a survey instrument. But that is never the best way to go. Respondents want to know that there is a purpose to the survey, and they will not hesitate to leave an unfinished survey if they become bored or frustrated with it. One of the main causes of incomplete surveys is respondent fatigue — the shorter the survey, the better.

To keep a survey as focused and brief as possible, be sure to spend time at the outset thinking about what your objectives for conducting a survey are, before even drafting any survey questions. Only ask questions that can help you reach those objectives.

Next, decide how to operationalize each objective. What does each objective mean to you, and what will it mean to your audience? Identify the elements that go into each objective, and turn those into survey questions. If, during this process, you learn that you have quite a few objectives, and each one has several elements, you should decide if individual questions can be used to address multiple objectives, or if you should consider conducting separate surveys.

2. Pay careful attention to question and response wording.

Crafting good questions is equally important to collecting high quality data. Every word can be important to letting your respondents know what information you’re asking them to provide. Here are a couple examples of common survey errors.

Suppose someone sends me a web survey and I come across the following question:

How much do you like chocolate and cilantro?

• A great deal
• Some
• A little
• Not at all

Well, I LOVE chocolate, but I don’t care for cilantro at all — so how can I answer that question? That question is what survey professionals call “double-barreled”; it is really asking about two separate things, but it only provides one response range.

Ok, most people would not include chocolate and cilantro in the same question, but here’s one that I’ve seen before, and maybe you have, too:

How would you rate our website's design and features?

• Excellent
• Good
• Fair
• Poor

You might have high praise for the website design but feel just ok about the features. It is impossible for you to answer that question in a way that will provide meaningful data — you’ll either underrate the design, or overrate the features.

Other questions might be worded very clearly and concisely, but the response ranges are not. As an example, look at the following question:

How would you describe your mood today?

• Happy
• Sad
• Angry
• Content

What if you are feeling both sad AND angry? Or if you are feeling worried or excited? And what exactly is the difference between “happy” and “content”? Response ranges require a great deal of thought to make sure the choices are mutually exclusive, the list is exhaustive, and the options are clear. If they are not, the survey will return meaningless data.

3. Think strategically about the survey’s structure.

The order in which questions appear on the survey can be very important to respondents. Have you ever taken a survey where the first question was something personal, such as your annual income or how many children you have? Did you complete the survey? Probably not.

Asking these types of personal questions off the bat can give respondents pause, and sometimes make them leave the survey altogether. If you need to ask those questions, save them for the end, once the purpose of your survey has been made clear and the respondent trusts how you’re using their information.

Instead, start your survey with some interesting questions that will grab respondents’ attention and make them want to go further in your survey. Ask general questions that will encourage someone to think, “That’s a good question, and I’ve wanted to share my opinion on that with someone.”

4. Consider your methodology and, if possible, consult an expert.

It’s not easy to get people to complete surveys of any kind today. Phone surveys used to be the most common type of survey, but today less than 10% of people who are contacted by phone complete them. Web surveys are increasing in popularity, and people seem to be more trusting of them, but response rates just aren’t what they used to be. There are so many scams out there, that people are (rightly!) wary of every email they receive and every link they click.

If you’re thinking about conducting a web survey, do everything you can to encourage your audience to complete the survey, including making the experience easy on them by asking clear questions in a way that makes sense. A lot goes into creating a successful web survey — there have been countless journal articles published about survey research, there are entire professional organizations dedicated to the practice, and they even have doctoral degrees available in it.

We at SHERPA have extensive experience in developing, fielding, and analyzing successful surveys that have helped clients hone their existing strategies and discover new ones, and we would be happy to help you with your next web survey.