Yesterday I wrote about guidelines for proper survey design. Today’s post is the first in a two-part series about how to maximize your returns on that well-designed survey. I’ve written on this topic before, namely for an article for CustomerSat’s blog several years ago. I also gave a presentation on maximizing response rates at last year’s Allegiance Engage Summit. For those of you who missed all that, here it is in black and white.
First, let me clarify two terms: response rate and response volume. They are certainly related, but statistical validity is based on the number of responses (volume) and not the rate. Clients always ask me what the best response rates are, but consider this: when you have a 50% response rate on a sample of 100 versus a sample of 10,000, it’s going to mean two different things for your confidence in those findings.
So, assuming you have a solid population size and sampling method, I feel comfortable using the term “rate” going forward. Good response rates are essential for accurate, useful results. Low response rates and insufficient sample sizes:
- Erode the validity of your results
- Force you to qualify your reports and conclusions
- Lower confidence in your findings and recommendations
- The response rate is the percentage of people who respond to your survey, whether they submit just one page of the survey or all pages.
- To calculate: # responses / # invited*
- The completion rate is the percentage who complete the entire survey—that is, they answer all questions relevant to them and submit the last page of the survey. Completers are a subset of responders, since not all respondents will complete the entire survey.
- To calculate: # completed responses / # invited*
* # Invited will likely exclude bounces, wrong numbers, etc. (depending on your data collection methodology). You can certainly calculate the rates both ways, with those excluded or included; either way, I’d footnote your approach.
The reasons for differentiating response rates and completions rates are varied; and on the heels of my post about survey design, I’ll focus on the impact survey design and respondent engagement (your relationship with the customer combined with how you’ve chosen to survey the customer) have on achieving your response goals. The graphic below shows their relationship. I think it’s self-explanatory and points out that if you focus on good survey design combined with good respondent engagement, your response and completion rates will be high.
- the investment your company has made to support or deny its hypotheses
- presenting your findings to executives and to various other stakeholders
- making recommendations and basing strategic initiatives and direction on your findings
So, the obvious next question is, “What do I need to do to ensure that I get the maximum response rates for my survey?” Good question. Stay tuned for tomorrow’s post when I’ll outline 10 tips to help you achieve your goal.
I love the quad chart. This quickly sums up the desired outcome vs. potential pitfalls.
One may think that designing a questionnaire is easy. While some of us are able to "nail it" on the first draft, others struggle with wording, question order and other aspects of questionnaire design. See more questionnaire design