How to Survey

Introduction

This is an revised version of a 2008 web page (first created in 2003 for Sun Microsystems) that brings together in one location key information and resources for how to conduct surveys. The initial audience for this information was the Sun Sigma (Six Sigma) professional community.

How do I know about surveys and data collection? I was certified as a Sun Sigma Black Belt in 2002 and served as a Master Black Belt 2002-2010. I was also in one of the last classes that Dr. Deming taught on statistical management methods, in 1993.

Key Questions

Four questions to ask yourself before starting to create a survey:

1. Why survey?

A survey is one of many good ways to collect information from customers. It may or may not be the best way for your situation. Have you considered other options such as field studies, baseline research, interviews, and focus groups? Many customer groups get surveyed over and over and get very tired of questions: what do you know about the data already collected from the target group?

2. What are the rules?

Information protection, security, and privacy are some policy and legal areas to consider before developing your survey. Local laws about sweepstakes and contests also need to be considered for some survey incentives. If you work for a company, there may be different policies for internal corporate surveys and external customer surveys.

3. What questions?

Developing survey content is as much an art as it is a science. How do you form questions so that the results can be usefully analyzed? What will you do with the answer to each question? How many questions will your target audience answer before abandoning the survey? Is the way you ask the question clear to people from other contexts and countries? An excellent survey takes time and testing to perfect. If you are new to surveying, consider asking a consultant expert for support.

4. What tool?

If you are not having someone else create your survey, there are a number of tools available to you, details are available here under Tools and Services.

Tools and Services

1. Sample Size Calculator, Creative Research Systems web site tool

“This Sample Size Calculator is presented as a public service of Creative Research Systems. You can use it to determine how many people you need to interview in order to get results that reflect the target population as precisely as needed. You can also find the level of precision you have in an existing sample.”

2. Three-way Percent Calculators

“Precision and accuracy. Please be aware that there are certain limitations to all web-based calculators. The arithmetic used can lead to errors in some calculations when the numbers get very big or very small. If your work depends on being absolutely, positively accurate to the last decimal place – use a real calculator!”

3. Web-based Surveys

Web-based surveys are a best practice. You can create a custom web based survey using HTML, PERL, and CGI scripts. For those to whom these are not easily available, survey tool companies provide an easy alternative. Before starting to use any third party survey tool (especially a “free” version), be sure to consider who owns your survey data, how the privacy of your data is protected, and whether the tool company charges for larger numbers of responses.

• Zoomerang “Create custom web-based surveys and get rapid results. Start using the #1 online survey tool today!”
• VTSurvey “A web-based tool which enables end users to autonomously create and run online surveys, feedback or registration forms.” (last update: 2005)
• SurveyMonkey “Intelligent survey software for primates of all species. SurveyMonkey has a single purpose: to enable anyone to create professional online surveys quickly and easily.”
• SurveyGizmo “SurveyGizmo is an exceptionally powerful survey tool designed to make even the most advanced survey projects fun, easy & affordable.”

Reading

• “Ask Them Yourself” – How to survey your customers on the cheap, By Ellyn Spragins, FORTUNE – Small Business – Innovation, From the Dec. 2005 Issue of FSB
• “Keep Online Surveys Short” by (former Sun Distinguished Engineer) Jakob Nielsen – Alertbox, February 2, 2004
  “To ensure high response rates and avoid misleading survey results, keep your surveys short and ensure that your questions are well written and easy to answer.”
• “Raising Your Return on Innovation Investment” By Alexander Kandybin and Martin Kihn, Booz Allen Hamilton, 2004 (free – web site registration required)
  “There is also is a flaw in the methods by which most companies go about developing new products. Focus groups and surveys elicit consumer opinions, but people can’t know what they don’t know.”
• “Listening to the Voice of the Customer” by Mark Federman, Chief Strategist, McLuhan Management Studies, McLuhan Program in Culture and Technology, University of Toronto – November 28, 2001 (9 pages, PDF format)
• “Getting the truth into workplace surveys” by Palmer Morrell-Samuels, Harvard Business Review, February 2002 – Reprint R0202K
• How to Conduct Your Own Survey by Priscilla Salant, Don A. Dillman. John Wiley & Sons (1994) ISBN: 0471012734
• An alternative to the survey:
  • “Field Studies: The Best Tool to Discover User Needs” by Jared M. Spool, Originally published: March 13, 2007
    “While techniques, such as focus groups, usability tests, and surveys, can lead to valuable insights, the most powerful tool in the toolbox is the ‘field study’. Field studies get the team immersed in the environment of their users and allow them to observe critical details for which there is no other way of discovering.”
  • “Risks of Quantitative Studies” by (former Sun Microsystems Distinguished Engineer) Jakob Nielsen, Nielsen Norman Group: Alertbox, March 1, 2004. Follow up article: Accuracy vs. Insights in Quantitative Usability, Nielsen Norman Group: Alertbox, 21 November 2011
    “Number fetishism leads usability studies astray by focusing on statistical analyses that are often false, biased, misleading, or overly narrow. Better to emphasize insights and qualitative research.”

See Katysblog 1 May 2008 blog entry How to Survey, Part 2 (Best Practices) for more.
“How to Survey” was refreshed 26 November 2014.