Tag Archives: Six Sigma

How to Survey


This is an revised version of a 2008 web page (first created in 2003 for Sun Microsystems) that brings together in one location key information and resources for how to conduct surveys. The initial audience for this information was the Sun Sigma (Six Sigma) professional community.

How do I know about surveys and data collection? I was certified as a Sun Sigma Black Belt in 2002 and served as a Master Black Belt 2002-2010. I was also in one of the last classes that Dr. Deming taught on statistical management methods, in 1993.

Key Questions

Four questions to ask yourself before starting to create a survey:

1. Why survey?

A survey is one of many good ways to collect information from customers. It may or may not be the best way for your situation. Have you considered other options such as field studies, baseline research, interviews, and focus groups? Many customer groups get surveyed over and over and get very tired of questions: what do you know about the data already collected from the target group?

2. What are the rules?

Information protection, security, and privacy are some policy and legal areas to consider before developing your survey. Local laws about sweepstakes and contests also need to be considered for some survey incentives. If you work for a company, there may be different policies for internal corporate surveys and external customer surveys.

3. What questions?

Developing survey content is as much an art as it is a science. How do you form questions so that the results can be usefully analyzed? What will you do with the answer to each question? How many questions will your target audience answer before abandoning the survey? Is the way you ask the question clear to people from other contexts and countries? An excellent survey takes time and testing to perfect. If you are new to surveying, consider asking a consultant expert for support.

4. What tool?

If you are not having someone else create your survey, there are a number of tools available to you, details are available here under Tools and Services.

Tools and Services

1. Sample Size Calculator, Creative Research Systems web site tool

    “This Sample Size Calculator is presented as a public service of Creative Research Systems. You can use it to determine how many people you need to interview in order to get results that reflect the target population as precisely as needed. You can also find the level of precision you have in an existing sample.”

2. Three-way Percent Calculators

    “Precision and accuracy. Please be aware that there are certain limitations to all web-based calculators. The arithmetic used can lead to errors in some calculations when the numbers get very big or very small. If your work depends on being absolutely, positively accurate to the last decimal place – use a real calculator!”

3. Web-based Surveys

    Web-based surveys are a best practice. You can create a custom web based survey using HTML, PERL, and CGI scripts. For those to whom these are not easily available, survey tool companies provide an easy alternative. Before starting to use any third party survey tool (especially a “free” version), be sure to consider who owns your survey data, how the privacy of your data is protected, and whether the tool company charges for larger numbers of responses.
    • Zoomerang “Create custom web-based surveys and get rapid results. Start using the #1 online survey tool today!”
    • VTSurvey “A web-based tool which enables end users to autonomously create and run online surveys, feedback or registration forms.” (last update: 2005)
    • SurveyMonkey “Intelligent survey software for primates of all species. SurveyMonkey has a single purpose: to enable anyone to create professional online surveys quickly and easily.”
    • SurveyGizmo “SurveyGizmo is an exceptionally powerful survey tool designed to make even the most advanced survey projects fun, easy & affordable.”


  • “Ask Them Yourself” – How to survey your customers on the cheap, By Ellyn Spragins, FORTUNE – Small Business – Innovation, From the Dec. 2005 Issue of FSB
  • “Keep Online Surveys Short” by (former Sun Distinguished Engineer) Jakob Nielsen – Alertbox, February 2, 2004
      “To ensure high response rates and avoid misleading survey results, keep your surveys short and ensure that your questions are well written and easy to answer.”
  • “Raising Your Return on Innovation Investment” By Alexander Kandybin and Martin Kihn, Booz Allen Hamilton, 2004 (free – web site registration required)
      “There is also is a flaw in the methods by which most companies go about developing new products. Focus groups and surveys elicit consumer opinions, but people can’t know what they don’t know.”
  • “Listening to the Voice of the Customer” by Mark Federman, Chief Strategist, McLuhan Management Studies, McLuhan Program in Culture and Technology, University of Toronto – November 28, 2001 (9 pages, PDF format)
  • “Getting the truth into workplace surveys” by Palmer Morrell-Samuels, Harvard Business Review, February 2002 – Reprint R0202K
  • How to Conduct Your Own Survey by Priscilla Salant, Don A. Dillman. John Wiley & Sons (1994) ISBN: 0471012734
  • An alternative to the survey:
    • “Field Studies: The Best Tool to Discover User Needs” by Jared M. Spool, Originally published: March 13, 2007
        “While techniques, such as focus groups, usability tests, and surveys, can lead to valuable insights, the most powerful tool in the toolbox is the ‘field study’. Field studies get the team immersed in the environment of their users and allow them to observe critical details for which there is no other way of discovering.”
    • “Risks of Quantitative Studies” by (former Sun Microsystems Distinguished Engineer) Jakob Nielsen, Nielsen Norman Group: Alertbox, March 1, 2004. Follow up article: Accuracy vs. Insights in Quantitative Usability, Nielsen Norman Group: Alertbox, 21 November 2011
        “Number fetishism leads usability studies astray by focusing on statistical analyses that are often false, biased, misleading, or overly narrow. Better to emphasize insights and qualitative research.”

See Katysblog 1 May 2008 blog entry How to Survey, Part 2 (Best Practices) for more.
“How to Survey” was refreshed 26 November 2014.

1 Comment

Filed under Mentoring & Other Business

Sun Shot – Six Sigma

I am drinking a caffe latte and checking out the conference room I will be in for the next two days. As a Sun Sigma Master Black Belt, I have been asked to help two Sun groups based in Europe, the Middle East, and Asia Pacific to sort out their roles and responsibilities. A member of a Sigma team I ran several years ago asked me to get involved even though (or maybe because) I am not in their organizational area. It is always interesting to work with groups outside of my own: we both can learn new ways of solving organizational problems.

As usual with a very short Sigma project (Sun Shot as we call it), the focus is on change acceptance, communication, growing together in understanding, and building a list of next steps and future directions rather than in-depth problem solving. The team leaders (both from Europe) and I have been on the phone several times to develop the charter, then we met in-person yesterday to work out the details. Even though the next two days will be very hard work, I am looking forward to the discussion.

Leave a comment

Filed under Mentoring & Other Business

Shito Ryu Karate Black Belt

I just finished sewing on my daughter’s gold JKO (Japan Karate-Do Organization) patch onto her gi (uniform). Jessica earned her Black Belt in Shito Ryu Karate today! She has been studying with Shihan Roberto Alvelais at Pacific Martial Arts for eleven years, since she was six. Our whole family is very proud!

Below are some photos from Jessica’s black belt test today. The 2 hour test included her demonstrating individual elements (stances, punches, blocks), entire series of elements (kata), and then sparring with another brown belt (kumite). Prior to today, Jessica had to take referee training, write an essay, and take a series of shorter tests. After the sparring, she answered questions about the history of karate and the Shito Ryu style. In some of the photos, you can see Rob and the other black belts in the Pacific Martial Arts dojo sitting in a line. Rob was the sole judge but the other black belts (some of whom come from other karate styles) came to lend Rob and Jessica their support and encouragement.

In a blog entry called “Sigma belts & Karate belts” from June 2005, I wrote about the similarities and differences between Six Sigma and karate. Jessica’s preparations and today’s big test strongly reminded me of what I went through for my own Sigma certification here at Sun.

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson
Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

After the test, Rob said a few proud words as Jessica’s teacher and presented her with the actual black belt which he had been wearing. Then, it was time for family pictures and a celebration lunch.


Rob presenting Jessica with her black belt:

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

Rob and Jessica:

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

Jessica and her Grandparents:

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

Jessica and her new gold JKO gi patch:

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

Jessica and Grandpa Wade:

Jessica's Black Belt Exam, 19 August 2006,<br /> photo: copyright 2006 John Plocher and Katy Dickinson

Images by John Plocher and Katy Dickinson (Copyright 2006)

Leave a comment

Filed under Home & Family

Sigma belts & Karate belts

Two years ago when my daughter last competed in an AAU (Amateur Athletic Union of the US) regional karate-do championship tournement in Oregon, I originally wrote this. Jessica has been taking karate from Shihan (Master Black Belt)

Rob Alvelais
since she was 6. After 10 years of greater and lesser dedication to getting promoted, Jessica now holds an advanced brown belt in her dojo (karate school). She is preparing for her black belt test this year.

At the 2003 tournement after she competed, we sat with Rob and listened to him critique both the coaches and athletes. After working as a sigma Black Belt and Master Black Belt here at Sun, it was fascinating for me to spend two days watching the sport from which this management method has taken some of its terms. I found both similarities and differences:


  • There are lots of good ways to achieve the same end. There are at least 5 official styles of karate-do and a good judge has to be able to evaluate all of them. 
  • All of the styles are based on the personal teaching of a master with profound experience and lineage to earlier masters. Sun traces our sigma style to Lowry Manson and Rick Taylor at GE, and before them, to W. Edwards Deming
  • Progression up the ranks (from the 5-year-old white belt beginners to the 76-year-old 10th Dan level Black Belt, Master Kenzo Mabunii, Soke) is explicitly earned through tests set and judged by those of higher rank. The community’s communications are very rank-aware. 
  • Judgements of success or failure are immediate and public. We have sigma tollgate reviews with our Champions. The althletes, even tiny kids, immediately after they have demonstrated their kata (form) must stand at attention and listen to the scores of 5 judges before they can bow and walk off the mat. 
  • Active participation with your peers, continuous learning, and coaching of those below you in rank is expected no matter what seniority a person holds. At the tournement, during the
    morning there were Shihans competing for medals. The same Master Black Belts could be seen acting as judges in the afternoon. 
  • The tournement took a long time and the announced schedule gave only a general guide to what might happen when.


  • In karate-do, form is more important than results. Our sigma work can be more like karate-jitsu, or street fighting, where results are what really count. 
  • The athletic goal to be achieved is difficult but limited in both scope and duration. There is an athletic standard for excellence and it can be achieved. Sigma excellence is often more subjective than standardized. 
  • A major aspect of Karate-do learning and advancement is competition rather than cooperation. (However, Jessica informs me that in Rob’s dojo, no one can achieve black belt without the cooperation of the senior members as practice partners.) 
  • Karate-do is structured to train all participants as future leaders because there is no limit the the number of black belts or master black belts. The tournement division including very youngest kids was regularly announced as “our future black belts”. Because sigma ranks are actual jobs, advancement is more complex than just passing the knowledge test. 
  • Anyone can learn karate-do: there are no minimum standards or prerequisites for a beginner other than a willingness to work hard and persist. A sigma belt candidate must have a minimum level of performance and management sponsorship. 
  • In karate-do, there is no stigma for failure: everyone gets a medal after competition and everyone seems willing to go again. Enthusiastic hugs were the usual response after the
    scores were announced between people who had just finished pounding each other during kumite (sparring) matches. Since sigma projects cost time and money, failure can be very expensive in many ways: so, the barrier to entry is higher. 
  • The purpose of karate-do is developing and perfecting a personal and physical discipline. The purpose of sigma is to solve a business problem.

Leave a comment

Filed under Mentoring & Other Business