It’s not just Survey Monkey, it’s all of those web-based survey tools. They make it so easy to gather misinformation.
There’s an old saying that goes like this –
There are four types of knowledge:
- what we know
- what we don’t know
- what we know we don’t know
- what we think we know, but which is actually false
Of these, the fourth is the most dangerous.
An easily created and implemented survey looks like a great tool to find out the answers to critical business questions. Put up a survey tool and let your customers and prospects guide your strategy. Anyone can do it. However, the results often fall into the fourth category of knowledge. Here’s why.
The Problem with Self-selection or Opt In
Putting a survey tool on your website allows your customers to opt in. This means that you simply will not get data from people who you haven’t already connected with. It also means that you will get a larger share of responses from people who feel very strongly about the topic of the survey. These kinds of surveys tend to reinforce what you already believe and provide you no new information on the wants and needs of potential new customers.
The Danger of too Small a Sample Size
Isn’t it annoying when the parent of a first child, who happens to be quiet and well-behaved, proclaims that they know everything about parenting and that is why their child behaves so well? Isn’t it a little bit gratifying when their second child turns out to be a terror? Maybe the sample size of one child out of all the children in the whole wide world is too small of a sample size to tell anyone anything effective about raising children.
If your potential pool is all of the consumers of X product or service in all of X geographical region, maybe keeping your questions to a small pool of people who already are connected to your website or Facebook page is not the right approach. It can give you not just poor information, but information that is actually wrong.
Survey Monkey has its uses. It can be used as a tool to further engage your current customer base. But, if you are looking for a research tool to find out ways to increase sales, improve your margins or gain market share Survey Monkey ain’t it.
Beth Plutchak Consulting 
Are these problems with Survey Monkey per sec, or your choice of when to implement it?
Every time I go to the oracle site to check for updates I get the stupid Survey monkey. Since by default, Java only checks for updates once a month, it is possible to use a machines many days with zero day java vulnerabilities. I have disabled the dangerous java plugin on my browser after several attempts at black hole exploits that failed because I had manually updated java. I no longer have to manually update flash, but the folks at Java do not seem to have the ability to code their product to check for and install updates daily so that their product does not get out of date. Not only is this a problem, Oracle, rather than spending money on in house engineers to rectify this serious problem, uses survey monkey to either market, or see how customers use their product. It is ridiculous that the company who makes a product fails to understand how it is used. Money is wasted on these stupid surveys that are aimed at marketing new products when there is much work to be done on the java update process. I cannot believe that java cannot solve this problem when Opera, Chrome, Firefox, and flash have figured it out. Java is the program on my computer that is least secure, makes it more likely that hackers could gain complete control, and is the most difficult to check for the proper version and update. If all the money spent on stupid surveys went to solve this update problem with java, Oracle would benefit. Oracle needs to develop a “Manhattan project” of sorts and make solving this problem job number one. Until they do, I cannot trust that Oracle has security in mind and that security is job one to Oracle.
You said, keeping your questions to a small pool of people who already are connected to your website or Facebook page is not the right approach.
My question is if you are trying to gain market share or just improve your customer service and Monkey Survey is not the way to go about it, then what is your suggestion on what is the right way about web-based survey tools?
Thank you Teresa
Teresa – I don’t think online survey tools are effective for either building market share or gaining reliable insights into how your customers might buy from you. I have seen online survey tools used effectively, in one example, by a group that puts on a conference and was soliciting feedback about sessions. This replaced handing out paper feedback forms after each session. It worked well in this instance because it was A very tech savvy group.
The problem with a small sample set is, you don’t build market share by only talking to the people you already know. Hope that helps.
Dear Beth Plutchak,
I think this post is very insightful and I agree with you 100%. It’s so true that online survey tools are not quite effective for either building market share or gaining reliable insights of customers as they are claimed to be. For one, how do they make sure those survey takers are quality respondents? But I don’t think many people know this, apart from being too eager to jump onto bandwagons. Thank you for the post.