Surveys are and should be a powerful way of getting feedback. Properly used, they can provide great insight on where you are doing well and where you can improve. Too often, however, surveys are being abused. They may be thinly disguised marketing efforts, abusive self promotion, or a terrible substitute for “customer service.” Then there are those surveys that only allow you to give them the input they want to hear, “You MUST Be Ecstatic With Us!”
I’m genuinely interested in the power of surveys for insight and improvement, so I tend to respond to every survey I get–both to provide feedback that I hope helps improve my experience with a particular company, and to see how organizations are using surveys. While not scientific, my estimate is 90% of the surveys I get have nothing to do with getting honest feedback or trying to improve the customer experience.
Today, I got one from Zinio. Zinio was one of the original companies providing online magazines. To be honest, it Sucks! Based on alternatives that are available today, it’s probably one of the worst reading experiences I’ve ever seen. However, some of my favorite magazines are only available through Zinio, so I’m forced to use it.
This morning the email comes:
Hello! We’d appreciate it if you could help us learn more about how people like you are enjoying digital magazines.
Please take a few minutes to fill out this survey (10 minutes tops, we promise!). As a thank you, once you complete the survey, you can choose one free year of Star and OK! magazines, or a $5 Zinio credit (good towards any future subscription).
We appreciate your feedback. Just click here (link removed, so you don’t all pile on as me) to get started.
I thought, “Well good, maybe they might be able to improve the user experience and make me happier. At least I owe them my feedback.” So I started to respond to the survey. The first couple of questions were innocuous question, just some background, then I get to the questions about what I think of their products. I’m eager to respond.
The first question, is “From your experience, what are the top 3 needs Zinio meets better than any other provider.” I scan the list, looking for something to the effect of “The other providers are far better than Zinio in meeting all my needs.” I can’t find it, everything focuses on the superiority of Zinio. But I see “other,” so I decide, this must be where I provide my input.
Then next is “Which of the following taglines sums up what Zinio provides to you? Choose the three you like the best.” Again, I try to find something to the effect of “Zinio provides the most unfriendly user experience I’ve ever had,” or “Other providers have a far more engaging format and user interface.” Again, I am defeated because the only answers I’m allowed to give are different gradations about how wonderful Zinio is. So I go to my trustworthy “other” and put my response.
I complete the first page, hit next, and am met with error messages. See it turns out, I am only allowed to provide responses about how wonderful the service is, how great the experience is, and how superior they are. I am not allowed to provide any other input–which might be useful– or that reflects my honest experience.
I don’t know what the rest of the survey asks, unless I lie and tell them they are wonderful, I can’t go further in the survey, so I’m stuck on this page. So it would appear the only people that can complete the survey are those who are very happy with Zinio. And we all know how this data will be used. Internally, the reports will say, “We are so far superior to everyone else, we don’t need to improve,” when they are trailing edge (at least in my opinion) and will lose share and customers. They will also use this for marketing purposes, “Our customers tell us we are, by far, the best provider of online magazine content around,” when in truth, I and most of my colleagues do everything we can to choose other alternatives. Where they could have learned something, they apparently have no desire to learn anything, but want to engage in a deception about how wonderful they are.
While I’ve singled Zinio out, it’s only because this is the survey that came into my email today. Every day, I get surveys with similar designs. It’s nothing but pure deception. It misleads the company, giving them potentially inaccurate feedback about what customers think about their experience, causing them to be blind to their weaknesses. It’s deceptive to use in marketing, because if you only allow positive answers, then you are misleading the markets.
Surveys can be very powerful, but you have to be prepared to listen and hear what your customers are telling you. Furthermore, you have to be prepared to take action on the responses you get. This doesn’t mean agreeing with the responses, but this does mean letting people know what decisions you have made and actions you will be taking. Sometimes the results of surveys may point to things you can’t or choose not to do. That’s fine, just let people know and why.
If you are only interested in hearing what you want to hear, then don’t waste your time, or your customers’. It’s dishonest and pointless. Or even worse, you get people like me blogging about it, sharing your bad practice with the world.
Coincidentally, I just discovered my friend Tibor Shanto had a similar experience with another company. Make sure you read his post!
A couple of things Dave. First off, great post. Always important for stories like this to get out in order for those of us in the survey business to hear what the real customer experience is like. I wish I could say that your experience was isolated, but its not. The research industry has been thrown upside down by technology. Companies can now create and deploy surveys, target customers and prospects for extremely low costs, and have access to tools that can analyze the results. Unfortunately, having the tools doesn’t mean you know how to ask a good question (or series of questions) that gets an answer that is meaningful to your business. Corporate customers are abandoning full service research firms for the DIY route – and the result is big savings – and crappy, customer annoying, too frequent surveys combined with useless data. Whose to blame? The full service research firms didn’t see the technology changing fast enough and aligned their business with data collection instead of data analysis and consulting. Now, when clients complain about costs, they cave. When clients push back on survey design, research firms don’t stand up for themselves and we end up with 45 minute surveys. Unless the full service firms reinvent themselves and embrace the ability to collect and analyze data more inexpensively – look for more of these useless surveys in your inbox.
Keith, it’s great to see you back! I couldn’t agree with you more. The “easy” availability of technology has created the ability for organizations to both harass their customers with survey after poorly designed survey, as well as mislead themselves and customers with results that are pure garbage.
The technology will continue to get easier for people to abuse. Other quasi survey mechanisms including “likes,” “followers,” even Klout will create more misleading information, creating an ever escalating cycle of volume, garbage, and ultimately meaningless data which makes the consumer (business or individual), increasingly skeptical.
It’s an opportunity for the research companies to take real leadership in helping organizations understand this and develop and implement more impactful means of assessing the customer experience. I know you are in the forefront of that work! Thanks again for contributing. Regards, Dave
It’s a shame really. It would seem to me that you can learn so much more from what customers honestly don’t like. However, I think this is something we see all too often with companies that don’t really want feedback.
Considering all the implements a clever web business might use to better understand their consumers, I think surveys are the most obsolete. Only 10% of any sample will ever fill out a survey, an early bias, and among them any number of your fav customers, haters, and in no way a broad sampling of your users and their sophisticated needs.
All the more embarrassing in this survey is the idea of polling taglines…silly, thoughtless ones.
Thanks for the comment Joe. I think survey technology has it’s place—but as you mention it is badly abused, to the point of annoying customers and presenting misleading results. Good design can minimize the impact of survey bias (or at least acknowledge it) and overcome a lot of other challenges. The problem is 99.9% have just bad designs. Research firmts, as Keith points out, can do a lot to improve the design and accuracy/relevance of the results produced.