Survey Respondents are just big fat LIARS!

fake opinionsLast time I checked my Facebook account (approximately 4 minutes ago) I discovered a SHOCKING pattern – all my friends/peers/relatives are either holidaying in Bangkok, buying a flashy sedan (read Honda City) or pasting plagiarized motivational quotes on their walls like a maniac. After 5 minutes scrolling down all the updates I just had one feeling – SADNESS – for myself! It seems that everyone is doing something really important in their lives except me.

But wait a minute – it does not seems right – I am an optimistic person but for sure the world is not such a happy place as being projected to me. Suddenly I realized they all are BS!! and at that very moment I received the message from the angels – “all respondents are big fat liars!!” who just project an acceptable opinion.

Ok I might have exaggerated a bit on the angels part but I am sure you will agree with me here that survey responses are mostly farce.

Why is it that I don’t receive Facebook messages like “Oh God my Boss again insulted me in front of the whole staff” or “Shit my wife is cheating on me”. The answer is simple – ‘we humans tend to be coherent with the acceptable norms of the society’ – any action that results in being ostracized or ridiculed will be best avoided.

There are times when I just want to stand up and scream at the top of my voice that Pepsi is just a stupid sweet carbonated drink which only a lunatic will pay for to get diabetes in the long run – but no I will never express that so explicitly – cause maybe if so many people love Pepsi – they might be right! or I am just trying to save myself from the wrath of the Pepsi drinkers.

No matter how good the survey design is and how experienced the interviewer – what if I consciously try to project an image of an ideal person. Don’t we do the same in Facebook – we will just remain quiet – lurking in the shadows – till that elusive Bangkok trip materializes and then suddenly we all are hyper active in pasting photos on Facebook. Can I trust a person who has such a low self esteem that he will only open mouth when he is able to share something supposedly positive about himself?

The problem is not just specific to the expression of correct opinions – I am sure there are genuine respondents who provide their opinions but do you REALLY believe that every respondent’s response is just motivated to help brands improvise themselves? In today’s world how many of us actually intend to make things better without personal materialistic motives? I will say very few like Mother Teresa !

Let me explain it a bit more here – every respondent knows that there are screener questions and on the event of not fitting the profile they will be terminated out of the survey – therefore in a way researchers are provoking indirectly the respondents to avoid answers which may terminate them and devoid them of the incentives/gifts. Here a respondent will deliberately try to seek out answers which he perceives to fit the profile that researchers are looking for and not select those responses which are actually true.

So right now we have come across 2 problems – first respondents are scared of sharing opinions which they presume may create a negative impression about them? and secondly when the primary motivation is money I am not sure how authentic the responses are.

Now I am sure all market researchers will cry foul and obviously they should – after all I am raising uncomfortable questions.

But my intention is really simple – if there is an issue we better address it like a true professional.

I am not very confident to take business decisions  based on opinions of people who blatantly lie on public platforms like Facebook or who are motivated by the greed for incentives.

PS: If someone says that their respondents are only motivated by an inherent desire to help brands – trust me they are the same people who believe there is a Loch Ness monster!

Akshay Kanyal

Akshay Kanyal

Akshay Kanyal writes survey research reviews on his popular blog Online MR. He’s an avid blogger, brand consultant and a content marketing expert, helping business owners to craft content that sells.

He provides content marketing advice to start-ups and innovation driven companies. He can be contacted at editor@onlinemr.com

You can also connect with him on LinkedIn ; Facebook ; Twitter

Comments

  1. I agree people know about screeners and, even if they do not want to do our boring surveys, people hate being rejected, so they “bend” the truth and then quit on the survey. Wouldn’t it be great if panel companies allowed you to terminate ex post facto.

  2. Marko Vicković says:

    I agree mostly to what you said, Akshay. People do tend to answer in line with social desirability and sometimes, yes, they tend to cheat on screening part just to get incentives. So, those are problems. Still, you can do something:
    1 – try with “classic” market research methods. If possible, forget on Facebook and online and do face-to-face interviews (at homes or in malls) or phone interviews, without incentives. It is much difficult, much expensive, but the quality of data might be better, for sure you might find people who will say “that brand is crap”.
    2- try to use benchmarks and compare the findings with “average” results. So, forget the general picture (yes, it is more idealistic than it should be), try to find if your recent research is among top5 or top20 researches regarding possitive attitudes expressed toward the brand. If so, the brand (or product, concept, package, idea or whatever you examined) might be of good potential.
    3 – try to use tricky questions and test social desirability. Don’t ask “do you like the brand” or similar, try to measure how much effort respondent is going to invest to show you they really prefer the brand. Like, “would you like to make a short essay on your favorite CSD brand (just 200 words) that the brand might use as its inspiration for ad campaigns,…” (of course, for free). Or ask “tell me the three things you do not like regarding your favourite brand of… Please, describe it in detail”. Force them to work and measure how much (or how many of respondents) worked and use benchmarks and you will get clearer picture of what is really good brand.
    4 – if you are doing with panels try to use incentives that could not be considered as a salary for job (like 50 € for physicians for 30min. survey), incentives should be just a sign of attention, like the researcher do care for respondents.
    Hope it might help. Cheers.

  3. I am not sure if online surveys do work in India where people generally don’t want to respond for survey mails as they think it as a waste of their time. While conducting F2F interviews, you can pick genuine respondents and ask socially desirable responses in derived way. However face 2 face interviews are also prone to interviewer bias as they make a non eligible respondent a genuine respondent (specially when TG is hard to get).

  4. Akshay, is this your true opinion or have you written something to be provocative and newsworthy?

    The Advertising Research Foundation is currently in the process of determining some best practices to insure quality of online research, FOQ2. Please go to the ARF website, read about the work and see if you want to join the initiative. You’ll be in good company, with experts from over 30 different global companies. Results will be reported at Rethink: 2013 this March.

    As a research practitioner for many years I have a good handle on what works and doesn’t when it comes to uncovering the truth from respondents. However, FOQ2 is taking a scientific approach to establish objectively what it takes to insure the highest standards.

  5. Chris Platt says:

    There are lots of differerences between facebook and market research, the major difference is that research is anonymous and facebook is not. If you have a critical opinion in research, you don’t run the risk of being criticized by others. That difference alone essentially ruins your entire analogy.

    By the way, although people do like to gloat about positive things in their lives on facebook, many others are critical (e.g., bad customer service experiences, politics, etc.).

    • That’s the main point really, I agree with Chris. Your whole part about Facebook is just wrong, this is something completely different than market research.

      Then, regarding screener questions. I do agree that some market research compagnies have weak screeners and it is easy to guess the required target and fall into that to get the rewards but this is surely not good for their clients and at some point, the end clients will figure it out. So again, this is their business if they do it wrong, they will lose clients.

  6. What answers qualify or disqualify one from a screener should never be obvious. For that reason, we rarely tell people what the survey is about.

    If we invited people to a “dog owner study”, and the first question was “do you have a dog”, then the ease of ‘gaming the system’ is too tempting. However, if you invite someboyd to a “survey”, and the first question was “do you have a dog”, the person has a 50/50% chance of guessing which is/is not the qualifying answer. For all they know, we are after people who DO NOT have dogs.

  7. Well, there may be people on earth you believe that tattoos are necessary for good health. They could be the same people that believe what they read, what they hear and what they are told.

    Research was invented to provide an element of objective evidence of truth. There are a number of ways of going about it.

    One of them is to validate the statements/evidence. The validation should satisfy objection criterion for both internal validity (Do they earn enough to be overseas all the time) and external validity (Where are the pics & movies of them in Bali).

    Another is to evaluate evidence over time, ie if they say the same things (or lie consistently). Human being tend to forget their lies, or put another way, it’s easier to tell the truth. So, if one says the same thing repeatedly, it does seem to become more credible – a fact and politicians and advertisers depend on. But, without both internal and external validation, any statement, especially if it only benefits the projector of the statement, is waffle.

    Anybody who believes any social media information needs to grow up – seems like the light has come on in your world.

    Just remember – research is very different from web scapings.