Quant 2 - putting a questionnaire together

Okay, time to dive back into to quant research. Lee has added some useful builds on the first post, so take a look.

This time we're looking at how to design a questionnaire, specifically one that is about FINDING out rather than testing work. We mentioned before that this usually happens after you're pretty sure you know what you're going to find out, thanks to working very hard on some qualitative work before. It may be a first dip into what's out there. In any case, quant is mainly about proof.

What follows is a basic guide to some do's and don'ts for putting a questionnaire together. If you're a planner, I think knowing the specifics of what makes a good questionnaire will help you brief one in better, and give better feedback on the drafts you get back. Being able to be critical about methodology allows you to judge the validity of a research project the client's doing without you're input. And finally, there will be times when you're pitching and very much up against it. You might want to do some quick numbers research internally - which means writing your own.

Questionnaires are a big part of that because they enable you to be more analytical (enables you to express answers as number), more practical (collate lots of respondent answers), consistent (everyone asked the same questions) and objective (interviewing done by non-researchers).

We mentioned before that quant is the bit that REALLY speaks the client's language. So make sure it does what they want it to do. For the questionnaire, that means:

• Prove your answers to their brief.
• Have new ideas they can use in the future.
• Meet approval from their clients (the board).
• Fit their own style and standards.
• Prove something they (and you) suspect.

But they're not the only ones the questionnaire needs to accommodate. Some poor bugger has to analyse these things, so lay it out and code it as well as you can.

The interviewer needs it to be easy to follow, written in conversational tone, short, with no difficult questions and avoid embarrassing or intimate ones.

And the respondent wants it to be easy to follow, they need to see WHY a question has been asked. And for them most of all, make it short easy to follow and never difficult or embarrassing.

Thinking about respondents, they're the only inexperienced people involved, so take care of them. They want to be asked their opinion and feel they're making a contribution. They want to be able to choose and answer that fits them. They want to be treated with respect and they DO NOT WANT TO BE BORED OR CONFUSED. And it may be obvious, but they want an incentive.

So avoid:

• Asking them things they don't care about.
• Little eye contact.
• Machine gun delivery.
• Strange language (beverage?)
• Impossible feats of memory (how many books did you read this year?)
• Making them feel stupid.

The main questions areas are behavior (I wear Prada), attitude (Prada is stylish) and value (I want to be stylish even if if I go overdrawn).

Behaviour is largely who? When? How often? How much? How many?  They need to understand the question, so leave out marketing jargon, and technical stuff. No one want to tell you how often they notice the viscosity of caramel. use THEIR language, like, "When did you last decorate your living room?"

Avoid ambiguous words, like dinner (lunch? Supper? tea?) and split things into the most basic elements you can, like:

Instead of, 'If you were going to buy a new car tomorrow, which one of these would be the purchase price?' use 'Imagine you were thinking about buying a brand new car, roughly how much do you think you'd have to spend?'

or

Instead of asking two questions is one' Do you enjoy watching a playing sport?' Ask separately - 'Do you enjoy watching sport?' 'Do you enjoy playing sport?'.

You need encourage them to answer truthfully - so avoid implying standards, or causing guilt. Gently lead people into a range of answers rather then coming head on with 'How often do you brush your teeth?'. If you use self completion, and they know WHY you're asking it you can get away with pretty much anything...'We all know we're supposed to clean our teeth twice a day but it's often hard to find the time..'.

leave sensitive questions until you've built up a rapport. Use projection is difficult areas..'Why do you think people pick their noses?'

Attitude questions allow objective measurement of how people feel about stuff? Those previous issues about embarrassment and ambiguity still apply. The main methods tend to be: open ended questions, alternative choices, rating scales or choosing adjectives to describe.

Open ended questions (why do you say that?) are easy to ask but difficult to analyse. There will be inconsistencies in how different interviewers probe - and respondents verbal dexterity.

Choices between alternative statements  - like 'Are you for or against a smoking ban?' can be misleading due to oversimplification and not being able to choose.

Verbal rating scales are the most widely used - Agree a lot, agree a little, neither agree or disagree, disagree a lot, disagree a little.  They are clear (I think so) but you MUST have a mixture of positive and negative statements to agree or disagree with.

Numerical scales - 'How relevant is this boring post?' 1. Very relevant. 2. Fairly relevant etc. They work well too. Same rule as above.

Getting them to choose adjectives when it gets to fuzzy problems they maybe don't know how to answer. Here are a few words to describe the 'Northern Planner' blog' which, if any of them would you use to describe it?'...appealing, distinctive, informative, boring, cheeky, irritating, dull, predictable, sloppy, smug, egotistical, opinionated, confusing (I dare you to answer). I find these the most useful for quantifying reactions to creative, and establishing attitudes to a category, or a brand personality.

'Now I'd like you to do something a bit different that requires a bit of imagination.I'd like you to imagine Nike coming to life as a person. Which of these words and phrases would you use to describe them? Choose as many or few as you like.'

Which of these brands would you say is (read out statements)

                                    Lion Bar    Twix    Crunchie

Chewy

Healthy

Popular nowadays

Doing brand image by association like above is great for market mapping.

Okay, here's some final things to think about, for putting the questionnaire together...

1. Tailor it to the interview situation. The telephone will have no stimulus material, no show cards, needs simple questions and needs to be short. It also needs a really good introduction. So although phone is always cheaper, you need to be aware of its limitations. Hall tests - shopping centres etc - give you more freedom to use stimulus, even video. HOWEVER, they can be a bit public so you need to work hard to overcome self - consciousness. Online allows very low cost AND stimulus, and it takes away the subjectivity of the interviewer. But bear in mind that people get bored, and they cannot ask for clarification. It needs speed, clarity and as much visual stimulus as it can take.
2. In each case, you need to build a relationship with the respondent - let the structure do this for you - not least if you're going to be asking prying or odd questions.
3. The introduction needs a full and frank disclosure to put them at ease. Keep is short while allowing for rapport. Look for hooks - 'we're very interested in the opinions of dog owners like yourself'. Let them know how long it will take.
4. Design the structure for variety and interest. Easy questions at first, complex in the middle, taboo at the end, move form general to specific, logical sequence, introduce fun questions, leave a pleasant taste.
5. Get the flow right - avoid repeating questions, try using LINKING statements between sections, use as many involving activities as you can, vary the type of question used.
6. Finally, try not to ask them to contradict themselves, do behaviour before attitude, do brand questions before advertising stimulus. And remember you want people to think through  the issues - current opinion first, considered response to 'new' after.

So that's it. I'm sure it's too long, and a bit prescriptive, but sometimes it's okay to just have a list of do's and don'ts. It's served to remind me what I'm looking for when I brief a researcher, and next time a I do a quick internal job for a pitch or something, I'll use this to stop me being lazy.

The quick post would go something like this:

• Work from the client brief/objectives.
• List out and agree the topic areas before you do anything else.
• Be ruthless with questions- if you can cut it, do.
• Think about your key analysis breaks. How can the structure help?
• AND FINALLY AT ALL COSTS, PILOT IT! MAKE SURE IT WORKS BEFORE YOU START!!!

Comments

[Age]

These posts are incredibly useful to me NP, thanks a million.

At the place I'm currently at, there are really no research systems in place at all, in fact, research is regarded quite poorly. This is a great concern to me because as a junior planner I should be learning about this stuff in the workplace whilst getting some real experience under my belt if I want to move forward in this business.

Saying all this, that's why i really appreciate your posts about this topic. If not for the actual tips and insights you provide, but also for enlightening me on some planner specific tasks that I'm probaby not learning at the moment, and should be.

[NP]

Thanks Age, I'm really glad this stuff is useful. There's just not enough about basics out there.

[henry lambert]

NP, this is really great stuff. I wish I'd had this when I first started

[NP]

Thanks for the kind words. Hope it helps someone somewhere.....

[Chat]

theme site lince looks nice

[Sophia]

Very useful suggestions for my assignment. thank you.

[sohbet odalari]

Thank you for this very useful information.