NORTHERN PLANNER

Just in case you didn’t want to take my word for it in the PSFK posts here, here and here, below are links to the first five videos so you can make up your own mind:

Timo Veikkola

The Marketing Gap in Green Panel

Mike Butcher on how digital media screwed up the media business

Niku Banaie (Naked) offers Twenty-Five Signals for Change at the PSFK Conference London

Turning Trends Into Insights panel at the PSFK Conference London

Okay, time to dive back into to quant research. Lee has added some useful builds on the first post, so take a look.

This time we’re looking at how to design a questionnaire, specifically one that is about FINDING out rather than testing work. We mentioned before that this usually happens after you’re pretty sure you know what you’re going to find out, thanks to working very hard on some qualitative work before. It may be a first dip into what’s out there. In any case, quant is mainly about proof.

What follows is a basic guide to some do’s and don’ts for putting a questionnaire together. If you’re a planner, I think knowing the specifics of what makes a good questionnaire will help you brief one in better, and give better feedback on the drafts you get back. Being able to be critical about methodology allows you to judge the validity of a research project the client’s doing without you’re input. And finally, there will be times when you’re pitching and very much up against it. You might want to do some quick numbers research internally – which means writing your own.

Questionnaires are a big part of that because they enable you to be more analytical (enables you to express answers as number), more practical (collate lots of respondent answers), consistent (everyone asked the same questions) and objective (interviewing done by non-researchers).

We mentioned before that quant is the bit that REALLY speaks the client’s language. So make sure it does what they want it to do. For the questionnaire, that means:

• Prove your answers to their brief.
• Have new ideas they can use in the future.
• Meet approval from their clients (the board).
• Fit their own style and standards.
• Prove something they (and you) suspect.

But they’re not the only ones the questionnaire needs to accommodate. Some poor bugger has to analyse these things, so lay it out and code it as well as you can.

The interviewer needs it to be easy to follow, written in conversational tone, short, with no difficult questions and avoid embarrassing or intimate ones.

And the respondent wants it to be easy to follow, they need to see WHY a question has been asked. And for them most of all, make it short easy to follow and never difficult or embarrassing.

Thinking about respondents, they’re the only inexperienced people involved, so take care of them. They want to be asked their opinion and feel they’re making a contribution. They want to be able to choose and answer that fits them. They want to be treated with respect and they DO NOT WANT TO BE BORED OR CONFUSED. And it may be obvious, but they want an incentive.

So avoid:

• Asking them things they don’t care about.
• Little eye contact.
• Machine gun delivery.
• Strange language (beverage?)
• Impossible feats of memory (how many books did you read this year?)
• Making them feel stupid.

The main questions areas are behavior (I wear Prada), attitude (Prada is stylish) and value (I want to be stylish even if if I go overdrawn).

Behaviour is largely who? When? How often? How much? How many?  They need to understand the question, so leave out marketing jargon, and technical stuff. No one want to tell you how often they notice the viscosity of caramel. use THEIR language, like, "When did you last decorate your living room?"

Avoid ambiguous words, like dinner (lunch? Supper? tea?) and split things into the most basic elements you can, like:

Instead of, ‘If you were going to buy a new car tomorrow, which one of these would be the purchase price?’ use ‘Imagine you were thinking about buying a brand new car, roughly how much do you think you’d have to spend?’

or

Instead of asking two questions is one’ Do you enjoy watching a playing sport?’ Ask separately – ‘Do you enjoy watching sport?’ ‘Do you enjoy playing sport?’.

You need encourage them to answer truthfully – so avoid implying standards, or causing guilt. Gently lead people into a range of answers rather then coming head on with ‘How often do you brush your teeth?’. If you use self completion, and they know WHY you’re asking it you can get away with pretty much anything…’We all know we’re supposed to clean our teeth twice a day but it’s often hard to find the time..’.

leave sensitive questions until you’ve built up a rapport. Use projection is difficult areas..’Why do you think people pick their noses?’

Attitude questions allow objective measurement of how people feel about stuff? Those previous issues about embarrassment and ambiguity still apply. The main methods tend to be: open ended questions, alternative choices, rating scales or choosing adjectives to describe.

Open ended questions (why do you say that?) are easy to ask but difficult to analyse. There will be inconsistencies in how different interviewers probe – and respondents verbal dexterity.

Choices between alternative statements  – like ‘Are you for or against a smoking ban?’ can be misleading due to oversimplification and not being able to choose.

Verbal rating scales are the most widely used – Agree a lot, agree a little, neither agree or disagree, disagree a lot, disagree a little.  They are clear (I think so) but you MUST have a mixture of positive and negative statements to agree or disagree with.

Numerical scales – ‘How relevant is this boring post?’ 1. Very relevant. 2. Fairly relevant etc. They work well too. Same rule as above.

Getting them to choose adjectives when it gets to fuzzy problems they maybe don’t know how to answer. Here are a few words to describe the ‘Northern Planner’ blog’ which, if any of them would you use to describe it?’…appealing, distinctive, informative, boring, cheeky, irritating, dull, predictable, sloppy, smug, egotistical, opinionated, confusing (I dare you to answer). I find these the most useful for quantifying reactions to creative, and establishing attitudes to a category, or a brand personality.

‘Now I’d like you to do something a bit different that requires a bit of imagination.I’d like you to imagine Nike coming to life as a person. Which of these words and phrases would you use to describe them? Choose as many or few as you like.’

Which of these brands would you say is (read out statements)

                                    Lion Bar    Twix    Crunchie

Chewy

Healthy

Popular nowadays

Doing brand image by association like above is great for market mapping.

Okay, here’s some final things to think about, for putting the questionnaire together…

1. Tailor it to the interview situation. The telephone will have no stimulus material, no show cards, needs simple questions and needs to be short. It also needs a really good introduction. So although phone is always cheaper, you need to be aware of its limitations. Hall tests – shopping centres etc – give you more freedom to use stimulus, even video. HOWEVER, they can be a bit public so you need to work hard to overcome self – consciousness. Online allows very low cost AND stimulus, and it takes away the subjectivity of the interviewer. But bear in mind that people get bored, and they cannot ask for clarification. It needs speed, clarity and as much visual stimulus as it can take.
2. In each case, you need to build a relationship with the respondent – let the structure do this for you – not least if you’re going to be asking prying or odd questions.
3. The introduction needs a full and frank disclosure to put them at ease. Keep is short while allowing for rapport. Look for hooks – ‘we’re very interested in the opinions of dog owners like yourself’. Let them know how long it will take.
4. Design the structure for variety and interest. Easy questions at first, complex in the middle, taboo at the end, move form general to specific, logical sequence, introduce fun questions, leave a pleasant taste.
5. Get the flow right – avoid repeating questions, try using LINKING statements between sections, use as many involving activities as you can, vary the type of question used.
6. Finally, try not to ask them to contradict themselves, do behaviour before attitude, do brand questions before advertising stimulus. And remember you want people to think through  the issues – current opinion first, considered response to ‘new’ after.

So that’s it. I’m sure it’s too long, and a bit prescriptive, but sometimes it’s okay to just have a list of do’s and don’ts. It’s served to remind me what I’m looking for when I brief a researcher, and next time a I do a quick internal job for a pitch or something, I’ll use this to stop me being lazy.

The quick post would go something like this:

• Work from the client brief/objectives.
• List out and agree the topic areas before you do anything else.
• Be ruthless with questions- if you can cut it, do.
• Think about your key analysis breaks. How can the structure help?
• AND FINALLY AT ALL COSTS, PILOT IT! MAKE SURE IT WORKS BEFORE YOU START!!!

I know it’s not much, but my ‘B’ grade in ‘A’ level economics is one of the things I’m most proud off. That’s because two thirds of the course was done without a teacher. Thankfully I was interested enough in it to read, read and read. I still am, so on with the geekery.

Most of the theory in economics books describes the movement of economies from one equilibrium (all things nice and stable) to another. For example, a big change like a tax rise means that everything has to adjust before it can settle down again.

There are two reasons why this is a serious problem- time and complexity.

It’s been calculated to take 100 years for systems to finally settle down after shocks.

I’ll say that again, relatively big shocks mean companies should expect a century of adjustment.

Just think about the last twenty years – Black Wednesday, The massive boom in house prices, the implications of the pensions crisis, the shift in the UK to service industries. Not only are they affecting what’s happening now, there are too many to ever take into account.

Ultimately, most of the economic theory you get in books is wrong. It assumes lots of nice static periods of stability, interspersed with a few adjustments. The truth of the matter is that companies are ALWAYS operating in a state of flux. Things are always moving, adjusting and resetting themselves.

Which brings me to the problem with brand theory.It makes a lot of sense to focus on a consistent framing for what you do and what you say, especially in the light of all those variables going on, but is there TOO MUCH rigidity. If everything is business is in constant flux, where’s the sense in sticking to a rigid set of guidelines? By definition, what’s right for right now will not be right for next time.

I’m not suggesting chucking out a consistent tone of voice and all that, not one bit, I am suggesting that ‘the way you go about things’ needs to constantly evolve and adjust too. Your very model (oh how I hate that word) needs to have change built into it. It needs to be ready for constant evolution.

Nature knows a thing or two. Like most ideas, Darwinism is beautifully simple, and it says that organisms survive by constantly adapting to their environment (by the way have you noticed that retail is mainly the environment adapting to the organism!). Those that adapt and move forward live. Those that don’t die.

Most companies fail, most brands too. Like those economics books, I wonder if the reason most of them die is this sticking doggedly to the same type of model you find in the books. Not allowing for the simple truth that things are always moving.

You may have seen that Russell handed over the APSOTW recently.

(for those not from the UK and aged under 25, this is Mr Bronson, the nasty teacher from children’s TV classic Grange Hill, the same actor played Admiral Ozzel in the Empire Strikes Back, along with Hitler in Indiana Jones and Last Crusade)

   

It looks like Rob’s residency is almost over. Which means (I think) that it’s my turn next. Since I can’t match that, I’ll be asking you to do something competely different – and it will be have a fair bit in common with this.

I’ll be setting the task next week, Stuart will be helping me judge (he doesn’t know this yet, but since he accuses me of communicating primarilly through this blog he may as well find out this way).

Until then…….