NORTHERN PLANNER

Entries are in, thanks everyone.

Since I’m on holiday next week, I’ll make sure feedback is posted by Friday.

So, in the follow up to the last post, it’s time to look at the using qual v quant. Much later on, we’ll go on about how to physically go about each one, how to brief, who to brief, methodology and the like. For now, here’s some rough and ready guidelines.

Qualitative pre-testing is good for:

• Taking people through unfinished stimulus
• Being able to match the mood of real TV press consumption
• You can probe more emotional responses
• You can explore ‘reasons’ why in more detail
• You can dissect elements better – what’s working, what’s not…AND MOVE ONTO WHAT WOULD IMPROVE IT THEN AND THERE. Mood, characters, music, setting, copy style etc.

I think it’s much better in enabling you to develop work and avoid ‘killing’ it – purely because the methodology allows ongoing dialog. I was testing some work for a bed retailer, which was all about showing empathy. The executions were all about understanding how important it is to sleep well, some about losing your edge, some about how on form you can be. But they bombed. That would have been it, but we asked people why this stuff wasn’t relevant. And they told us that the right bed matters for the bits when you’re awake. That perfect ‘me moment’ of reading in bed, a cup of tea, or whatever. And the work that developed from that ended up on TV and saw sales uplift by 20% straight away.

BUT if your primary objective is to prove that stuff will work, there is safety in the numbers that quant provides…..and you can measure impact a bit better too.

When you brief it in to the research agency you should:

1. Allow time for recruitment, fieldwork and analysis. I cocked up big time on my first little qual project when I just assumed the recruiters could sort out respondents and a venue in a week. They couldn’t, and I made the mistake of settling for an imperfect sample. No surprise that the learnings were totally confusing, and, I hate to say it, the wrong work ran, and stiffed. Lesson learned.
2. Choose the right researcher…..researching work means that you need the type of person who knows how communications WORK. And try to do some groups yourself if you can. When paper testing an e  – commerce site, I made the mistake of using a researcher who, I found out after, had never really done much work in new media before. I got loads of great feedback on branding ideas, but very little on how the site’s navigation should work. Thankfully I had James Boardwell to develop the site afterwords, who’s forgotten more about web stuff than I’ll ever know.
3. Ensure the sample matches the audience in the creative brief (see point 1!)
4. Bloody brief them properly! Make sure they have a copy of the creative brief, and they really understand the task, who the audience is and how you think the work is supposed to work. I was away, and the account team asked a researcher to test the viability of brand identities. They forgot to tell him they were supposed to convey intelligent simplicity, so we just got back a list of what people liked and what they didn’t. We had to do all over again when I got back.
5. Work hard at the stimulus. Ask the researcher what they think will work best – but be firm about what your own preference is.  Make sure all the work is presented at the best level of ‘finish’. More worked up ideas invariably win over rougher scamps. You don’t want to kill some work because it didn’t have a shot in, when the others did. And make sure the researcher gets the work early enough to prepare. We sometimes do internal groups here, just a few people who are not creatives, and don’t work on the specific brand. It helps get a feel for the real research. Even then, people kept focusing on some press stuff that was done on the mac, as opposed to some line drawings. The bloody, sneaky, creative team had ‘finished’ the work they liked the best and not told me until they handed over the boards. It took a lot of probing to find out they didn’t work as well, and naturally I got accused of trying to kill the team’s favorite route. Not pretty.
6. Ask them to separate creative strategy (is the brief right), the creative idea (is the core concept right) and the execution (is the detail right..casting, mood etc). I remember jumping up and down when a researcher essentially killed a creative route about personalization. It was about different people liking different stuff, and every execution didn’t work because it only showed one example of something you like or you don’t. It was too polarising. But he never showed the executions together – or projected showing lots of different things together. So the core idea just died, even though we had tons of evidence that it was right. Execution was wrong, not much else was…..
7. Make sure anyone affected by the outcome can see at least one group, either by video or even better, they are there. It matters. And let the researcher know this as soon as you can. One of the biggest shocks I had this year was going to a small scale presentation, only to find it was to half of the global team. It wasn’t fair to me, it’s not fair to anyone.

Quant is a tough one. It isn’t possible to make a realistic test of work in a laboratory situation. It just isn’t. The Stella Artois reassuringly expensive’ stuff failed in research……yet it’s one of THE most successful campaigns.

However, some think it makes sense to have some sort of appraisal of the work’s viability. The demand for pre-testing is huge, like it or not. And there are plenty of agencies just dying to carry it out.

They tend to have 7 measures of effectiveness:

1. Points communicated
2. Impact/stand out
3. recall of brand
4. Brand imagery
5. Emotional involvement
6. Preference or attitude shift
7. Open ended questions

They all have a habit of skewing it to structure, logic and rationality. Beware. And it’s hard for people to describe in a multi-choice format how they FEEL, or project how they would behave.

So when you brief a quant agency:

1. Make sure the sample matches the one in your creative brief
2. Do what you can to make sure the questions match how you think the work will make people respond. This is bloody hard when most agencies use a standardised procedure. You need to make the researcher your friend and do what you can. At the very least, make sure it’s acknowledged what the differences are. I was faced with a researcher who was adamant that quality messages on some supermarket POS were a waste of money. His methodology was all about what people would do that day, instead of long term behavior and consideration . I had to get him to admit he hadn’t addressed LONG TERM behavior anywhere.
3. Remember, this form or research is highly defensive, it stops bad work running, but it’s very hard to use it to find out what you need to do to improve it.

So there you go, some hard and fast guidelines to what some call pre-testing, but I like to call development research. The difference is crucial.

Anyone bored yet? Is this useful? Too basic? Do let me know.

I’ll tell you one thing, it’s great going right back into so called standard practice now and again. It’s easy to get into bad habits.

Before we start, let me just say that I’m neither suggesting for or against creative development research. It’s helped me in the past it’s true, but some would argue that putting ideas into research strips out any chance of something surprising or new eventually running. Whatever you may think, it’s unlikely you’ll be able to avoid doing it at some stage. And when you do, use it to make the work better, not kill it.

Like I said before, read John Steel’s Truth Lies and Advertising for some great in depth advice and case studies (Sega in particular). You may be required to use qual, or quant and we’ll do them in detail later. For now, here are a few pointers…..

1. You are NOT asking consu mers to judge the work, to decide if it’s right or not, or give an opinion on if it’s good or not. You’re finding out if the work DOES what you want it to DO. And if not, WHY NOT? What would need to CHANGE? You’re helping the work move forward, not kill it. Unless there really is nothing to salvage. Then you have a problem. Either the brief was well off beam, or the wrong work has been allowed to develop.
2. You’re looking for insights into HOW people are likely to react to the idea……reducing the chance of it not working. It’s a pretty defensive process on the face of it. It can stop you making bad work, but it’s a bit harder making it help create good work. That said, there will be times when a nugget comes out that shows you how to deliver the work ina better, fresher way, but don’t expect them. and it can destroy it. By definition, you’re telling people something new, stuff they’re not used to. So ALL comments should be taken with a hefty pinch of salt. So make sure it’s all done in a constructive way. The ideas are usually fine, it can be little things, like the voice over, or the music, or sometimes people just don’t like the look of a certain character.
3. You really do need to define the criteria you’re judging the work against. You MUST define how you want the work to work, then you can compare the actual response. I think you need to ask these questions of the work first:

What’s the commercial objective? Increase sales? Generate sales leads? Price premium?

Who do you want to effect? Make loyal users more loyal? Convert rejectors?

How will the work enhance the brand relationship and opinions of it? Enhance brand values? Benefits?

How will the work achieve it’s affect? Dramatize product benefit? Engaging portrayal of how it was made? A world without it?

By the way, that’s a pretty good set of criteria for your own creative reviews. The worst thing yo can say about creative work is that you JUST, you like/don’t like it.

Anyway, make sure you, the client and the researchers acknowledge what the criteria is BEFORE.

And by the way again, there only seems to be four main aims for advertising:

1. Sales…..get them to register a worthwhile opportunity
2. Persuasion…communicate a credible and convincing point of difference
3. Involvement…..the become involved and identify with the work
4. Salience….it stands out as different

And, of course, you should be looking to address at least more than one of these aims!

This is getting quite long, so we’ll take a break for tomorrow. We’ll be continuing with some pointers on qual pre-testing v quant.