This is really interesting. And, I read it here at The Toad Stool first.
Arnold tested the ad “1984” – that epic Apple ad that still gets mentioned in research groups and is one of the most famous ads of all time globally.
Except, they didn’t test it as an ad – they tested a re-created ‘animatic’ or ‘board-o-matic’ version.
Here’s the result, including the full animatic version (note how long they run – about 3 mins 30 secs to describe a 60″ commercial) AND the excerpts from the focus groups. Now, although the results are seriously shocking, bear in mind that the animatic they used was really good quality – they’re rarely that good. Also, it takes every bit of a finished film – which usually is better than the original script. So, to all intents and purposes, this is better than most stimulus produced for testing.
So, what does it really tell us?
That 1984, the commercial idea, is no longer relevant to consumers today?
That testing ads doesn’t work?
(both of which probably have a good degree of truth to them)
Or that we’re asking the wrong questions? Because, that for me is the right answer.
Ask people what they understand – dig for the message that is told. Then dig for the values – the softer associations and what they mean for this particular company. These are all questions that people can reasonably respond to. Without speculating or guessing at the answer.
But, DON’T ask people what they THINK of an ad. NOR what they would do to improve it. Don’t get them to list their likes and dislikes. And don’t ask them whether they felt it was clear who it was for. These are all questions that people can only guess at. Or that they’re not really qualified to answer.
We wouldn’t listen to our parents tell us how to edit/adapt/improve/change an ad developed by your best people, unless we heard the comment and said “Damnit, they’re right!” So why do we listen to consumers doing the same thing when we’ve asked them questions that are just as speculative?
Consumers can’t tell us the answers, but they can react to what they see and give us some indication of where we’re doing well and where we need to do some more thinking. Let’s not damn research for being useless if we don’t use it right – but let’s not blindly follow 20 people from the suburbs just because they’re “real people” either!
And, in case you forget the original ad, here it is…