Benchmarks

Recently I was asked what I thought was the most important things to remember in consumer research.  After thinking about the subject, I would have come up with a long laundry list.  However, at the time I said, "Use benchmarks in your research and validate your findings."  Their response was that the use of benchmarks was expensive.  I believe that not using them can be even more expensive.  Benchmarks provide two very important perspectives to the data.  They allow for two relevant comparisons.  First they provide a comparison with the market as it exists at the time and second, they provide a comparison with past research.  In the case of the latter, there are too many variables acting on our research to expect that any research can be carried out without a bias or freak of nature (random fluctuation) generating misleading results.

What do I really know about Historical Data or Norms?  Not much.  I do know, however, that they are collected across time, in a variety of locations, under a variety of environments, by a variety of interview protocols, and by a variety of interviewers.

What do I know about the collection of data?  I know that data are affected by when, how, where and the conditions surrounding the research.  Just looking at the Purchase Intent Question, we found that how the question is asked can change the Definitely Would Buy scores for a single brand by 15%.  We also found that the locations of testing can change the DWB results of the same brand by 20%.

When you use historical data, ask yourself, are the data from tests conducted under the same conditions of the study at hand or are the historical data from a mixture of all types of conditions?  What should we be comparing our data to?

If we do not have a benchmark, how do we know if we have biased data?  How do we know that it was not one of those random cases where most respondents just happen to be agreeable or disagreeable?  We don't if we do not have a benchmark.  The benchmark is the trigger for letting you know if something out of the ordinary is affecting your data.

Should we be aware of the age of the data in our Historical database?  As markets evolve, does the data become dated?  Are we comparing our results with antiquated data?  Should our comparisons be with the markets of today or those of yesterday?  Benchmarks will not only deliver current comparisons but will also let you know how the markets are changing over time.  Benchmarks are a valuable tool in more ways than one.

A LITTLE ON THE LIGHTER SIDE:

Real courtroom questions:
 

•     Now doctor, isn't it true that when a person dies in his sleep, he doesn't know about it until the next morning?
•     Were you present when your picture was taken?
•     Did he kill you?
•     How far apart were the vehicles at the time of the collision?
•     You were there until the time you left, is that true?
•     How many times have you committed suicide?