Sequential Monadic Testing -- Researching Research

Prior to the mid 1960's the state of the art in consumer research was the Paired Comparison Blind Test. In the mid 1960's, the Single Product Blind Test started to emerge as the prime tool for assessing acceptability of a product.

In the early 1970's, in an effort to get more bang for the buck, the Sequential Monadic Test design emerged. The SM design took on various protocols for interviewing where following the use of the first product either a complete battery of questions was asked or just the overall assessment question plus an open-ended question about likes and dislikes were asked. Following the use of the second product, some people would only conduct a paired comparison interview while others would conduct a single product interview, followed by a paired comparison interview.

In the late 1970's, I conducted a research program aimed at understanding the effects of the different interviewing protocols. The focus was to understand the order effects in the SM design. Two actions were taken as a result of the research. Initially, we dropped the asking of direct questions (questions about specific attributes) following the use of the first product and prior to the use of the second product. We found that by asking questions about specific attributes, we affected how the respondents assessed the second product that was different from the criteria used in the assessment of the first product. Consider the following data.
 

Product	Base		Ratings
		Used First	Used Second	Average
A	163	6.84	6.25	6.55
B	159	6.53	6.48	6.50
C	156	6.51	6.38	6.44
D	162	6.89	6.44	6.66
Range		.38	.23	.22

Which set of data would you use: the results of the first use, the second use, or the average? (We used the results of the first product interview. This obviously defeats the purpose of the second placement.)

There is no doubt that you get more numbers for your research dollar with the Sequential Monadic design, but at what cost? I encourage anyone utilizing the Sequential Monadic design, to look at their data by order of use (questioning) to determine if they are introducing an element of error into the results via the protocol. Many will say that by balancing placement, they eliminate the order effect. True, if the order effect is an average order effect. Not true, if it is a product by order interaction (which in my experiences happens most frequently). A product by order interaction can have disastrous implications.

Later, we encouraged the reduction, if not elimination, of the Sequential Monadic design. The reasoning was that we were trying to reap the benefits of both the single product and paired comparison design. These two types of research serve different purposes, (see Views Pair vs. Single Product Testing September 3, 1994), and my experience is that you need to run them separately to achieve the desired research results.