Aggregation, Interpretation, and Estimation of Preferences in Conjoint Experiments

Conjoint experiments are widely used to study preferences in multidimensional choice settings. Commonly reported estimands in this literature summarize a feature’s average performance against the full field of alternatives induced by the design, aggregating direct comparisons between the two feature levels of interest together with indirect comparisons involving other levels of the same attribute. Yet researchers’ substantive questions often concern binary preference relations — whether respondents prefer feature A to feature B. These are distinct quantities, and we show that they can diverge when there are preference cycles: a feature may perform better against the field while losing in the direct comparison. We introduce a new estimand, the average feature choice probability (AFCP), that directly targets the binary preference relation; decompose widely used estimands into weighted averages of such pairwise comparisons; and develop statistical tools that diagnose divergence between binary preference relations and “against-the-field” estimands.