Maximize Power: Online Panels and the Price-Quality Trade-OffLaura Seelkopf

16 January 2025

Laura Seelkopf (LMU), 17:15 - 18:45, presents her project  in the Interdisciplinary Research Seminar of the GRK "Collective Decision-Making”.

Location: Room 0079, Von-Melle-Park 5

Abstract

Survey-experimental research with online access panels of marketing research
companies has increased markedly in the social sciences. These allow for fast access
to quota-representative samples, which promise generalizable claims combined with
causal inference. When selecting among companies researchers are confronted with
a large price range – where high-price companies advocate their services as higher
quality with respect to respondent attentiveness, honesty, and representativity along
non-quota characteristics. Hence, researchers face a potential trade-off between
sample quality and the statistical power their research budget can buy. Our study
offers insights into this trade-off for the largest European economy. We asked all
marketing companies competing on the German market to offer bids for scientific
survey experimental research. Selecting three companies with a low, medium, and
high price, and corresponding self-reported low, medium, and high panel quality, we
fielded identical vignette, priming, and conjoint experiments to these three quota-
representative samples, together with standard socio-demographic and attitudinal
questions relevant to research in political behavior. Our evidence suggests that at
least for experimental designs, the price-quality trade-off is less severe than often
assumed. We thus recommend maximizing statistical power by minimizing costs
per respondent.

Keywords: survey methods; survey experiments; public opinion; online-access panels; statistical power

Find the abstract as PDF here.