The Synthetic Mirage: Why AI Respondent Panels Are Selling Market Research Buyers a Methodology That Doesn’t Exist
This is the most technically rigorous independent analysis of the synthetic respondent industry available anywhere. Drawing on deep quantitative research expertise, extensive evaluation of vendor tools and claims, and awarded patents in statistical analysis and data imputation, this report examines the significant gap between what synthetic panel vendors are selling and what their technology actually delivers.
The research is organized across three volumes:
The Core Report
This is the central document. It covers the four fundamental problems with synthetic respondent methodology: the black box problem, the illusion of individuality, unknown source data, and false precision in sampling. Written for research buyers, agency leaders, and anyone responsible for making decisions based on market research data. If your organization is using or evaluating synthetic panels, this is the document that tells you what your vendor is not telling you.
The Commercial Analysis
This goes deeper on the incentive structure driving adoption of flawed methodology across the industry. It documents the validation gap — the complete absence of peer-reviewed, independently replicated evidence that synthetic panels produce research-grade outputs — and examines who bears the cost when decisions get made on data that does not reflect reality.
The Technical Primer
This is for readers who want to understand exactly what is happening under the hood. It covers how large language models actually work, what token sampling is and why it bears no relationship to demographic sampling, why the RAG argument vendors use as a legitimacy claim does not resolve the core methodology problem, and why the people who built these models cannot explain how they reason — which means your vendor certainly cannot either.
Together these three documents represent something that does not exist elsewhere in the market research industry: a technically grounded, methodologically rigorous, vendor-independent analysis of the fastest-growing and least-scrutinized tool in the research technology stack, written by someone who has spent a decade building the kind of quantitative systems these vendors are claiming to replicate.