In short
- 1In a year, the interviews Rep Data throws out for poor quality grew 80 percent. Half the written answers it removes now look machine-written.
- 2This is not one supplier's problem. Pew screened an opt-in online sample and nearly half of it failed. In another study, an AI built to pose as a respondent passed 99.8 percent of standard data-quality checks.
- 3We see it in our own fieldwork. On our last 25 studies we estimate we have removed between 35 and 50 percent of consumer interviews before anyone looked at the results.
- 4So buy fewer interviews and screen them harder. That costs you speed and detail, and a calibrated simulation gives them back.
- 5If your last wave came back clean, look at your checks before you trust the sample. Bring us that wave and its exclusion report, and in 30 minutes you will see which checks it is missing.
Rep Data opened its ESOMAR Congress 2026 talk with a game: two open-ended answers to “Why did you choose your last car?” One was written by a person, the other by an AI agent. Read as text, the two answers cannot be told apart, and the in-survey basics do not help: 32 and 43 seconds, and no paste on either. The respondent metadata behind each answer tells them apart. The study behind the talk, Detect. Trace. Learn., covers 11,778 interviews from a US consumer survey fielded on real panel traffic between June 1 and June 28, 2026, with eight quality signals applied to every interview: three before the questionnaire opened and five on behavior during and after the interview.
If you buy research, this is your problem before it is the panel’s. Across Rep Data’s projects, Pew’s samples, and our own waves, the human answers that feed pricing, concept, and brand decisions are worse than the delivered data suggests, and in Rep Data’s projects the decline is measured in months. Simulated consumers can cover the ground that fewer trustworthy human interviews leave open, but only when the human surveys they are calibrated against, meaning tuned to match, are clean.
Quality trend across Rep Data projects
+80%
growth in completes excluded for poor quality, July 2025 to July 2026 (Rep Data, ReDem)
+141%
growth in terminations for AI-suspect open ends, January to July 2026 (Rep Data, ReDem)
Open-end quality across Rep Data projects
11%
of completes removed for inattentive or fraudulent open-end answers, July 2026 (Rep Data, ReDem)
52%
of open-end exclusions show signs of AI-generated text, July 2025 to July 2026 (Rep Data, ReDem)
What the study found
Rep Data ran its pre-survey tool, Defender, alongside its in-survey and post-survey tool, ReDem, on the same interviews. Pre-survey checks flagged 15.9 percent of interviews. In-survey and post-survey checks flagged 17.2 percent. Only 3.8 percent were flagged by both. The two layers correlated at r = 0.08, close to zero, meaning they caught different people, and 77 percent of flagged respondents triggered exactly one signal. Put the layers together and 3,461 interviews carried at least one quality flag, 29.4 percent of the 11,778 that had already passed the sample suppliers’ own pre-field screening, by our arithmetic on the published counts.
How often each of the eight signals fired, share of 11,778 interviews
Pre-survey, respondent metadata at entry
In-survey and post-survey, response data
Source: Rep Data, Detect. Trace. Learn. (ESOMAR Congress 2026). Bars scaled to the largest value (10.7%). Rep Data publishes per-signal rates and layer totals separately, so the five behavioral signals shown here do not sum to the 17.2 percent that layer flagged. Coherence flags answers that contradict each other, input behavior covers typing and paste patterns, and grid responses covers straightlining.
The team went looking for fraud archetypes, recurring combinations of signals that would describe a typical bad respondent. It found no distinct ones. Quality problems were spread across independent behaviors at different points in the respondent journey, and even the strongest pairs covered a small fraction of the flagged cases: fingerprint with open-end content (138 interviews) and open-end content with input behavior (131), out of the 3,461 flagged. The authors state their limits plainly: one US consumer survey, operational quality signals rather than verified fraud labels, and an adversary that keeps adapting.
What other studies find
The two AI figures have different denominators. Rep Data’s 52 percent is a share of the open ends it removed, not of all open ends. When Pew coded every open end in its own sample, probable AI text came to 1 percent. Both can be true at once: Pew coded every open end by hand before screening, Rep Data machine-flagged the ones it had already removed, and Pew’s sample was fielded in November 2024 while Rep Data’s figure covers July 2025 to July 2026.
What we see in our own fielding
Training simulated consumers takes human answers, so we collect surveys from thousands of consumers a month through the panel aggregators we work with, and we run our own quality layer on top of theirs. Across our last 25 studies, our own estimate is that we removed between 35 and 50 percent of consumer interviews before analysis, counting our in-survey controls and post-field checks. The panel had already made its own removals before that. Those figures describe our own fielding, not an industry rate.
Our own human waves
35-50%
of consumer interviews removed before analysis, our estimate across our last 25 studies
34 of 140
completes that passed first-pass checks, from a June 2026 executive and opinion-leader wave
On our own executive and opinion-leader wave the first-pass numbers were far worse. We fielded it in June through four sources in the US, and scored 140 completes on the first pass: 34 passed, 51 were rejected outright, and 55 needed manual adjudication. On a later day in the same wave, 74 records had already been removed as fraud or ineligible before that day’s scoring began, and 30 of 138 scored completes reached identity validation, a manual check of name, company, and role against a public profile. The failures were not subtle. Our adjudicators rejected records claiming senior roles they could not reconcile with the person answering, and respondents rated the reputation of companies called “shellz”, “brokylnz”, and “catzems”. One record claimed a general manager role at a global enterprise software company, another an executive sales director role at a global consumer electronics brand.
If your last wave came back clean, you either found the end of the rainbow or your quality control has not caught up. In Rep Data’s projects, in Pew’s sample, and in our own waves, nobody reports near-zero fraud.
Calibration lets a smaller human wave cover more ground
At NextMinder we build digital twin populations, our simulated consumers, from government microdata, real survey responses, and licensed behavioral signals, then back-test those populations against human surveys, category by category and country by country, wherever a comparison survey exists. In calibrated studies, simulated readings match human results with 85 to 95 percent accuracy. That figure assumes the human results are themselves human, which makes the removal work described above an input to the calibration.
You still need the panel: the screening signals live there. Simulated consumers let you spend the human budget on fewer, better-screened interviews and still cover more concepts, prices, and segments than the human wave alone can reach.
What to ask before your next wave
| Ask your panel | Ask your research partner | Ask anyone selling simulated consumers |
|---|---|---|
| Where do the checks run: before entry, inside the survey, after it? | What share of completes was excluded, and on which signals? | Which human data calibrates the simulated population, from which panel, and screened how? |
| How many independent signals, and do they include digital fingerprint and duplicate detection? | Are open ends screened for AI-generated text, and by what method? | How is accuracy measured, against which human benchmark, in which category and country? |
| When were the thresholds last recalibrated? | Is the exclusion report part of the deliverable? | What happens to the simulated population when the human wave turns out to be dirty? |
The first two columns follow Rep Data’s own advice to research buyers: use more signals, treat quality as a moving target, ask where the checks happen, and remember that different tools catch different problems. The third column is the one we want asked of us. Settle the last question before it arises: ask for the exclusion report before any human wave anchors a simulation, ours included.
Buy fewer human answers and screen them harder. A calibrated simulation covers the rest
Online panels were sold on the promise of scale at low cost. Where suppliers publish figures, a growing share of what arrives now fails quality checks, and at Rep Data the completes excluded for poor quality grew 80 percent in twelve months. Human answers are the input the whole calibration rests on, so buy fewer of them, screen them with several independent signals, and make the exclusion report part of the deliverable.
Screening harder is paid for in money, in fieldwork days, or in questions you no longer have room to ask, and none of that shows up in the quality report you eventually read. On one executive wave in June we scored 140 completes, passed 34, and sent 55 more to adjudication by hand. Rep Data’s advice to buyers names the checks to demand and says nothing about what they cost. That is the trade a calibrated simulation removes, because the human wave only has to set the anchor while the simulated population carries the speed and detail the smaller wave gave up.
Book a 30-minute walkthrough
Bring your most recent wave and the exclusion report behind it. In 30 minutes we will show you which quality checks that wave is missing. You will also see whether the checks it did run are enough to anchor a calibrated simulation of your category, and what that simulation would let you test without buying more human interviews. You leave with the list of checks to demand from your next supplier, whichever one you choose.
Schedule time with our teamNextMinder builds calibrated digital twin populations for consumer insight across the Americas. Accuracy figures refer to back-tested agreement between synthetic and real human survey answers in markets with a comparison survey; methodology available on request. Our own removal rates describe waves we fielded and are not an industry rate. Rep Data figures are from the ESOMAR Congress 2026 presentation and research brief (Berger and Mittermayr, 2026): the exclusion trends come from Rep Data’s projects assessed with ReDem; the signal-overlap figures come from one US consumer survey of 11,778 interviews, after suppliers’ own pre-field screening; both report operational quality signals, not verified fraud labels.
