Design / Research · Test first
Your AI buyer panel may agree too easily
Source published 2026-10-16 · Brief published 2026-10-17 · Updated 2026-10-17
What changed
NN/g published a comparison on 16 October of 120 synthetic respondents and an earlier survey of 604 people answering a UX terminology question. Both datasets supported the same broad conclusion, but synthetic responses contained fewer distinct answers. This is a narrow research finding, not proof that AI can replace buyer interviews.
What this actually means
Editorial interpretation: agreement is not the same as coverage. A simulated panel can produce a neat dominant answer while leaving out language or concerns that matter to a smaller audience. For B2B work, a less frequent objection may come from the person who can block a purchase. Treat generated reactions as prompts for investigation, not as customer testimony or a percentage of market demand.
Who should care
B2B teams using simulated personas to choose website messages, plan positioning or predict objections before speaking to customers.
The implications for your team
The commercial risk is misplaced confidence. Imagine a team choosing a product-page headline because several generated procurement personas prefer it. That exercise can help identify questions to ask, but it cannot show how actual procurement staff interpret the claim. Copy based only on plausible answers may miss a real requirement, such as implementation effort or approval documentation. This example illustrates a risk; it is not a finding from NN/g's terminology study.
Editorial recommendation
Your next useful move
Check one upcoming message or page decision. Label which inputs came from people, business records or simulation. If the recommendation depends mainly on generated reactions, collect human feedback before treating it as validated.
What could change in practice
The immediate change should be a more honest research record. Generated ideas remain available, but untested assumptions stay visible. Human interviews may confirm a broad theme, reveal a neglected concern or show that the question itself was poorly framed. None of those outcomes promises higher conversion.
How to put it into practice
These are recommended next steps. Confirm the feature or workflow is available in your account before changing production settings.
Name the decision and the uncertainty
Write the specific choice, such as which objection a product page must answer. State what would change the team's recommendation. Avoid asking AI to pronounce a general winner.
- Who owns it
- Marketing lead
- Check before moving on
- Research brief contains one actionable question and an explicit unknown.
Separate hypotheses from observations
Place AI-generated objections in a hypothesis list. Keep actual interview notes and permitted sales feedback in a separate evidence log with their source and date. Never turn generated quotes into customer proof.
- Who owns it
- Researcher or content strategist
- Check before moving on
- Every supporting statement is identifiable as observed or hypothetical.
Ask real people without leading them
Recruit relevant buyers across the roles involved in the purchase. Use neutral questions about their current process and what they understand from the page. Record recruitment gaps and follow your consent requirements.
- Who owns it
- Research lead and sales liaison
- Check before moving on
- Notes preserve unexpected answers and identify which roles were not reached.
Review exceptions before publishing
Compare themes and disagreements. Route important minority concerns to the relevant expert, then revise the message or explain why a concern is out of scope. Keep unresolved assumptions for the next test.
- Who owns it
- Page owner and product or sales expert
- Check before moving on
- Decision record links the selected wording to real feedback and lists remaining uncertainty.
How to check whether it worked
The useful output is a traceable message decision: clear source labels, relevant human input, recorded disagreements and no simulated evidence presented as customer research. Evaluate any later page change separately with suitable behavioral or conversion measures; a research discussion alone does not establish commercial lift.
What to keep in mind
The study concerned one terminology question, not B2B purchasing. Human and synthetic samples differed in role composition and response format; the platform sometimes rephrased the question. Synthetic responses were generated in February 2026. Publication on 16 October does not make them a test of every current model.
Read the original
Nielsen Norman Group · 2026-10-16
Synthetic Users vs. Human Respondents: Same Conclusion, Less Variation ↗