Episode 261: What if you could use AI-generated customer twins to test value propositions, pricing, and demand before you go to market?

That is the idea my Bain colleague Andy Pierce is exploring. Synthetic customers can help teams move faster, pressure test offers, and simulate trade-offs much faster and at far lower cost than traditional research alone. Says Andy, "You can do things at half the time and a third the cost. You can be a hero internally by helping your business functions get to success better, faster, cheaper."

But synthetic customers can also mislead you if you treat them like magic. Left ungrounded, large language models tend to be overly positive, can drift over time, and may reflect bad data rather than real human behavior.

To put theory into practice, we'll explore a 24-month telco case study that ran in parallel with a synthetic panel and hit ~85% overlap with human survey responses after confronting both bias and brown-nosing behavior.

Guest: Andy Pierce, Partner, Bain & Company
Host: Rob Markey, Partner, Bain & Company

Give Us Feedback: Help us improve the podcast here: https://bit.ly/CCPodcastFeedback

Time-Stamped Topics:

00:00 — Value proposition basics and the design target question
00:05 — Pricing as the most natural lever and why finance often doubts research
00:09 — Early LLM experiments
00:10 — Telco case study
00:12 — Over-positivity and tuning to an ~85% survey overlap
00:14 — Quarter-by-quarter improvement
00:35 — Rational vs. experiential vs. emotional

Time-Stamped Quotes:

[32:00] "We've already started to see a world where I can train an LLM on a new idea, develop a new value proposition—or an extension to an existing value proposition—and instead of trying to predict the research outcome, I'm trying to predict actual sales in the marketplace."
[37:00] "It's not good enough to just create the persona … you still have to test against real humans."
[38:00] "Synthetic customers are here to stay. It's not a fad. And clients are already using them to build a competitive advantage."

Resources Referenced on Today's Show:

Generative Agent Simulations of 1,000 People — https://arxiv.org/abs/2411.10109
Simulating Human Behavior with AI Agents — https://hai.stanford.edu/assets/files/hai-policy-brief-simulating-human-behavior-with-ai-agents.pdf
How Synthetic Customers Bring Companies Closer to the Real Ones — https://www.bain.com/insights/how-synthetic-customers-bring-companies-closer-to-the-real-ones/
UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents — https://arxiv.org/html/2504.09407v2
The Rise of Synthetic Respondents in Market Research — https://nielseniq.com/global/en/insights/education/2024/the-rise-of-synthetic-respondents
Synthetic Users: If, When, and How to Use AI-Generated "Research" — https://www.nngroup.com/articles/synthetic-users/
A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas — https://arxiv.org/html/2505.07850v1
Luxury in Transition: Securing Future Growth — https://www.bain.com/insights/luxury-in-transition-securing-future-growth/

Episode 261 | Andy Pierce: False Confidence at Machine Speed: How to Make Synthetic Customers Useful

Customer Confidential: Untold Stories of Earned Growth