← Back to DashboardBriefing ID: brf-1791715871978
RESEARCH RADAR

One thing worth reading this week

11 October 2026 · Curated for research-led data visualisation & design practice

01METHOD·UX Collective · 2026Article
The Idea

While synthetic users offer rapid simulation, LLMs suffer from an inherent sycophancy bias that uncritically validates design proposals and ignores real-world constraints. As a result, relying on AI personas risks creating false confidence unless strictly confined to low-level artifacts rather than genuine user research.

Why is this worth your time?

This is useful if you are evaluating whether to test interactive concepts or narrative workflows against simulated audiences. It gives you clear evidence of the 'prompt-to-please' failure mode, where language models systematically flatter your hypotheses while ignoring practical friction like user adoption and resource limits. Pay attention to how this dynamic distorts evaluative feedback, reinforcing why synthetic testing cannot substitute for empirical human observation.

Read:the section on 'The problem of overoptimistic LLMs', focusing on the 2025 study of the 'Alice' persona to understand where simulated feedback breaks down.
#METHOD#user-research#design#ai
▸Full Academic Abstract & CitationExpand

Full Abstract

“Human-centered design demands a limited use of AI-generated personas, if at all A big debate among UX researchers is whether to use synthetic users in their studies. There’s one group of researchers who consider it no less than an abomination, unethical to use an artificial persona to test a product that serves humans. The second group defends it as a helpful tool that makes user research more efficient. Looking at it through a pragmatic lens, I understand both perspectives. UX teams may consider using synthetic users for low-level artifacts, the parts of a study you can reasonably simulate. Still, the word synthetic already tells you what you’re working with. These users aren’t real. As researchers and designers, our goal is to build products that solve real people’s problems. We need human feedback. What if synthetic users can help us solve those problems without losing touch with the humans we design for? How far do we let them take us? The problem of overoptimistic LLMs A major challenge for any UX researcher trying to create a synthetic user is trusting a system that seems to always give them the answer they want to hear. LLMs are way too positive and tend to flatter the person prompting them. Many companies are using this “prompt-to-please” research to their advantage, knowing the risks it entails. It’s simply too convenient to have a system that generates test users who love your product and only give positive feedback. This, of course, creates a false sense of satisfaction that’s by all means unrealistic. A 2025 study showed how this overly positive tendency of LLMs has affected UX research. Its title is a participant’s verdict: “She was useful, but a bit too optimistic.” To conduct this study, the researchers gave eight UX designers an interactive AI persona called Alice to work with across three activities: user research, ideation, and prototype evaluation. Alice was fast and helpful, but she agreed with almost everything. The study describes “a consistent tendency to agree uncritically with design proposals.” For example, a designer asked Alice whether a two-hour session about organic farming in her restaurant would help her gain customers. “Yeah, that could be really effective,” she answered. No doubts or constraints. The researchers saw the same across proposals: responses stayed positive while the real-world complexities (cost, user adoption, resources needed) went unquestioned. Senior designers described the conversations as “one-way” and “s”

Source: UX Collective
Feedback:
This Week's Themes
METHODUSER-RESEARCH
One Idea to Keep
“A visualisation does not merely communicate data; it shapes what people consider relevant, uncertain, and actionable.”