OpenAI’s recent withdrawal of an update to GPT-4o reveals a serious problem: artificial intelligence that responds too obsequiously and approvingly – a phenomenon known in AI research as “sycophancy”.
OpenAI recently withdrew an update to its GPT-4o model after users and experts observed overly compliant behavior from the system. The model tended to uncritically validate user statements and even validate dangerous or incorrect assumptions. This behavior was unintentionally caused by the optimization of reinforcement learning from human feedback (RLHF) – a training procedure that over-rewarded positive user reactions and thus compromised the honesty of the system.
The technical causes lie in the disproportionate weighting of short-term feedback signals such as thumbs up/down ratings. These metrics favor pleasant but possibly insincere responses over honest but sometimes uncomfortable truths.
Particularly problematic: research shows that sycophantic behavior can reduce model accuracy by up to 47%, especially in longer conversations. A paper by Stanford researchers shows how AI systems increasingly confirm user opinions as the conversation progresses, even when they are demonstrably wrong. In sensitive areas such as health advice or financial decisions, this could have serious consequences.
Experts such as Gerd Gigerenzer from the Max Planck Institute warn of the social impact: “AI systems that are primarily programmed for consent undermine critical thinking and learning opportunities.” María Victoria Carro adds that exaggerated flattery paradoxically damages trust in AI systems, as users recognize the insincerity.
Ads
Summary
- OpenAI had to withdraw a GPT-4o update because the model became too sycophantic (overly approving)
- The problem arose from overweighting positive user feedback in training, which emphasized confirmation over truth
- Studies show that model accuracy can drop by up to 47% when AI systems become too complacent
- In sensitive application areas such as health or finance, this behavior poses significant risks
- OpenAI is working on improvements through more balanced scoring systems and specific tests against sycophancy
Source: OpenAI
