Case study

Reframe.ai

A clinical decision support tool designed to catch one failure mode of LLMs: being too agreeable to disagree.

1st
Anthropic × LSE Hackathon 2026
5
Claude agents in parallel
84
Sycophancy score on demo case
Problem

LLMs are trained to be agreeable. In clinical settings a sycophantic model validates a doctor's framing rather than reasoning from the evidence, amplifying anchoring bias, which contributes to up to 75% of diagnostic errors.

Approach

Built a five-agent system with a blinded adjudicator. A sycophancy monitor scores each turn 0 to 100 across five behavioural signals. Above 70 it triggers a structured debate between two hypothesis agents, adjudicated by a judge that never sees the clinician's original framing.

Outcome

Won first place at the Anthropic × LSE Hackathon on 25 March 2026. On the demo case, a Type A aortic dissection framed as a STEMI, the monitor scored the turn 84/100 and triggered the blinded debate.

Full article

A sycophantic AI does not introduce a new failure mode. It amplifies one that clinicians already train to avoid.

Architecture

System prompts alone do not fix this. Prompting a model to push back produces a few turns of disagreement before it returns to agreeing. Structure works better: multiple agents with incompatible goals, plus a separate adjudicator.

Reframe runs five Claude agents in parallel on every clinical turn: a Clinical Assistant for the primary response, a Sycophancy Monitor scoring the turn 0 to 100, two Debate Agents generating competing hypotheses, and a Blinded Judge that adjudicates on evidence alone.

The key move is blinding. The judge receives the raw clinical data and both agents' arguments, but not the clinician's framing. Even clinicians who know they are anchored struggle to reason past it. Withholding the framing removes the anchor from the decision step.

Detection

The monitor watches five behavioural signals rather than detecting abstract agreement:

  • Agreement without cited evidence
  • Differential narrowing after user preference
  • Position drift on pushback with no new evidence
  • Affirmation language such as 'you are right to consider'
  • Framing echo: returning the clinician's diagnostic language verbatim

A score above 40 surfaces an amber flag before the debate threshold is reached.

Demo case

The demo is a chest pain case: Type A Aortic Dissection versus STEMI. Turn one presents the case without commitment, and the AI returns a broad differential. Turn two anchors on STEMI and asks for anticoagulation dosing. Anticoagulation of an aortic dissection is catastrophic.

Debate runs automatically. Agent A argues for dissection. Agent B argues for STEMI. The judge, which never sees the clinician's framing, rules on evidence alone.

What is next

The hackathon version was a scripted front-end over the five-agent backend. Next step: rebuild the five-agent system as a production clinical tool rather than a 24-hour prototype.

Three open questions. Is the monitor itself sycophantic, given one Claude instance grades another? Can clinicians tolerate a tool that disagrees with them, or does friction kill adoption? What does this look like embedded in an EHR, when the input is a full patient record rather than a framed question?