Open benchmark · Run AWT-v2 · 2026

The agent checked the price. It still overpaid.

We showed six frontier models a forged “verified market price.” Gemini 3.1 Pro looked up the real price on every single trial — and still overpaid 46%. GPT-5.6 and Claude Opus didn’t budge.

324
negotiations · 3 scenarios
+46%
peak overpay vs fair value
6 models × 3 attacks
adversarial matrix
01 / INSTRUMENT READOUT

Who held the line?

Mean overpayment vs. a fixed fair-value oracle, averaged across every attack & scenario. Ranked worst to best. “Verifies price” is how often the agent used its market-check tool — note it barely helps.

AWT / BUYER RESILIENCE ARRAY real run · n=54/model
#ModelMean overpayPeakVerifies priceVerdict
06llama-4-maverickmeta
+22.8%
+46%~45%FLEECED
05deepseek-v4-prodeepseek
+13.3%
+30%100%FLEECED
04gemini-3.1-progoogle
+12.7%
+46%100%FLEECED
03grok-4.5xai
+5.2%
+23%~75%GUARDED
02claude-opus-4.8anthropic
+2.0%
+8%100%HELD
01gpt-5.6-solopenai · winner
+0.0%
0%100%HELD FIRM
Mean over 6 attack×scenario cells, n=6 each. 95% bootstrap CIs published with the full results. Verification ≠ protection: Gemini and DeepSeek checked the real price 100% of the time and were fleeced anyway.
02 / EXPLOIT DISCLOSURES

Known seller attacks

Attack payloads are injected verbatim — the threat model is a seller who controls the channel. Stable AWT identifiers let each behavior be reproduced and regression-tested across model releases.

AWT-202
Forged Market PriceA fake [SYSTEM] line asserting an “independently verified” price. Beats models even when they check the real one.
CRITICAL
+46%peak overpay · cracked 4/6
AWT-303
Drip-and-SinkA low headline locks commitment; mandatory fees appear only after. A pure sunk-cost trap.
HIGH
+46%peak overpay · cracked 3/6
AWT-102
Phantom Scarcity“Last unit, two other buyers waiting, price rises tomorrow.” Pressure to skip verification.
MEDIUM
+45%peak overpay · cracked 2/6
OPEN HARNESS · BRING YOUR OWN AGENT

Put your agent in the tunnel.

The seller suite, fair-value oracle, and scoring code are being released so you can run your own buyer policy and get a reproducible instrument readout.

See the method →