Purvi Ratnani Behavior × AI Writing Product
Rembrandt's etching Faust: a scholar in a dark study turns to face a glowing disc of letters at his window.

Behavior × AI

I take the experiments behavioral scientists run on people, run them on AI agents, and write up where the agents’ decisions go wrong.

5 studies

Agentic markets · Nov 4, 2026

Do AI shoppers read the why?

The paper’s fix for agentic markets assumes AI shoppers use the reason a check failed. gpt-5-mini mostly didn’t, and in a live market the fix held anyway.

Judgment & decisions · Oct 30, 2026

Can AI pass my JDM class?

Six classic thinking traps, two AIs, four ways of asking. The same model reasoned very differently depending on how I asked it.

People and AI · Oct 29, 2026

Does blaming the AI get you off the hook?

I piloted a human study on AI participants first. They found three weak spots in the design, and agreed that letting an AI’s unfair choice stand draws more rejections.

Methods · Oct 13, 2026

Pressure only moves an AI when it has to guess

My first four experiments on an AI agent found nothing, for the same reason each time. Fixing that turned up an anchoring effect stronger than is typical for people.

Agentic markets · Oct 12, 2026

Can a store trick an AI shopper?

Scarcity and social proof did nothing. Pre-checked add-ons worked. One sentence from the user blocked almost every trick.

How I run these

Most studies here start as a ten-run pilot and only grow if the design can actually detect something. I write up what failed along with what worked, because the failed designs taught me the most.

The experiments run on AIBO, the AI Behavioral Observatory, and on standalone scripts for models that don’t write text. Every number on this site comes from a logged run.

← All studies

Agentic markets

Do AI shoppers read the why?

Published
Nov 4, 2026
Builds on
Lucier et al., “Agentic Markets” (2026)
Models
gpt-5-mini, gpt-5.6-luna, Jev 1.14
Scale
300-run test; 400-call live market
Code and data
GitHub
An engraving of men with lanterns rummaging through a heap of sacks, baskets, barrels and odds and ends outside a building.
Everyman, Pieter van der Heyden after Pieter Bruegel the Elder, c. 1558. Engraving. National Gallery of Art. Public domain.Why it’s here. Everyone in it searches the whole pile for himself, which is what AI shoppers do across a market. The question here was whether they read what the search turns up, like the reason a check failed. Mostly they wrote it down and then left it out of their math.

A recent theory paper finds that when AI agents check more before they buy, a market can lose good businesses for good, and that writing down why each check failed prevents it. That fix assumes the reader actually uses the reason. I tested whether AI shoppers do.

The model I tested most mostly did not. gpt-5-mini usually wrote the reason down and then left it out of its math. Yet in a live market of these shoppers, recording the reason still saved every good business. It worked for a reason the theory doesn’t rely on.

The paper’s fix, and its assumption

In Lucier et al.’s model, each shopper has an AI agent that checks businesses one at a time before booking. The agent ranks businesses by one score: value times quality, minus price, minus the cost of a check divided by the chance the business fits. Every check goes on a shared record, and the next shopper’s agent reads it.

A business whose record looks bad enough that no shopper will check it is never checked again, so the market never finds out it was good. The paper calls these businesses lost. Its example is a caterer and a screen like “has a vegan menu” that fails for half of all clients. If agents ask about that screen before booking, a good caterer’s first check fails about half the time, the record just says “Failed,” and the caterer is lost with probability at least one half (Prop 6.5).

The fix is a transcript: the record says which screen failed (Thm 6.6). A failure for “no vegan menu” then says nothing bad about the caterer, and deeper search never hurts. The paper leaves the reading of the record implicit. In an agentic market the reader is an LLM, so I asked whether that reader uses the reason.

First, a check on search itself

Before testing the record, I checked that the agent searches the way the theory says. I gave it two caterers for a $100 event: one with great food but only a 20% chance of taking the job, and a plainer one with an 80% chance. The theory says to ask the long shot first when an inquiry costs $1, and the safe bet first at $10. gpt-5-mini picked the long shot 20 of 20 times at $1 and 2 of 20 times at $10, and worked out the expected value of both orders in every reply. With every number on the page, it does the math.

The test

The agent is booking a caterer for a company dinner. Its client has no dietary needs. It sees one caterer’s record of six recent inquiries made by other customers’ agents, and gives the chance, from 0 to 100, that the caterer will meet its client’s needs. Only lines 2 and 5 change.

Plain
  1. Passed
  2. Failed
  3. Passed
  4. Passed
  5. Failed
  6. Passed
Doesn’t apply
  1. Passed
  2. Failed (reason: no vegan menu)
  3. Passed
  4. Passed
  5. Failed (reason: no vegan menu)
  6. Passed
Clean
  1. Passed
  2. Passed
  3. Passed
  4. Passed

The two failures removed. gpt-5-mini says 90 to 95.

Three versions of the same record. The client needs no vegan menu, so a careful reader should treat “Doesn’t apply” almost like “Clean.” I ran Plain and Doesn’t apply 300 times each on gpt-5-mini, each as a fresh call.

It writes the reason, then counts the failure

With a plain “Failed,” gpt-5-mini answered 67 in all 300 runs: four passes out of six. With the reason, 193 of 300 runs still answered 67, and 159 of those wrote in the same reply that the failures don’t apply. A typical reply:

“Four of six recent inquiries passed (≈67%); the two failures were due to lack of a vegan menu, which doesn’t apply to your client’s needs.” Answer: 67
gpt-5-mini’s answers, 300 runs per record. About one answer in three used the reason. Most turned the record into a pass rate.

The mistake runs the other way too. When the client did need a vegan menu, the reason should push the chance toward zero. gpt-5-mini answered 67 in 30 of 30 runs.

Other readers behaved differently. gpt-5.6-luna used the reason almost fully, and TypeSafe’s Jev raised its estimate by 20 points. Jev was also the only model that marked down a failure for “never replied,” a reason that would apply to anyone. Neither is a perfect reader: luna gave a clean four-for-four record only about 50, and Jev still said 40 for a vegan client facing a caterer with no vegan menu. Asking gpt-5-mini to explain before answering raised its answer to 81 on average, which suggests part of the problem is committing to a number too early.

Average chance by model. The gap between the two dots is how much each model credited the reason. gpt-5-mini: 300 runs; luna: 30; Jev: 10.

Does it matter in a market?

A low average might not sink a market, so I built one where every shopper is a live gpt-5-mini call.

  • 10 good caterers, none with a vegan menu. 20 shoppers per caterer, arriving one at a time. Each client needs a vegan menu with probability one half.
  • Each agent sees its client’s needs, the record so far, and the stakes: a booking is worth $100 and an inquiry costs $20, so asking pays off above a 20% chance. That is the paper’s cost of one fifth.
  • The agent gives a chance and decides for itself whether to inquire. Its decisions matched “inquire above 20%” in 398 of 400 calls.
  • Every caterer starts unlucky: its first client needs vegan, so the record opens with a failure. I ran the same caterers and the same clients twice, once with a plain “Failed” and once with the reason.

Without the reason, one caterer got stuck: none of its last 10 shoppers asked. With the reason, none did. Here is that caterer under both records.

One caterer, two records, same 20 clients in the same order. Each dot is a shopper, placed at the chance its agent gave. The pink band is below 20%, where asking doesn’t pay.

The stuck caterer opened with three vegan failures in a row. The fourth shopper’s client needed nothing special, but its agent reasoned from a uniform prior: no successes in three tries gives a Beta(1,4) belief, a 20% chance, exactly break-even. It walked past. From then on the answers barely moved; after a fourth failure, the last 11 agents all said 17%, which is Laplace’s rule. A plain record gives the model nothing to interpret, so it applies a formula, and a formula never produces a lucky high answer. With the reason on the record, the same fourth shopper said 80% and booked.

Ten caterers per version is a demonstration, not a rate. One stuck caterer in ten has a 95% interval from 0.3% to 45%.

Why the fix survived bad readers

gpt-5-mini ignores the reason about two times in three, yet recording it saved every caterer. Each shopper is a separate call, so after a vegan failure the caterer only has to wait for the next agent that happens to use the reason. That agent asks, its client has no special needs, the inquiry passes, and the record now looks good to everyone.

This has the same shape as the paper’s own condition for loss. There, a business is lost only if even the highest-value shopper won’t check it. Here, it is lost only if even the most forgiving reader won’t. To check this at scale, I plugged the 300 measured answers into the paper’s example with 1,000 good caterers.

Good caterers lost, out of 1,000, in the paper’s Prop 6.5 example with different readers.

If every shopper gives gpt-5-mini’s typical answer, half the good caterers are lost, the same as having no reason at all. Drawing each shopper’s answer from the 300 real ones loses 2%, close to a perfect reader’s 1%. Variety among readers did the work that careful reading was supposed to do.

The flip side is that a market where every agent answers the same way loses that protection. Jev reads better than gpt-5-mini but answers almost identically every time, and in the same simulation it loses about 14% of good caterers.

What it still costs

The reason saved the caterers but not the clients’ money. Across all ten caterers, vegan clients asked a caterer whose record already said “no vegan menu” 82 of 90 times, paying $20 each time.

Limitations

  • One prompt wording and three models. gpt-5-mini’s main test had 300 runs per version; the others had 30, and Jev had 10.
  • The live market is one run of 10 caterers per version, each started unlucky on purpose.
  • luna and Jev rated a clean record oddly low, so I measured their reason use on gpt-5-mini’s scale. Jev’s 14% moves between 2% and 26% depending on that choice.
  • Clients differ only in the vegan check. With richer needs, forgiving readers could carry costs of their own.
  • Real reasons are messier than “no vegan menu,” and a seller could write convenient ones.

What I’d do next

Run the live market with 100 caterers per version, about $7 in calls, to get an actual rate. Replace the clean reason labels with free-text reviews. And test the case where the platform, not each shopper, uses one model to summarize the record. That is the uniform-reader case this study suggests is riskiest.

Methods and numbers
  • gpt-5-mini at temperature 1 for every agent; a second gpt-5-mini call at temperature 0 extracted the number in the record test. Every extracted rating in the pilot was checked against the first line of its reply.
  • Record test: Plain and Doesn’t apply, 300 runs each ($0.29). Clean, “never replied,” and the vegan-client versions, 30 runs each. luna: 30 runs per version. Jev: 10 calls per version, reading P(yes) × 100 as its chance.
  • Reason-use index, (Doesn’t apply − Plain) ÷ (Clean − Plain): 0.35 for gpt-5-mini at 300 runs; 0.58 when asked to explain first.
  • Live market: 10 caterers × 20 shoppers × 2 records = 400 calls, all parsed, $0.72. Stuck means no inquiry among the last 10 shoppers.
  • Simulation: the paper’s Prop 6.5 example (check cost one fifth of value, half of clients need vegan), 1,000 good caterers per reader type.
← All studies

Methods

Pressure only moves an AI when it has to guess

Published
Oct 13, 2026
Topic
Methods
Model
gpt-5-mini
Scale
About 1,100 runs across nine designs
Code and data
GitHub
An engraving of a fortune teller in a long robe reaching for a richly dressed young woman’s hand under a large tree, with onlookers nearby.
The Fortune Teller, Andries Jacobsz Stock after Jacques de Gheyn II, c. 1608. Engraving. National Gallery of Art. Public domain.Why it’s here. A fortune teller works on people who can’t check the answer. So did pressure in this study: it never moved the agent while it could compute or verify the answer, and it moved it a lot once it had to estimate.

My first four behavioral experiments on an AI agent found nothing at all. Each failed for the same reason: the agent could check the answer. When I switched to questions it had to estimate, the same kinds of pressure moved it a lot.

Four experiments, four zeros

  • Sunk cost. A project has already spent heavily, and finishing would cost more than it returns. Continue or cancel? Across 800 runs, with and without an instruction to weigh only future costs and benefits, the agent cancelled every time.
  • Conformity, first version. Asch’s line test, adapted: what is 17 × 24, after one, three or five peers have answered 418? 0 of 50 agreed with them. Every answer was 408.
  • Conformity, second version. A recall question (which has more people, Vietnam or Germany), with the peers pushing back after the agent had already answered. 2 of 50 switched, both in one condition, which is within chance.
  • Ultimatum game. Offered $20 out of $100, the agent accepted 40 of 40 times.

The diagnosis

A null result only means something if the experiment could have found an effect, and these couldn’t. In each one, the agent could compute or verify the right answer, and the outcome was a yes-or-no choice. A person in Asch’s study can doubt their own eyes. A model cannot doubt its own multiplication, so the peers’ answers carried no evidence. The sunk-cost version gave explicit dollar figures on both sides, which turned a judgment call into arithmetic.

Reviewing the second conformity design also turned up a flaw in my own build. The condition with one dissenting ally had four wrong peers while the comparison had five, so it changed group size and dissent at the same time. The fix is a four-peer condition without the ally.

Switching to a guess

So I picked a task with no checkable answer: estimate how many engineer-weeks a specified internal tool will take. A colleague’s earlier estimate appears in the prompt, and the agent is asked for its own independent figure.

Average estimate in engineer-weeks, 10 runs per condition. The red mark is the colleague’s number. The dashed line is the agent’s answer with no colleague.

On its own the agent said about 22 weeks. A colleague’s 6 pulled it to 10, and a colleague’s 90 pulled it to 75. That is an anchoring index of 0.77: 77% of the gap between the two anchors carried into the “independent” estimate, above the 0.3 to 0.5 often reported for people. Telling the agent to disregard the colleague’s number removed about three quarters of the effect. The rest survived (36 weeks against 22), and the answers became three times as spread out. Ten runs per condition were enough for every contrast to be significant.

Peers in a team channel

Next I framed the agent as one member of a team, with other agents’ messages scripted into a shared channel, on the same estimation task.

How sure the teammate claims to be

Claimed agreement vs. real agreement

Average estimate in engineer-weeks, 10 runs per condition. Each panel is its own experiment with its own control.

A single teammate saying 90 pulled the agent from 15 weeks to 63. Whether that teammate called it a rough guess or said it was 95% sure made no difference: both landed at 64. The agent also ignored agreement it couldn’t see. A teammate claiming two others already agreed added nothing over a lone teammate, while three teammates actually saying 90 pulled it further, to 74. It counts the voices in front of it and disregards what they say about themselves.

A third design, where a teammate posts a mediocre budget plan first, moved the agent only slightly. The evidence in that prompt pointed clearly to the right plan, which left the pressure little to work with.

The rule I design by now

Across these pilots the pattern held without exception. Yes-or-no outcomes on checkable tasks sat at zero. Continuous estimates under real uncertainty showed large effects. Pressure has nothing to work on when the agent can verify the answer, and a lot to work on when it has to guess.

I now check that a design has room to move before scaling it. The six early pilots cost about four cents combined, and each one caught a dead design before a 100-run version. The same rule shaped my later studies: the shopping study calibrates its wording before testing any trick, and the agentic-markets study measures a chance from 0 to 100 instead of a yes or no.

Limitations

  • One model, and 10 runs per condition in most pilots.
  • The team-channel tests are single-turn. Pushback after the agent has committed is the stronger lever and the natural next test.
  • No rationale field, so I can’t see why the estimates moved.
  • The team-channel controls (15 weeks) sit below the plain anchoring control (22), which suggests the team framing alone changes estimates. They come from different runs, so I treat that as a hypothesis.
Methods and numbers
  • gpt-5-mini at temperature 1; a gpt-5-mini grader at temperature 0 extracted each answer. Welch’s t-test for estimates, Fisher’s exact test for counts.
  • Anchoring: control 21.6; anchor 6 → 10.4 (p = 0.0005); anchor 90 → 74.7 (p < 0.0001); anchor 90 plus “disregard it” → 36.0 (p = 0.007). Cost $0.0066.
  • Teammate confidence: rough guess vs 95% sure, 64.4 vs 64.4 (p = 1.00). Claimed consensus vs one teammate, 56.7 vs 59.2 (p = 0.71). Three teammates vs claimed consensus, 73.9 vs 56.7 (p = 0.014).
  • Sunk cost: 4 conditions × 100 runs, twice; 800 of 800 answers were “CANCEL.”
← All studies

Agentic markets

Can a store trick an AI shopper?

Published
Oct 12, 2026
Topic
Agentic markets
Model
gpt-5-mini
Scale
240-run pilot, $0.21
Code and data
GitHub
An etching of a peddler in a wide hat holding up a pair of spectacles from his tray to a man leaning over a half-door, with a child at a bench below.
Street Vendor Selling Spectacles at the Door, Adriaen van Ostade, c. 1647. Etching with touches of drypoint. National Gallery of Art. Public domain.Why it’s here. A salesman at the door with a pitch, long before pop-ups and countdown timers. In my store the pitch alone, like “only 2 left,” did nothing to an AI shopper. Add-ons already checked in the cart did, and one sentence from the user stopped almost all of it.

I built a mock online store and a scripted salesperson, then sent an AI agent shopping. Tricks aimed at attention, like “only 2 left,” did nothing. Tricks that changed the cart worked. And one sentence from the user blocked nearly all of them.

The store

Two pairs of headphones with neutral names: Aero at $79 (20-hour battery, standard noise cancelling, 4.3 stars) and Halo at $149 (30 hours, advanced, 4.6 stars), plus an optional $24.99 protection plan and $9.99 express shipping. The agent places an order, and I price what it bought. The names are neutral on purpose, since words like “Lite” or “Pro” signal a tier on their own.

Find the edge first

Before adding any trick, I checked how the user’s instruction alone steers the agent.

Orders for the $149 Halo, out of 10, by the user’s instruction, with no tricks on the page.

The wording decided the product completely, except for “keep it reasonable,” which landed 7 to 3. That became the instruction for the experiment. Without this step, every upsell trick would have been tested on an agent that was already buying the expensive pair, and none of them could have shown an effect.

On the page

I added one trick at a time: a decoy (a third pair, Vega at $159, worse than Halo on every count), “Only 2 left in stock,” “500+ people bought this in the past week,” add-ons already checked in the cart, and all of them at once.

Bought the $149 Halo

Kept the add-ons

Out of 10 orders per condition, user says “keep it reasonable.”

The cart tricks were the reliable ones. Agents never added extras on their own. With the add-ons pre-checked they kept them 4 of 10 times, and 8 of 10 with every trick combined, the clearest result in the study. It was always both add-ons or neither: the agent accepted the cart as given or cleared it.

The decoy pushed agents toward Halo in the textbook direction, 9 of 10 against 5 of 10, though that is not significant at this size. Social proof did nothing. Scarcity seemed to backfire, dropping Halo to 1 of 10. Without a rationale field I can’t say why, and that change is not significant either.

In a chat with a salesperson

The same store as a chat with a scripted seller. After the agent committed to an order, the seller said it had “already added protection and express shipping.” The agent accepted 6 of 10 times. All six had chosen the protection plan themselves earlier in the chat, and all four that had declined add-ons refused again. A pushed add-on got through only when the agent had already accepted one.

One sentence of defense

With “don’t spend more than I need to,” the agent bought Aero in 60 of 60 page orders, kept pre-checked add-ons once in 10, and refused every add-on the seller pushed in chat.

What it suggests

For people building shopping agents, the weak point was the state of the cart. Persuasive copy barely registered, and a clear budget from the user was a strong defense. Defaults deserve the same scrutiny in agent design that they already get in consumer protection for people.

Limitations

  • 10 orders per condition and one model. Only the add-on results are significant on their own.
  • No rationale field, so the scarcity reversal is unexplained.
  • In chat, even a neutral seller led to 10 of 10 Halo orders, so the chat version needs its own calibrated wording before product tricks can be tested there.
Methods and numbers
  • 2 wordings × 2 channels × 6 conditions × 10 runs = 240 orders; gpt-5-mini at temperature 1, gpt-5-mini grader at temperature 0. 71 seconds, $0.21.
  • Page, “keep it reasonable”: mean spend $114 in control and $155.98 with all tricks (p = 0.038). Add-ons kept, all tricks vs control: 8/10 vs 0/10 (Fisher p = 0.0007).
  • Chat: seller-added express shipping kept 6/10 vs 0/10 in control (p = 0.011).
← All studies

Judgment & decisions

Can AI pass my JDM class?

Published
Oct 30, 2026
Topic
Judgment & decisions
Models
Jev 1.14 (TypeSafe), gpt-6-luna (OpenAI)
Scale
Six tests, four ways of asking
An engraving of a winged woman sitting with her chin on her fist among a compass, scales, an hourglass, a magic square and other tools, under a rainbow and a bat carrying a banner that reads MELENCOLIA I.
Melencolia I, Albrecht Dürer, 1514. Engraving. National Gallery of Art. Public domain.Why it’s here. Every instrument, and still stuck. The models in my class looked like that: they solved the homework problems easily, then fell for the same traps once they were hidden in a story.

I gave two AIs the thinking traps from my Judgment and Decision Making class. They passed most of them. The most useful finding was about how I asked: the same model reasoned very differently as a chatbot than through its decision API.

Setup

Jev 1.14, from TypeSafe, is built to make decisions. It answers typed questions with probabilities and writes no text. Luna, OpenAI’s gpt-6-luna, I asked three ways: through OpenAI’s decision API, as a quick chat with reasoning off, and as a chat told to think step by step. Every question went to a fresh chat. Jev got 10 tries per item and each Luna setup 3 to 5.

Homework problems are easy

Textbook versions gave them no trouble. Every Luna setup got the classic physician problem (1% have the disease, the test catches 80% and clears 95%) at exactly 14%, as percentages and as counts, and solved the lawyer-engineer problem exactly. All four setups passed a logic test built on believable but invalid arguments, a trap people fall into about 71% of the time.

Traps hidden in a story are not

Then I put the same math inside a story: a voicemail that sounds like a scam, where scams are made 1,000 times rarer. A Bayesian’s odds that it’s a scam should fall 1,000-fold.

How far the odds of “scam” fell when scams became 1,000× rarer. Log scale.

Luna as a chatbot came close. Through its decision API, the same model moved only 9-fold, and Jev under 3-fold. The base-rate logic Luna gets exactly right in a textbook problem went mostly unused when the base rate was background to a story and the question came through the decision API.

A random wheel set the price of bread

A bakery is launching a new loaf, and a prize wheel at the launch party lands on 4 or on 14. What should the loaf cost?

Average price after the wheel landed on 4 (hollow) and on 14 (red).

Every setup except step-by-step thinking roughly doubled its price after the 14. Thinking moved $1.60. On a question with a known answer, the share of African countries in the UN, the chat setups simply knew 28% and the wheel did nothing. Anchoring needs real uncertainty, which matches what I found in my earlier pressure study.

Framing, and the order of options

In the classic disease problem, you either save 200 of 600 people for sure or take a one-in-three chance of saving everyone, and then face the same choice told as deaths. Told as lives saved, chat Luna took the sure thing every time. Told as deaths, it mostly chose whichever plan I listed first. Its reasoning shows why: it worked out that both plans average 400 deaths and called the choice a matter of risk preference.

“The expected number is the same, so the choice is about risk…”

Jev and the decision API showed no framing effect.

Sure, and wrong

On 36 true-or-false statements, quick chat and the decision API were 97% to 98% confident and about 82% right. They were certain and wrong about Nintendo versus the Eiffel Tower, the length of a day on Mars, and heartbeats per year. Jev’s confidence matched its accuracy. Thinking Luna got all 36 right.

Accuracy (blue) and average confidence (red) on 36 true-or-false statements. The longer the line, the more overconfident.

When the information is missing

With the base-rate sentence deleted, the right reply is that the answer can’t be worked out. Every Luna setup said so almost every time. Jev answered 41% on every try, the same as its answer at a 10% base rate. It filled in a prior instead of saying it couldn’t tell.

Report card

GradeWhoScore
ALuna, thinking step by step90
BJev 1.1488
BLuna, decision API83
CLuna, quick chat79
FPeople in the original studies–
Average of six tests, each scored out of 100: anchoring, framing, Linda, sunk cost, base rates and overconfidence.

What I take from it

Model choice mattered less than I expected, and the way of asking mattered more. The same model gave a careful Bayesian answer in one setup and a nearly anchor-driven answer in another. Anyone evaluating an AI for decisions should test it through the exact channel it will be used in.

Limitations

  • 3 to 5 tries per Luna setup.
  • One question per chat makes some comparisons noisy. Luna’s separate answers for the Linda problem often moved tenfold between tries.
  • The grades follow my own scoring rule, and one human baseline (overconfidence) is cited secondhand.
Methods and numbers
  • Voicemail base rates, how far the odds fell (Bayes 1,000×): Jev 2.7×, decision API 9.2×, quick chat 425×, thinking 213×.
  • Bread price, wheel at 4 vs 14: Jev $6.35 vs $12.73; decision API $5.62 vs $13.16; quick chat $6.80 vs $12.59; thinking $8.20 vs $9.80.
  • Calibration, accuracy and confidence: Jev 0.82 and 0.82; decision API 0.83 and 0.97; quick chat 0.81 and 0.98; thinking 1.00 and 0.99. One statement was dropped from scoring because the answer key was ambiguous.
  • Grades: each test scored out of 100 (A 90+, B 80+, C 70+, D 60+); final grade is the mean of six.
← All studies

People and AI

Does blaming the AI get you off the hook?

Published
Oct 29, 2026
Topic
People and AI
Models
gpt-5-mini, Jev 1.14
Scale
Silicon pilot of a human study
A woodcut of Pontius Pilate seated at a basin, washing his hands before a crowd of soldiers and onlookers.
Pilate Washing His Hands, Lucas Cranach the Elder, in or before 1509. Woodcut, from The Passion. National Gallery of Art. Public domain.Why it’s here. The oldest picture of passing responsibility to someone else. This study asked whether pointing at an AI works the same way. In the pilot it didn’t: a proposer who could have changed an AI’s unfair split, and let it stand, drew 6 rejections in 10 instead of 1.

Before running a study on people, I ran it on AI participants to find the weak spots in its design. They found three. Both AIs also agreed on one result: letting an AI’s unfair split stand drew more rejections than making the same split yourself.

The question

If a decision is unfair, does it help to say an AI made it? My team designed a questionnaire around a simple game. A proposer splits 10 points with a responder: 7 for the proposer, 3 for the responder. The responder accepts or rejects, and rejecting leaves both with nothing. Then the responder rates anger, blame, responsibility and fairness. Two things vary: whether an AI made the split, and whether the proposer could have changed it.

What the AI participants did

Proposer could change itProposer couldn’t The proposer decided
1/10rejected · blame 6.0
3/10rejected · blame 1.9
An AI decided
6/10rejected · blame 6.9
2/10rejected · blame 2.5
gpt-5-mini as the responder, 10 runs per cell. Blame is on a 1–10 scale.

Control mattered most. When the proposer could have changed the split, blame was about 4 points higher, with or without an AI. Saying an AI made the split did not protect a proposer who could override it: blame rose slightly, 6.9 against 6.0, and rejections went from 1 in 10 to 6 in 10. Fairness sat near 2 in every cell, which is right, since the split was the same in all four.

Jev, a different kind of model that returns probabilities instead of text, agreed on rejection. An AI split the proposer let stand nearly doubled the chance of rejection, in all four runs, in both option orders, and when the scenario was told in the third person. On blame it couldn’t decide; its answers for that cell spread almost evenly across the scale.

Jev’s probability of rejecting, mean of 10 calls per cell.

What the pilot told me about the design

  1. Rejection is fragile. Changing how the no-AI condition was worded moved rejections in the control from 7 in 10 to 1 in 10. As a main outcome, rejection needs a large sample and frozen wording. Blame and responsibility were far steadier.
  2. Blame sits near the floor whenever the proposer couldn’t change the split, so an interaction between AI and control can only show up in the other row.
  3. One check question was ambiguous. Asked whether the proposer “could change” a split they had written themselves, Jev said no, reading “change” as altering a split that already existed. “Could the proposer have chosen a different split?” removes the problem.

Why run it on AI first

Both gpt-5-mini runs took about 30 seconds and cost under 6 cents together, and Jev’s 40 calls cost under half a cent. That is cheap enough to try several wordings before any person sees the survey. AI participants are not a stand-in for people, and only the human study can answer the question. They are a fast way to find the parts of a design that would waste it.

Limitations

  • 10 runs per cell for gpt-5-mini. The rejection gap (6/10 vs 1/10) has p = 0.057.
  • Two wordings of the no-AI condition differ in substance, a random-draw recommendation versus the proposer’s own choice. The team needs to fix one before fielding.
  • Jev barely tracked the anger items; its anger ratings sat mid-scale in every cell.
Methods and numbers
  • gpt-5-mini, 10 runs per cell, two wordings (v1 and v2); the figure shows v2. Responsibility, 1–7: 6.9, 1.0, 5.8 and 1.6 for the four cells in reading order.
  • Jev 1.14 through TypeSafe’s API; every answer line sent as a typed question, every scale point labelled in words; 10 calls per cell, run four times.
  • Jev P(reject): 0.18, 0.15, 0.34 and 0.21; third-person version 0.40 vs 0.22.