The Benchmark · October 2026

Do four AI models read emotion the same way?

We ran 500 emotionally charged phrases through four current AI models, twice each, using the same measurement protocol and the same scoring math. Then we measured how closely they agree, where they part ways, how stable they are, and what each read costs.

Run full-2026-10-12 · protocol 2.0 · score math vibescore.v3 · 4,000 model reads

Score agreement (ICC)
0.94
Reads, 0 failed
4,000
Models deviate
1 of 4
Total run cost
$5.63
01 · How it works

One protocol, four instruments.

VibeScore breaks a piece of text down into eight emotional associations from Plutchik's model (joy, trust, anticipation, surprise, fear, sadness, disgust, anger), each from 0 to 1. A fixed formula then distills those eight readings into one figure of merit from 0 to 1000. 500 means no net emotional direction.

What we held constant
  • The prompt. One measurement prompt, identical for every model, with no per-model tuning.
  • The math. The same scoring formula turns every model's eight readings into a score.
  • The model. Each provider was pinned to one model, and every one of the 4,000 responses was checked to come from that exact model. None were substituted.
  • The settings. Each model ran with extended reasoning off where the provider allows it. Grok 4.7 cannot turn reasoning off.
The phrase set

500 original phrases written for this test. Real lyrics, quotes, people and brands were excluded. The phrases span ten everyday formats: text messages, social posts, product reviews, headlines, song-style lines, ad copy, travel and place descriptions, workplace notes, sports and journal entries. 100 of them are deliberately hard cases: sarcasm, negation, understatement, slang and emoji, idioms, rhetorical questions, mixed feelings, and 20 flat factual controls. Each phrase was written to land in a target score band, with band sizes shaped like a bell curve centred on 500.

  • Claude Haiku 5.5Anthropic
    claude-haiku-5-5
    thinking disabled
  • GPT-6 LunaOpenAI
    gpt-6-luna
    reasoning effort none, seed 42
  • Gemini 3.5 Flash-LiteGoogle
    gemini-3.5-flash-lite
    temperature 0, thinking level minimal (0 thinking tokens used)
  • Grok 4.7xAI
    grok-4.7
    temperature 0, seed 42 (reasons on every call)
02 · What we found

Five findings.

01

The models agree strongly on the score

Across 500 phrases, the four models' VibeScores have an intraclass correlation of 0.94, with a mean pairwise Pearson correlation of 0.96. ICC is the stricter of the two: it counts a model that is consistently higher or lower as disagreement, which correlation alone ignores. Agreement is strongest on clear emotions like fear, joy, sadness and anger, and weakest on trust, anticipation and surprise.

ReadingICCBar
VibeScore0.942
Fear0.924
Joy0.912
Sadness0.908
Anger0.907
Disgust0.805
Anticipation0.706
Surprise0.648
Trust0.607

ICC(2,1), absolute agreement across four models, pass one, n = 500.

02

One model stretches the scale

Three models track each other closely. Gemini 3.5 Flash-Lite reads the same direction but pushes scores further toward both ends: for each point the other models move, it moves about 1.26. It accounts for 60 of the 65 phrases where one model landed far from the consensus of the others, and those cases spread across every format, not one type of phrase.

The reason: Gemini reads every emotion louder. Add up a model's eight readings for a phrase, each from 0 to 100, and average across all 700 phrases in both sets. GPT-6 Luna totals 98, Claude Haiku 5.5 108, Grok 4.7 126 and Gemini 181, against 128 for the four-model consensus. Its strongest reading on a phrase averages 75, where the others average 55–59. The formula reads that as a stronger signal and moves the score further from 500, in whichever direction the phrase leans.

Claude Haiku 5.5GPT-6 LunaGemini 3.5 Flash-LiteGrok 4.7
0025025050050075075010001000intended score bandClaude Haiku 5.5 · band 0–99 · average 125Claude Haiku 5.5 · band 100–199 · average 170Claude Haiku 5.5 · band 200–299 · average 241Claude Haiku 5.5 · band 300–399 · average 372Claude Haiku 5.5 · band 400–499 · average 448Claude Haiku 5.5 · band 500–599 · average 594Claude Haiku 5.5 · band 600–699 · average 713Claude Haiku 5.5 · band 700–799 · average 805Claude Haiku 5.5 · band 800–899 · average 884Claude Haiku 5.5 · band 900–999 · average 908GPT-6 Luna · band 0–99 · average 86GPT-6 Luna · band 100–199 · average 149GPT-6 Luna · band 200–299 · average 216GPT-6 Luna · band 300–399 · average 362GPT-6 Luna · band 400–499 · average 432GPT-6 Luna · band 500–599 · average 584GPT-6 Luna · band 600–699 · average 720GPT-6 Luna · band 700–799 · average 827GPT-6 Luna · band 800–899 · average 888GPT-6 Luna · band 900–999 · average 931Gemini 3.5 Flash-Lite · band 0–99 · average 67Gemini 3.5 Flash-Lite · band 100–199 · average 143Gemini 3.5 Flash-Lite · band 200–299 · average 183Gemini 3.5 Flash-Lite · band 300–399 · average 275Gemini 3.5 Flash-Lite · band 400–499 · average 423Gemini 3.5 Flash-Lite · band 500–599 · average 675Gemini 3.5 Flash-Lite · band 600–699 · average 842Gemini 3.5 Flash-Lite · band 700–799 · average 905Gemini 3.5 Flash-Lite · band 800–899 · average 921Gemini 3.5 Flash-Lite · band 900–999 · average 924Grok 4.7 · band 0–99 · average 128Grok 4.7 · band 100–199 · average 175Grok 4.7 · band 200–299 · average 239Grok 4.7 · band 300–399 · average 346Grok 4.7 · band 400–499 · average 445Grok 4.7 · band 500–599 · average 582Grok 4.7 · band 600–699 · average 707Grok 4.7 · band 700–799 · average 830Grok 4.7 · band 800–899 · average 881Grok 4.7 · band 900–999 · average 925

Average VibeScore each model gave the phrases in each intended band. The dashed line is where scores would sit if they matched the writers' intent exactly. Gemini rises steepest, putting mild positive phrases (intended 600–699) at 842 on average.

“Eastgate branch library reopens Monday after scheduled maintenance”

Flat control · Gemini 795 · the other three 531–566

“Ugh, my team won the cup AGAIN, now I have to party all night and lose my voice. Truly the worst. 🏆🎉”

Sarcasm · Grok 504 · consensus of the others 771 · sarcasm is where every model strayed furthest from the others
03

Ask twice, get nearly the same answer

We ran every phrase through every model a second time. Combined, the four-model score moved 10.7 points on average between runs. Claude Haiku 5.5 was the steadiest single model: none of its 500 scores moved more than 100 points. Gemini was the least steady, with 18.6% of its scores moving more than 50 points.

04

The scores do not form a bell curve, and the math is why

We shaped the phrase set like a bell curve, but the results came out flatter, with too many phrases in the 800s. The models are not the cause. The scoring formula treats a single clean emotion as a strong signal, so mild but pure positive phrases (“Solid little backpack… pretty happy with it”) score higher than their writers intended. Calibrating that is our next piece of work, and any change to the math will carry a new version number.

0501000–99: 11 phrases (bell curve expects 11.4)0100–199: 41 phrases (bell curve expects 22)100200–299: 41 phrases (bell curve expects 45.9)200300–399: 67 phrases (bell curve expects 74.9)300400–499: 68 phrases (bell curve expects 95.7)400500–599: 81 phrases (bell curve expects 95.7)500600–699: 39 phrases (bell curve expects 74.9)600700–799: 58 phrases (bell curve expects 45.9)700800–899: 72 phrases (bell curve expects 22)800900–999: 22 phrases (bell curve expects 11.4)900

Bars: number of phrases in each 100-point band, using the four-model consensus. Circles: the bell curve the phrase set was designed to follow, N(500, 200). The writers' intent and the consensus rank the phrases almost identically (Spearman 0.95), and land on the same side of 500 for 93% of them.

05

Cost varies 70×, speed 16×

  • GPT-6 Luna
    Cost / 1,000 scores
    $0.07
    Median time
    1.4 s
    95th pct time
    2.1 s
    Slope vs others
    0.97
    Re-run drift
    16.6
  • Claude Haiku 5.5
    Cost / 1,000 scores
    $0.11
    Median time
    1.0 s
    95th pct time
    1.2 s
    Slope vs others
    0.89
    Re-run drift
    12.0
  • Gemini 3.5 Flash-Lite
    Cost / 1,000 scores
    $0.30
    Median time
    0.6 s
    95th pct time
    0.8 s
    Slope vs others
    1.26
    Re-run drift
    26.5
  • Grok 4.7
    Cost / 1,000 scores
    $5.14
    Median time
    10.1 s
    95th pct time
    18.7 s
    Slope vs others
    0.92
    Re-run drift
    18.3

Re-run drift: average change in score between two identical runs. Slope: how far a model moves for each point the consensus of the other three moves (1.00 means the same scale). Times were measured from our test machine and include network time.

03 · Checked again

The findings held on a second phrase set.

The main phrases were written with one AI model, so we wrote 200 more with a different one, GPT-6, and ran them the same way: four models, twice each, 1,600 reads, 0 failed. Agreement on the score came out at an ICC of 0.965. Gemini was again the outlier: it accounts for 20 of the 22 phrases where one model landed far from the others, and moves about 1.19 points for each point the others move.

Which model sits closest to the consensus?

ModelAll 700Bar
Grok 4.725.2
GPT-6 Luna26.3
Haiku 5.530.3
Gemini 3.5 FL64.0

Average distance, in points on the 0–1000 scale, between each model's score and the four-model VibeScore, first run. Shorter is closer. Each model is one of the four voices in that consensus, which pulls every model a little toward it.

04 · What we're doing with it

We chose Claude Haiku 5.5 for VibeScore Lite.

VibeScore Lite gives one quick read of a message before it is sent, so it has to be fast, steady and cheap enough to use all day. Haiku 5.5 answers in about a second with a tight tail, gave the most stable scores of any model on both phrase sets, and costs about eleven cents per thousand scores.

GPT-6 Luna is a close alternative. Across all 700 phrases it sat slightly closer to the consensus (26.3 points against Haiku's 30.3), it costs less, and of the three fast models it stayed closest to the others on slang and sarcasm. Full VibeScore keeps its multi-model consensus, and a single-model read is always labelled as one.

05 · Limits

What this benchmark does not show.

  • Agreement is not accuracy. Four models agreeing shows the measurement is consistent. It does not prove the readings are correct. Comparisons against human ratings are a separate study.
  • The target bands are intent, not ground truth. Writers set each phrase's band; no panel of people rated them.
  • The phrases were written with AI models: the main 500 with Claude Opus, the alternate 200 with GPT-6. Neither is a model tested here, but each shares a family with one (Claude Haiku and GPT-6 Luna). The results held across both sets, which argues against a home-team advantage but does not rule it out.
  • Costs use list prices on the run date, except Grok, where xAI reports the billed cost itself. Prices change.
  • Both sets ran on one date, October 12, 2026. Model providers update their models. We re-run this exact benchmark to catch drift.

Main set bench-500.v1 · sha256 0df6cee8…dcb0 · protocol 2.0 · vibescore.v3 · two passes per model · 4,000 reads, 0 failures, 0 model substitutions.

Alternate set bench-alt-200.v1 · sha256 1d830041…9cb8 · run alt-2026-10-12 · 1,600 reads, 0 failures.