Navers lab
← PPTBench

The leaderboard

Every configuration on the same 500 tasks, with the score, hard-gate pass rate, conditional detail score, cost and token count kept together.

All configurations

PPTBench model leaderboard
# Model Harness Effort Score Gate pass Detail (conditional) Generation cost Avg tokens
1 GPT-6-Astra Codex High 77.34 80.8% 95.7 $320.69 156,884
2 GPT-6-Astra Codex XHigh 72.60 74.8% 97.1 $966.82 529,559
3 GPT-6-Astra Codex Max 68.76 71.0% 96.8 $1,598.36 849,174
4 GPT-6-Astra Codex Medium 68.16 71.6% 95.2 $207.78 95,222
5 Kimi K3 OpenCode High 67.80 73.8% 91.9 $1,158 3,486,485
6 GPT-6-Astra Codex Low 59.42 63.0% 94.3 $178.64 82,298
7 GPT-5.6 Sol Codex Max 49.28 52.6% 93.7 $988 1,370,852
8 Qwen 3.8 Max Claude Code XHigh 47.69 53.2% 89.6 $307 1,067,521
9 GPT-5.6 Sol Codex XHigh 36.24 40.0% 90.6 $495 531,671
10 Claude Opus 5 Claude Code XHigh 35.91 43.6% 82.4 $685 896,363
11 Claude Opus 5 Claude Code Max 33.54 40.8% 82.2 $719 1,001,465
12 GPT-5.6 Terra Codex Max 33.46 37.4% 89.5 $283 565,341
13 Claude Opus 5 Claude Code High 33.12 40.8% 81.2 $365 511,632
14 GLM-5.3-Flash OpenCode Max 32.42 36.8% 88.1 $52.53 2,381,120
15 GPT-5.6 Luna Codex Max 31.02 36.4% 85.2 $166 1,056,879
16 GPT-5.6 Sol Codex High 28.28 31.8% 88.9 $285 254,199
17 GPT-5.6 Terra Codex XHigh 25.84 29.6% 87.3 $125 207,341
18 GPT-5.6 Luna Codex XHigh 24.63 29.0% 84.9 $87 410,524
19 Claude Opus 5 Claude Code Medium 24.13 31.2% 77.3 $132 159,533
20 GPT-5.6 Sol Codex Medium 22.20 25.4% 87.4 $182 146,793
21 GPT-5.6 Luna Codex High 19.47 23.8% 81.8 $57 255,343
22 Claude Opus 5 Claude Code Low 17.90 23.6% 75.8 $73 87,561
23 GPT-5.6 Terra Codex High 16.89 19.4% 87.1 $81 152,878
24 GPT-5.6 Sol Codex Low 11.12 13.0% 85.5 $123 93,026
25 GPT-5.6 Terra Codex Medium 10.10 12.0% 84.2 $62 111,330
26 GPT-5.5 Codex Medium 9.64 11.4% 84.6 $262 243,908
27 Claude Sonnet 5 Claude Code XHigh 8.98 12.8% 70.2 $214 895,365
28 Claude Sonnet 5 Claude Code High 8.76 12.4% 70.6 $137 572,255
29 GPT-5.6 Terra Codex Low 8.60 9.8% 87.8 $58 119,274
30 GPT-5.6 Luna Codex Medium 8.15 10.0% 81.5 $31 131,364
31 Claude Sonnet 5 Claude Code Medium 5.99 9.2% 65.1 $99 371,068
32 GPT-5.6 Sol Codex None 4.99 5.8% 86.0 $117 83,655
33 Claude Sonnet 5 Claude Code Low 4.03 6.4% 63.0 $61 194,207
34 GPT-5.6 Terra Codex None 3.55 4.2% 84.5 $45 70,522
35 GPT-5.6 Luna Codex Low 3.03 3.6% 84.2 $21 76,986
36 GPT-5.6 Luna Codex None 1.53 1.8% 85.0 $22 77,709

Cost and token trade-offs

Each line is a model family and each point is one reasoning effort. The horizontal axes are logarithmic so lower-cost configurations remain legible alongside runs that use one or two orders of magnitude more compute. Only configurations with published accounting appear in these charts.

Cost–quality frontier
0 20 40 60 80 100 $10 $30 $100 $300 $1k total generation cost (USD, log scale) mean score GPT-6-Astra · Low: 59.42 ($178.637) GPT-6-Astra · Medium: 68.16 ($207.779) GPT-6-Astra · High: 77.34 ($320.685) GPT-6-Astra · XHigh: 72.60 ($966.824) GPT-6-Astra · Max: 68.76 ($1,598.362) GPT-6-Astra GPT-5.6 Sol · None: 4.99 ($117) GPT-5.6 Sol · Low: 11.12 ($123) GPT-5.6 Sol · Medium: 22.20 ($182) GPT-5.6 Sol · High: 28.28 ($285) GPT-5.6 Sol · XHigh: 36.24 ($495) GPT-5.6 Sol · Max: 49.28 ($988) GPT-5.6 Sol GPT-5.6 Terra · None: 3.55 ($45) GPT-5.6 Terra · Low: 8.60 ($58) GPT-5.6 Terra · Medium: 10.10 ($62) GPT-5.6 Terra · High: 16.89 ($81) GPT-5.6 Terra · XHigh: 25.84 ($125) GPT-5.6 Terra · Max: 33.46 ($283) GPT-5.6 Terra GPT-5.6 Luna · Low: 3.03 ($21) GPT-5.6 Luna · None: 1.53 ($22) GPT-5.6 Luna · Medium: 8.15 ($31) GPT-5.6 Luna · High: 19.47 ($57) GPT-5.6 Luna · XHigh: 24.63 ($87) GPT-5.6 Luna · Max: 31.02 ($166) GPT-5.6 Luna GPT-5.5 · Medium: 9.64 ($262) GPT-5.5 Claude Opus 5 · Low: 17.90 ($73) Claude Opus 5 · Medium: 24.13 ($132) Claude Opus 5 · High: 33.12 ($365) Claude Opus 5 · XHigh: 35.91 ($685) Claude Opus 5 · Max: 33.54 ($719) Claude Opus 5 Claude Sonnet 5 · Low: 4.03 ($61) Claude Sonnet 5 · Medium: 5.99 ($99) Claude Sonnet 5 · High: 8.76 ($137) Claude Sonnet 5 · XHigh: 8.98 ($214) Claude Sonnet 5 GLM-5.3-Flash · Max: 32.42 ($52.526) GLM-5.3-Flash Qwen 3.8 Max · XHigh: 47.69 ($307) Qwen 3.8 Max Kimi K3 · High: 67.80 ($1,158) Kimi K3
Token–quality frontier
0 20 40 60 80 100 50k 100k 300k 1M 3M average generation tokens per task (log scale) mean score GPT-6-Astra · Low: 59.42 (82k tokens) GPT-6-Astra · Medium: 68.16 (95k tokens) GPT-6-Astra · High: 77.34 (157k tokens) GPT-6-Astra · XHigh: 72.60 (530k tokens) GPT-6-Astra · Max: 68.76 (849k tokens) GPT-6-Astra GPT-5.6 Sol · None: 4.99 (84k tokens) GPT-5.6 Sol · Low: 11.12 (93k tokens) GPT-5.6 Sol · Medium: 22.20 (147k tokens) GPT-5.6 Sol · High: 28.28 (254k tokens) GPT-5.6 Sol · XHigh: 36.24 (532k tokens) GPT-5.6 Sol · Max: 49.28 (1371k tokens) GPT-5.6 Sol GPT-5.6 Terra · None: 3.55 (71k tokens) GPT-5.6 Terra · Medium: 10.10 (111k tokens) GPT-5.6 Terra · Low: 8.60 (119k tokens) GPT-5.6 Terra · High: 16.89 (153k tokens) GPT-5.6 Terra · XHigh: 25.84 (207k tokens) GPT-5.6 Terra · Max: 33.46 (565k tokens) GPT-5.6 Terra GPT-5.6 Luna · Low: 3.03 (77k tokens) GPT-5.6 Luna · None: 1.53 (78k tokens) GPT-5.6 Luna · Medium: 8.15 (131k tokens) GPT-5.6 Luna · High: 19.47 (255k tokens) GPT-5.6 Luna · XHigh: 24.63 (411k tokens) GPT-5.6 Luna · Max: 31.02 (1057k tokens) GPT-5.6 Luna GPT-5.5 · Medium: 9.64 (244k tokens) GPT-5.5 Claude Opus 5 · Low: 17.90 (88k tokens) Claude Opus 5 · Medium: 24.13 (160k tokens) Claude Opus 5 · High: 33.12 (512k tokens) Claude Opus 5 · XHigh: 35.91 (896k tokens) Claude Opus 5 · Max: 33.54 (1001k tokens) Claude Opus 5 Claude Sonnet 5 · Low: 4.03 (194k tokens) Claude Sonnet 5 · Medium: 5.99 (371k tokens) Claude Sonnet 5 · High: 8.76 (572k tokens) Claude Sonnet 5 · XHigh: 8.98 (895k tokens) Claude Sonnet 5 GLM-5.3-Flash · Max: 32.42 (2381k tokens) GLM-5.3-Flash Qwen 3.8 Max · XHigh: 47.69 (1068k tokens) Qwen 3.8 Max Kimi K3 · High: 67.80 (3486k tokens) Kimi K3