Sub-4GB
6 models tested[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 149.0 tok · 2.27s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 895.8 tok · 4.73s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 591.2 tok · 3.26s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 386.0 tok · 2.24s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 579.2 tok · 3.2s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 269.2 tok · 1.66s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 272.0 tok · 3.77s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1018.0 tok · 7.54s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 575.4 tok · 4.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 324.0 tok · 2.66s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 562.0 tok · 4.31s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 585.6 tok · 4.47s
30/30
llama3.2:3b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 115.0 tok · 1.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 152.0 tok · 0.72s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 238.0 tok · 1.08s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 135.8 tok · 0.7s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 249.2 tok · 1.11s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 152.0 tok · 0.75s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 427.8 tok · 2.22s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1149.2 tok · 4.82s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 852.2 tok · 3.61s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 822.0 tok · 3.46s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 1328.0 tok · 5.57s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 1/5
· 974.6 tok · 4.17s
26/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 65.0 tok · 0.71s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 175.8 tok · 0.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 0/5
· 24.0 tok · 0.21s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 175.8 tok · 0.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 275.0 tok · 1.38s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 0/5
· 115.2 tok · 0.62s
20/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 46.0 tok · 0.81s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 0/5
· 232.2 tok · 1.26s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 137.0 tok · 0.88s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 113.0 tok · 0.69s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 169.4 tok · 0.95s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 0/5
· 113.0 tok · 0.68s
20/30
4-8GB
4 models tested
qwen3:8b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 383.6 tok · 3.53s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1356.8 tok · 9.63s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 779.0 tok · 5.56s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 1272.0 tok · 9.04s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 2112.2 tok · 15.03s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 1017.2 tok · 7.23s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 1850.0 tok · 16.07s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 0/5
· 4096.0 tok · 32.79s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 1007.0 tok · 8.22s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 1037.0 tok · 8.63s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 1611.0 tok · 13.11s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 945.0 tok · 7.83s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 78.0 tok · 0.99s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 370.0 tok · 2.34s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 193.0 tok · 1.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 224.0 tok · 1.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 310.0 tok · 1.95s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 0/5
· 143.0 tok · 0.93s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 80.0 tok · 1.13s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 465.0 tok · 3.16s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 156.6 tok · 1.17s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 10.0 tok · 0.2s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 437.4 tok · 2.98s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 0/5
· 381.0 tok · 2.64s
25/30
8-12GB
4 models tested[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 299.0 tok · 4.83s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 908.4 tok · 10.56s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 522.2 tok · 6.38s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 445.4 tok · 5.5s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 550.6 tok · 6.66s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 474.4 tok · 5.82s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 295.4 tok · 4.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1192.2 tok · 14.65s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 518.6 tok · 6.69s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 297.8 tok · 3.97s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 635.0 tok · 10.04s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 503.0 tok · 6.37s
30/30
deepseek-r1:14b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 237.0 tok · 5.02s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 513.6 tok · 6.29s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 877.0 tok · 10.71s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 430.0 tok · 5.42s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 420.8 tok · 5.34s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 472.8 tok · 6.0s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 327.0 tok · 4.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1890.4 tok · 22.69s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 911.0 tok · 11.0s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 1375.0 tok · 16.54s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 2202.2 tok · 26.6s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 2088.6 tok · 25.14s
30/30
12GB+
3 models tested
mistral-small3.2:24b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 78.0 tok · 3.62s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 264.0 tok · 5.0s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 156.0 tok · 2.9s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 165.0 tok · 3.05s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 201.6 tok · 3.72s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 183.8 tok · 3.4s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 298.0 tok · 3.95s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1254.2 tok · 9.96s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 530.4 tok · 4.5s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 532.0 tok · 4.52s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 754.6 tok · 6.3s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 5/5
· 451.0 tok · 3.84s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 331.8 tok · 10.76s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — multi-step: 5/5
· 1417.6 tok · 35.75s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — logical deduction: 5/5
· 688.6 tok · 17.35s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — adversarial (CRT trap): 5/5
· 923.2 tok · 23.05s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — combinatorics: 5/5
· 1371.2 tok · 34.17s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — distractor filtering: 2/5
· 2806.0 tok · 76.01s
27/30