Sub-4GB
6 models tested
gemma4:e2b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 132.4 tok · 2.2s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 248.0 tok · 1.56s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 316.0 tok · 1.88s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 5/5
· 481.6 tok · 2.72s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 392.0 tok · 2.28s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 202.0 tok · 1.32s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 227.0 tok · 3.76s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 305.2 tok · 2.54s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 56.4 tok · 0.73s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 5/5
· 381.4 tok · 3.05s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 43.0 tok · 0.63s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 223.0 tok · 1.96s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 135.2 tok · 2.02s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 458.8 tok · 2.01s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 315.0 tok · 1.45s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 5/5
· 3752.4 tok · 15.84s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 306.0 tok · 1.44s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 208.0 tok · 1.03s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 11.0 tok · 0.48s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 16.0 tok · 0.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 19.0 tok · 0.18s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 43.0 tok · 0.3s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 37.0 tok · 0.28s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 54.0 tok · 0.38s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 65.0 tok · 0.78s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 47.0 tok · 0.34s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 35.0 tok · 0.29s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 98.0 tok · 0.55s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 37.0 tok · 0.3s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 17.0 tok · 0.24s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 11.0 tok · 1.6s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 48.6 tok · 0.39s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 20.0 tok · 0.26s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 134.4 tok · 0.78s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 39.0 tok · 0.34s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 17.0 tok · 0.26s
25/30
4-8GB
4 models tested[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 133.0 tok · 1.75s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 198.8 tok · 1.51s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 236.6 tok · 1.77s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 1676.0 tok · 11.96s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 425.8 tok · 3.09s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 222.6 tok · 1.71s
25/30
mistral:7b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 16.0 tok · 0.56s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 35.0 tok · 0.25s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 24.0 tok · 0.2s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 47.0 tok · 0.34s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 46.0 tok · 0.33s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 21.0 tok · 0.21s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 13.0 tok · 0.75s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 42.0 tok · 0.4s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 22.0 tok · 0.28s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 24.0 tok · 0.29s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 47.0 tok · 0.45s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 19.0 tok · 0.28s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 373.0 tok · 19.24s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 770.0 tok · 13.68s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 0/5
· 4096.0 tok · 36.87s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 4096.0 tok · 33.21s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 925.0 tok · 7.62s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 457.0 tok · 3.9s
20/30
8-12GB
4 models tested
deepseek-r1:14b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 243.6 tok · 4.04s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 334.0 tok · 4.12s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 258.8 tok · 3.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 5/5
· 645.6 tok · 7.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 240.0 tok · 3.03s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 261.0 tok · 3.37s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 184.0 tok · 4.36s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 323.0 tok · 3.85s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 290.0 tok · 3.47s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 1/5
· 3900.2 tok · 46.25s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 241.4 tok · 2.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 199.0 tok · 2.48s
26/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 196.0 tok · 3.59s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 242.8 tok · 3.07s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 203.0 tok · 2.61s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 3664.8 tok · 42.6s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 350.2 tok · 4.24s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 184.0 tok · 2.53s
25/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 146.2 tok · 3.25s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 295.0 tok · 3.81s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 199.0 tok · 2.67s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 4096.0 tok · 49.76s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 370.6 tok · 4.66s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 142.0 tok · 2.13s
25/30
12GB+
3 models tested
mistral-small3.2:24b
Champion
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 14.0 tok · 2.32s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 17.0 tok · 0.48s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 21.0 tok · 0.55s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 5/5
· 228.8 tok · 4.18s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 47.0 tok · 1.0s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 20.0 tok · 0.59s
30/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 186.0 tok · 6.93s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 288.0 tok · 7.26s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 275.0 tok · 6.86s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 1/5
· 1185.2 tok · 29.17s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 254.0 tok · 6.37s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 255.0 tok · 6.51s
26/30
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 1 — floor: 5/5
· 164.8 tok · 3.92s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 2 — restraint (refusal): 5/5
· 334.0 tok · 2.84s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 3 — distractor resistance: 5/5
· 271.8 tok · 2.38s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 4 — multi-passage synthesis: 0/5
· 1593.6 tok · 12.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 5 — precise extraction: 5/5
· 469.4 tok · 3.89s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.]
Tier 6 — long-context needle: 5/5
· 238.0 tok · 2.26s
25/30