Code Gauntlet — code-gauntlet-v1
Code Generation Tool Calling RAG / Q&A Reasoning / Math | About

The Gauntlet

Six escalating tests, run identically across every model. Find what actually holds up on your hardware — organized by VRAM tier, not marketing size class.

Sub-4GB

6 models tested
gemma4:e2b Champion
1.8 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 1193.8 tok · 7.7s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 1813.6 tok · 9.56s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1823.8 tok · 9.72s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 1204.0 tok · 6.32s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 2066.0 tok · 10.79s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 827.2 tok · 4.45s
30/30
gemma4:e4b
3.4 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 1079.2 tok · 9.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 1855.6 tok · 13.44s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1790.4 tok · 13.13s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 916.0 tok · 6.75s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 1368.2 tok · 9.98s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 39.0 tok · 0.59s
25/30
granite4:3b
2.9 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 171.0 tok · 1.1s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 0/5 · 410.0 tok · 2.02s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 492.2 tok · 2.4s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 161.0 tok · 0.84s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 538.8 tok · 2.63s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 31.0 tok · 0.24s
20/30
alibayram/smollm3
2.7 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 1/5
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 4/5
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 1/5
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5
16/30
llama3.2:3b
3.1 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 115.0 tok · 1.01s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 202.0 tok · 0.94s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 0/5 · 765.4 tok · 3.15s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 321.4 tok · 1.4s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 614.0 tok · 2.54s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 29.0 tok · 0.27s
15/30
phi4-mini
3.7 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 106.6 tok · 1.62s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 0/5 · 252.4 tok · 1.31s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 1/5 · 1021.8 tok · 4.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 264.2 tok · 1.38s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 821.4 tok · 3.96s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 32.0 tok · 0.31s
11/30

4-8GB

4 models tested
qwen3.5:9b Champion
5.7 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 781.0 tok · 7.48s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 0/5 · 730.0 tok · 6.21s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1246.0 tok · 10.25s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 702.0 tok · 5.91s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 846.0 tok · 6.99s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 860.0 tok · 7.13s
25/30
qwen2.5-coder:7b
5.1 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 297.0 tok · 2.1s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 231.6 tok · 1.65s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 511.0 tok · 3.45s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 325.0 tok · 2.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 698.0 tok · 4.66s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 42.0 tok · 0.41s
25/30
qwen3:8b
6.3 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 2170.0 tok · 15.44s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 0/5 · 4096.0 tok · 29.61s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 2700.0 tok · 19.28s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 2003.2 tok · 14.25s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 4084.8 tok · 29.62s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 650.8 tok · 4.68s
20/30
mistral:7b
5.6 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 188.6 tok · 1.24s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 0/5 · 321.0 tok · 2.01s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 718.4 tok · 4.47s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 295.8 tok · 1.87s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 356.0 tok · 2.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 41.2 tok · 0.3s
15/30

8-12GB

4 models tested
gemma4:12b-it-qat Champion
8.0 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 943.2 tok · 12.04s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 1690.0 tok · 19.45s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 2118.6 tok · 24.57s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 900.4 tok · 10.43s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 2015.4 tok · 23.33s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 646.4 tok · 7.59s
30/30
gemma4:12b
8.4 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 981.4 tok · 12.42s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 1835.4 tok · 22.13s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1883.6 tok · 22.8s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 1063.2 tok · 12.82s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 1993.2 tok · 24.05s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 631.0 tok · 7.69s
30/30
deepseek-r1:14b
10 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 992.0 tok · 13.62s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 2296.8 tok · 28.76s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1591.6 tok · 19.6s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 2588.8 tok · 32.23s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 4096.0 tok · 50.92s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 512.0 tok · 6.29s
25/30
qwen3:14b
10 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 2007.4 tok · 23.97s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 1/5 · 4070.6 tok · 48.31s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 2572.2 tok · 30.24s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 1621.8 tok · 18.97s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 0/5 · 3525.6 tok · 41.65s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 414.6 tok · 4.92s
21/30

12GB+

3 models tested
qwen3:32b Champion
22 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 2103.0 tok · 52.02s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 2559.8 tok · 63.04s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 3312.0 tok · 81.74s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 1807.8 tok · 44.31s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 3332.2 tok · 82.22s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 545.0 tok · 13.36s
30/30
gemma4:26b
17 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 1089.8 tok · 10.83s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 4/5 · 1654.6 tok · 12.73s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 1908.8 tok · 14.75s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 1186.8 tok · 9.2s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 2133.8 tok · 16.48s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 5/5 · 611.4 tok · 4.9s
29/30
mistral-small3.2:24b
15 GB measured VRAM
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 1 — floor: 5/5 · 295.2 tok · 7.18s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 2 — edge-case handling: 5/5 · 411.0 tok · 7.26s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 3 — multi-class interface: 5/5 · 844.0 tok · 14.86s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 4 — bug-fix: 5/5 · 257.6 tok · 4.64s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 5 — schema cross-reference: 5/5 · 669.0 tok · 11.76s
[This tool call rendered an interactive widget in the chat. The user can already see the result — do not repeat it in text or with another visualization tool.] Tier 6 — strict JSON format: 0/5 · 46.0 tok · 0.97s
25/30