← Back to leaderboard

gemma4:12b

8.4 GB measured VRAM · 8-12GB tier · rag-gauntlet-v1

Exact passages, questions, and expected answers for this category are held back from publication to avoid training-data contamination — see About for why. This page shows per-tier pass/fail counts and timing only.

Tier 1 — floor 5/5

147 tokens · 8.046s (representative seed)

Tier 2 — restraint (refusal) 5/5

295 tokens · 3.821s (representative seed)

Tier 3 — distractor resistance 5/5

199 tokens · 2.708s (representative seed)

Tier 4 — multi-passage synthesis 0/5

4096 tokens · 49.9s (representative seed)

Tier 5 — precise extraction 5/5

405 tokens · 5.063s (representative seed)

Tier 6 — long-context needle 5/5

142 tokens · 2.198s (representative seed)