Tier 1 — floor
0/5
seed 1 (FAILED): expected exactly 1 tool call, got 0
17 tokens · 1.867s
assistant
{"name": "get_weather", "arguments": {"city": "Seattle"}}
Tier 2 — restraint
5/5
seed 1 (passed): correctly answered directly, no spurious tool call
18 tokens · 0.253s
assistant
{"name": "get_weather", "arguments": {"city": "Tokyo"}}
Tier 3 — selection
0/5
seed 1 (FAILED): expected exactly 1 tool call, got 0
18 tokens · 0.255s
assistant
{"name": "get_time", "arguments": {"city": "Tokyo"}}
Tier 4 — extraction
0/5
seed 1 (FAILED): expected exactly 1 tool call, got 0
56 tokens · 0.538s
assistant
{
"name": "book_flight",
"arguments": {
"origin": "Chicago",
"destination": "Denver",
"date": "2026-03-24",
"passengers": 3
}
}
Tier 5 — chaining
0/5
seed 1 (FAILED): first turn: did not call get_user_location
32 tokens · 0.365s
assistant
{"name": "get_user_location", "arguments": {}}
{"name": "get_weather", "arguments": {"city": "<user-location>"}}
Tier 6 — strict schema
0/5
seed 1 (FAILED): expected exactly 1 tool call, got 0
61 tokens · 0.574s
assistant
{
"name": "create_calendar_event",
"arguments": {
"title": "Budget Review",
"date": "2026-04-02",
"priority": "high",
"attendees": ["Maria", "James"]
}
}