Without gateway
With gateway
Simple step
Hard step
Repeat call
Knock frontier offline
Model gateway routing diagram
Without a gateway, an agent sprawls separate arrows to three models labelled fast, mid-tier, and frontier, and carries the retry, key, cache, and formatting plumbing itself.
AGENT
+ retry / fallback
+ per-provider keys
+ cache check
+ format each request
FAST / CHEAP
model
MID-TIER
model
FRONTIER / REASONING
model
Toggle the gateway on, then send different requests to watch it route, cache, and fall back.