01
Prompt caching: pay for a replayed context once
You should be able to: Explain what prompt caching does and when it is worth it.
A coding agent — and most agents — replays a large, mostly-static prefix on every turn: a long system prompt, the tool definitions, the file or retrieved context. Without caching you pay full input price for all of those tokens every single turn, even though they did not change. On a long session that is the bulk of the bill, spent re-reading what the model already saw.
Caching changes the deal. You pay a one-off premium to write the prefix into a cache, then a fraction of the input rate on every subsequent read of it. The varying part of the request — the new instruction, the latest tool result — is priced normally. For any workload that replays a stable prefix, this is the single biggest lever on cost, and it is why the inference-as-yield economics of an agent depend on getting it right.
The catch is that caching is a feature of the model and the provider, not something a gateway invents. Gatewayz passes it through, but a model that does not support caching is billed at the normal input rate — so caching is an optimisation you confirm, never one you assume. The next lessons are how you mark it and how you prove it worked.
Source: Inference as Yield
02
Mark the breakpoint, then prove the cache hit
You should be able to: Add a cache_control breakpoint and read the usage fields that confirm it took effect.
You mark what to cache with a cache_control breakpoint on a content block. Everything before the breakpoint is cached, so you put it at the end of your static prefix — the system prompt and tool definitions, not the varying instruction. In an OpenAI-compatible request the system content becomes a list of typed blocks:
{
"model": "anthropic/claude-sonnet-4-5-20250929",
"messages": [
{ "role": "system", "content": [
{ "type": "text", "text": "<your long, stable system prefix>",
"cache_control": {"type": "ephemeral"} }
]},
{ "role": "user", "content": "the varying instruction" }
]
}
Then prove it. The response usage block reports cache activity — cache_read_input_tokens and cache_creation_input_tokens alongside prompt_tokens and completion_tokens. On the first call you expect a creation count; on a repeat you expect a non-zero read count. If cache_read_input_tokens is zero on a repeat, the breakpoint is not taking effect — usually the prefix is below the provider's minimum cacheable length, or it changed between turns. Reads are billed at the cache rate and writes at the write rate, so a cache that silently is not hitting is the exact failure that "does not error, it costs you": the call succeeds and the bill is wrong.
Source: Building with Gatewayz
03
Tool calling: the model asks, your code answers
You should be able to: Describe the tool-call round-trip and whose job each half is.
Tool calling lets a model do more than talk: you declare a set of tools (functions with names and parameters) in the tools field, and when the model decides it needs one, it returns a tool_calls entry naming the tool and the arguments. That is the model asking. It has not done anything — it has requested that you do.
The other half is yours. You execute the tool, then send the result back as a message with role "tool" so the model can continue with the answer in hand. Gatewayz is explicit about this boundary: it returns tool_calls and expects you to run them and feed the results back — it does not run the agent loop for you. Every coding agent that works this way (Cline, Aider, Claude Code) runs its own loop; the gateway forwards the calls.
This is the shape of every agent: a loop of model-asks, you-act, model-continues, until the model answers instead of calling. Understanding that the gateway carries the messages but does not drive the loop is what stops you looking for a feature that is deliberately not there — the loop is application logic, and it is yours to own the way the build course said you own the output.
Source: Building with Gatewayz
04
Forcing a tool, and structured output
You should be able to: Force a specific tool call and request structured output, and know which parameters are forwarded.
Sometimes you do not want the model to choose. tool_choice lets you force a specific tool, honoured end to end through the gateway — you hand it the tool and say "use this one":
{
"tools": [{"type": "function", "function": {"name": "read_file", "parameters": {}}}],
"tool_choice": {"type": "function", "function": {"name": "read_file"}}
}
The related parameters travel with it. Gatewayz forwards tools, tool_choice, parallel_tool_calls and response_format to the provider unmodified — response_format is how you ask for structured output (JSON that conforms to a shape), and parallel_tool_calls governs whether the model may request several tools at once. A parameter a given provider does not accept is pruned rather than passed through as an error, which is convenient and is also exactly why the next lesson exists: pruned-and-succeeded and honoured-and-succeeded look identical unless you check.
Source: Directing Agents Well
05
The capability check, and verifying through a gateway
You should be able to: Check a model's capabilities before relying on a feature, and verify it behaved after.
Before you send, ask the catalog. GET /v1/models returns each model with a supported_parameters list and a capabilities object, so you can check whether a model supports tools, vision or prompt caching before you build on it — capabilities.prompt_caching being the one that decides whether your cache_control does anything. Checking first turns a silent degradation into a decision you made on purpose.
After you send, verify with the numbers. For caching, read cache_read_input_tokens; for tools, confirm the tool_calls came back and your round-trip completed. This is the same verify-first discipline the build course teaches, aimed at the two features most likely to fail quietly, because a pruned parameter and a missed cache both return a perfectly successful response.
Gatewayz builds this honesty into its own measurement, which is the standard to copy: its coding-agent benchmark publishes measured results only, and if a run sends cache_control but gets zero cache reads back, it emits a warning and publishes no cost-advantage claim from that run. A number you cannot verify is not a number you publish — the estate's oldest rule, enforced by the gateway on itself. Hold your own caching and tool-use claims to the same bar, and the advanced features become dependable instead of merely available.
Source: The Verifiable Record
