01
One model, many providers, different economics
You should be able to: Explain why the same model can have very different cost, speed and reliability across providers.
An open-weight model is a file. Any provider with the right hardware can serve it, and many do. That means the exact same model exists at several places at once, each with its own price per token, its own latency, its own capacity, and its own reliability record. They are genuinely substitutable — the output is the same model — but the economics are not the same at all.
Even closed models, served only by their maker, sit next to close substitutes: a smaller model from the same lab, or a competing model that clears the same bar for your task. The point stands. For almost any request, there is more than one acceptable answer to "what serves this", and they differ in cost and speed by large multiples.
Once you see this, hardcoding one provider looks like what it is: paying whatever that provider charges, tolerating whatever latency it has, and inheriting its outages, when substitutes were one configuration change away. The rest of the course is about not doing that.
Source: Gatewayz — the unified inference gateway
02
The four axes: cost, latency, quality, availability
You should be able to: Score an inference choice on the four axes and know which one your workload is bound by.
Every inference choice trades along four axes. Cost, per token, is the one everyone sees. Latency is how fast the answer arrives, which for interactive products is a feature. Quality is whether the model actually clears the bar the task needs — and a cheaper, smaller model often does, which is where most savings hide. Availability is whether the provider answers at all when you call.
The discipline is to know which axis binds each workload. A user-facing autocomplete is latency-bound and quality-tolerant. A legal summary is quality-bound and can wait. A high-volume classification job is cost-bound above all. The same four axes, weighted differently, give different right answers — which is exactly why one global default is wrong for a system that does more than one kind of work.
This is also the honest answer to "which model is best". There is no best; there is best-for-this-request-on-these-weights. A serious inference strategy is a set of rules mapping kinds of request to the choice that clears their bar most cheaply.
03
The gateway pattern — one endpoint, many backends
You should be able to: Describe the gateway pattern and what it decouples in your application.
A gateway is a single endpoint that sits in front of many providers. Your application talks to the gateway; the gateway talks to whichever provider should serve this call. Because the interface it exposes is stable, the choice of who actually serves a request becomes a configuration and routing concern rather than something wired into your code.
This is the same move software has made many times: put an abstraction where a hardcoded dependency was, so the thing behind it can change without the thing in front of it noticing. The estate's own gateway package states the boundary plainly — it "takes configuration and an error factory. It does not read your environment or import your error class." That boundary is what makes it a seam rather than a copy of provider code.
What the pattern buys you: you can add a provider, drop one, or change which model serves a request without touching application logic. The choice of inference stops being a thing you did once and becomes a dial you can turn — which is the precondition for everything routing, failover, and cost control depend on.
Source: Flashy OS — @flashyos/llm-gateway (the inference seam)
04
Routing: choose per request, not per codebase
You should be able to: Explain intelligent routing and give an example of a rule that lowers cost without lowering quality.
Once a gateway makes the choice a dial, routing is the logic that turns the dial per request. A router can pick a model based on cost, latency or quality targets — sending a simple request to a small cheap model and only escalating to a frontier model when the task warrants it, without the caller changing a line. Gatewayz describes exactly this: routing that selects the optimal model for a request without requiring endpoint changes or code rewrites.
The savings from routing come from a simple fact established earlier: a smaller model often clears the bar. If most of your requests are easy and a few are hard, routing the easy ones to cheap capacity and reserving the expensive model for the hard ones cuts cost sharply while quality on the requests that mattered is unchanged. Providers that publish this pattern report meaningful cost reductions from it; the mechanism, not any specific figure, is what to internalise.
Routing is also where cost control and reliability meet: the same layer that picks the cheapest acceptable model can pick the next acceptable one when the first is slow or down. That is the subject of the build course — here the point is only that the choice belongs per request, and a gateway is what makes that possible.
Source: Building with Gatewayz
05
Lock-in is a cost you pay later
You should be able to: Explain why a hardcoded provider is a deferred cost and how a gateway converts it to a switchable one.
Hardcoding one provider feels free because the bill for it arrives later. It arrives as the day that provider raises prices and you cannot leave; as the outage that takes your product down because you had no fallback; as the better, cheaper model you cannot adopt without a migration. None of these show up in the first sprint, which is exactly why the decision gets made carelessly.
A gateway converts that deferred, unbounded cost into a bounded, present one: the modest engineering of routing through a seam. Having paid it, price rises become a routing change, outages become an automatic failover, and a new model becomes a config entry. The estate built its gateway for precisely this reason — the inference seam "lived twice as byte-identical copies", and two copies of the thing every AI call passes through "drift expensively".
The lesson generalises past AI: put the seam where the vendor is, before you are locked in, because the cheapest time to gain the option to switch is before you need it. With the case for choosing inference made, the next question is the one underneath it — why inference can be a source of yield at all.
Source: Inference as Yield
