01
One seam, not one integration per app
You should be able to: Explain why an AAO routes inference through a shared seam rather than wiring providers per app.
@flashyos/llm-gateway exists because the inference code "lived twice as byte-identical copies" in the estate, and "two copies of the thing every AI call passes through drift expensively". The fix was one seam: a provider-agnostic package every product calls, so there is a single place where inference is configured, metered and failed over.
For an AAO the argument is stronger than DRY. When the whole organization's agents call inference through one seam, a provider change is one environment edit for everyone, cost is attributable in one accounting, and an outage is absorbed in one breaker rather than in each app separately. The seam is the inference layer of the stack, and Gatewayz is what it points at.
The boundary is deliberate: the package "takes configuration and an error factory. It does not read your environment or import your error class — that boundary is what makes it a package rather than a copy with extra steps." You hand it config and how to build an error; it hands you inference. That is what lets one seam serve a whole estate without dragging each app's specifics into it.
Source: Flashy OS — @flashyos/llm-gateway (the inference seam)
02
Wiring Gatewayz as the provider
You should be able to: Configure the seam to route inference to Gatewayz with Anthropic as the automatic fallback.
The seam is created with a config block and an error factory. Provider selects who serves inference; the other configured provider becomes the automatic fallback:
const gateway = createGateway(
{
provider: process.env.LLM_PROVIDER, // 'anthropic' | 'gatewayz'
anthropicApiKey: process.env.ANTHROPIC_API_KEY,
gatewayzApiKey: process.env.GATEWAYZ_API_KEY,
gatewayzBaseUrl: process.env.GATEWAYZ_BASE_URL, // https://api.gatewayz.ai/v1
},
(code, httpStatus, message) => new AppError(code, httpStatus, message),
);
Set LLM_PROVIDER=gatewayz, GATEWAYZ_API_KEY to your gw_ key from Secret Manager (never a file or a repo), and GATEWAYZ_BASE_URL to https://api.gatewayz.ai/v1. Then every call — gateway.call({ model, systemPrompt, userPrompt, maxTokens }) — returns text, inputTokens, outputTokens, a requestId and which provider served it. Cutover is configuration: "changing providers is an environment change, never a deploy", and flipping the estate onto Gatewayz is exactly this variable, not a code change.
Source: Getting Set Up with Gatewayz
03
What the seam guarantees on top of Gatewayz
You should be able to: Name the reliability guarantees the seam adds, and the one Gatewayz-specific trap it handles.
The seam adds production guarantees around the raw call. Degradation never costs availability: "an unknown provider id, or a primary without credentials, falls back rather than failing." Failover is automatic behind a circuit breaker — after three consecutive failures the primary is skipped for 60 seconds and heals on read. And a caller's own 4xx is not counted as a provider failure (since 0.2.0), so a bad model id does not trip the breaker — with one exception the seam keeps, a 429, because that is the provider saying "not now".
One trap is specifically Gatewayz-shaped and worth knowing: "Gatewayz answers an unknown or unpriced model id with 503 service_unavailable, which is a client error wearing a server status". The seam still counts that and still trips the breaker, because from the outside it is indistinguishable from real provider trouble. The lesson: a wrong model id routed to Gatewayz can look like an outage, so read the requestId and the error body before blaming the provider.
And the empty-success guard, which matters more through a gateway: "a response with no content and zero output tokens — how some gateways report a failed upstream call — is raised as an error and falls over, rather than being returned as a free, silent non-answer." The seam turns a gateway's quiet failure into a loud one the failover can catch.
Source: Building with Gatewayz
04
Cost is the caller's, and what is not yet routed through the seam
You should be able to: Attribute cost across the estate and name the traffic that should not yet go through the seam.
The seam "reports tokens and a requestId; it does not price the call." Pricing is the estate's own job, computed against its model registry, because "a provider's aggregate usage feed cannot attribute a figure to one request". So Gatewayz's dashboard total is a reconciliation input, and the per-call token counts the seam returns are the source of truth for what each agent's work cost — which is how an AAO attributes inference spend to a function, a client, or an initiative.
Know the current edge honestly. The estate seam lists tool use and prompt caching as "not yet verified through a gateway" — a fact about the seam's own coverage, not about Gatewayz, which forwards both (see Prompt Caching and Tool Use). Until the seam's semantics matrix covers them, an estate agent that needs tool calls or cache_control should call the Gatewayz endpoint directly with the verify-first checks, rather than assume the seam carries them untouched.
That is the whole integration: one seam, Gatewayz behind it, failover and metering and cutover-as-configuration for the entire organization, and a written line around the two features still being brought through. It is what lets the estate say inference is a layer it operates rather than a dependency it hopes holds — the posture an AAO needs under a workforce that runs on inference.
Source: Inference in the Agentic Economy
