For twenty years, scale was your friend. Add users, spread fixed cost, watch margin climb. The economy of scale was the whole financial promise of SaaS.

AI breaks that promise. Intelligence carries a real marginal cost. Every prompt spends tokens, compute, and latency. Grow without control and your cost per user goes up, not down. The companies shipping agents fastest are discovering this the hard way: a feature that delighted in the demo becomes a variable-cost problem in production.

This is not a pricing problem you negotiate with a provider. It is an architecture problem. And it is decided in one place: the boundary between your product and the model.

The real problem is what you send the model

Most teams send too much.

The default agent build ships the entire system prompt, the full knowledge base, every registered tool, and a heavy slice of the live UI to the model on every single turn. It works in a demo with three tools and one document. It falls apart at fifty tools, a real knowledge base, and a production report on screen.

Three failures compound from there.

Token inflation. You pay to send context the model does not need for this turn. The bill scales with usage, and usage is the thing you were trying to grow.

Accuracy decay. Long context windows suffer from lost-in-the-middle. Bury the relevant tool or fact in a wall of irrelevant ones and the model misses it. More context produces worse answers, not better.

Model lock-in. Hard-wire one model into your stack and you inherit its price, its latency, and its outages. When a cheaper or faster model ships next quarter, you cannot move.

A control surface, not a black box

The fix is a layer that sits between your product and the model and gives your team control over what reaches it. We call it the inference gateway. It does two jobs. It decides what reaches the model on every turn: what data, what context, which model. Then it measures the result, so cost, latency, and quality become things you can see and tune instead of a surprise at the end of the quarter.

This is undifferentiated infrastructure. Every team going AI-native is rebuilding the same gateway, and it gets harder every month as models and tools multiply. None of it is the thing your customers pay you for. The build-versus-buy question answers itself: buy the gateway, put your engineers on the agent only you can build.

Decide: what data

A production UI is dense. A single report can hold hundreds of thousands of tokens of underlying data. Dumping that payload at the model is the most expensive and slowest thing an agent can do, and most of it is noise for the question actually being asked.

The gateway runs a pre-model computation pass. It serves the agent only the data required to answer, not the entire surface.

In one representative report workload, that means the difference between roughly 940k tokens of raw report data and the 20k to 40k the agent actually needs to analyze. Latency drops with it, from ten to twenty seconds down to two to three. Same answer. A fraction of the cost. A fraction of the wait.

That is the clearest proof of the whole thesis. You do not control cost by capping usage. You control it by controlling what computation happens before the model ever runs.

Decide: what context

Data is one input. The harder problem is everything else: which user details, which pieces of knowledge, which tools belong in this specific turn of this specific conversation.

The real-time context engine assembles that per prompt. It selects the relevant user context, the relevant knowledge, and the relevant tools for each message, balancing accuracy, latency, cost, and outcome. The model sees a focused, current picture instead of the entire catalog every time.

The piece most teams miss is tool selection. The lost-in-the-middle problem applies to tools as much as to text. Hand the model forty tools when three are relevant and you have not just raised the cost, you have lowered the odds it picks the right one. Assembling the right tools per turn is an accuracy lever, not only a cost lever.

This is what it means to engineer context instead of stuffing it. The window stays compact and current because something is deciding what belongs in it, turn by turn.

Decide: which model

Not every task needs your most expensive model. Routine data entry and a cross-system workflow are not the same job, and paying frontier-model rates for the simple one is waste by default.

The gateway selects the model per task. You can set the best model for a given task agent, and with dynamic model selection the product agent picks the best model for the job on its own, today across our supported providers. Simple turns go to fast, cost-effective models. Heavy reasoning gets the heavyweight. The decision is made at the gateway, not frozen into your codebase.

This is also how you escape lock-in. When a better model ships, you adopt it at the gateway and your whole product benefits, with no rewrite. The model becomes a swappable input instead of a permanent dependency.

Measure: see and tune what you decided

A gateway that decides but does not measure is just a faster black box. The control only matters if you can see what it is doing and adjust it. That is the second half of the layer, and it is what turns the cost claims above into numbers your own team generates.

Cost, by action. The analytics attribute spend down to the specific action the agent performed, not a single monthly total. You see which capability is driving the bill, the way you would see that one query is driving a database bill. When a single tool starts eating a disproportionate share of spend, it shows up as its own segment, and you know exactly where to optimize.

Quality, on real traffic. Trust Lab takes real production conversations and turns them into tests. It grades whether the agent did what the user actually wanted, runs the checks on a schedule, and flags regressions before users hit them. When you change a prompt, a tool, or a model at the gateway, you measure what that change did to quality instead of hoping.

Put the two together and a change stops being a guess. Tighten the context, route a step to a cheaper model, and you can watch what it does to spend and to quality before it reaches a customer. That is the loop: decide, measure, tune. It is also the honest answer to model routing. You do not believe a cheaper model is good enough for a step. You test it on your own traffic and see.

The enterprise owns the outcome

Stack it all and the cost curve bends back in your favor. Less data, the right context, the right-sized model, every turn, with the spend and the quality visible so you can keep tuning. Growth improves margin again because the layer between your users and the model is doing real work and showing its work.

There is a strategic point underneath the financial one. Frontier models are commoditizing. When everyone can call the same model, the durable advantage is not which model you use. It is the control surface around it: what reaches it, what comes back, and whether you can measure and improve both. That control belongs to you, the enterprise that owns the outcome. The gateway is how you hold it without building it.

The model is rented. The control over it should be yours.

This is the first piece in a series on the stabilization layer. Next, we go deep on the pre-model data optimization that takes a 940k-token report down to 40k. After that: the real-time context engine, dynamic model selection, and the measurement loop that keeps all of it honest in production.

Mickey Alon — Co-Founder & CEO, Foldspace AI