A frontier model went dark within days of launch. If your inference path has a single point of failure, that was your warning.
Here is the uncomfortable part most AI architecture diagrams skip. Your inference provider is a dependency you do not control, and you have been treating it like a utility that never goes down.
On June 9, two flagship models from a major US lab launched. On June 12, both were suspended under a US export-control directive. Three days. That was the first time a US-lab frontier model got pulled offline that fast, and it should reset how you think about continuity for anything that calls a model in a production path.
If your application had one of those model IDs hard-wired into a request loop, your June 12 was spent writing an incident report instead of shipping. The lesson is not "pick a different lab." The lesson is that single-provider inference is now a continuity risk on the same tier as a single-AZ database or a single-region control plane. We learned those lessons the hard way years ago. We are about to relearn them with models.
I run cloud security and platform teams in regulated environments, and I want to walk through the actual mechanism of making inference survivable on AWS. Not slideware. Things you can put in a backlog Monday.
Treat the model as a swappable backend, not a hard dependency
The first failure mode is application code that talks directly to a vendor SDK with a model string baked in. That couples your business logic to one company's roadmap, one company's pricing, and, as of this month, one government's export posture.
Put a gateway in front of it. Concretely, your services call an internal inference endpoint that speaks one stable contract. Behind that endpoint, you route to Amazon Bedrock as a primary and to a second provider as an independent fallback. Bedrock hosts multiple model families under one API surface, and as of late April it added OpenAI frontier models, Codex, and managed agents in preview. That is useful, but do not let "multi-model inside one vendor" convince you that you have redundancy. A single export directive or a single account-level issue can take the whole surface with it. Real redundancy means a second control plane you can fail over to, owned by a different company.
The gateway pattern buys you four things at once. A place to enforce request-level authorization. A place to attribute cost. A place to pin versions. A place to reroute when a backend disappears. You are not building this to be clever. You are building it so that swapping a model is a config change, not a code deploy under pressure.
Pin your versions, then schedule the re-validation
The opposite mistake is just as dangerous as hard-coupling. GPT-5.5 Instant became the ChatGPT default in early May and is exposed through a floating "chat-latest" alias. Floating aliases are wonderful for a chat window and a quiet catastrophe for a regulated workload. The model under that alias can change without notice, which means your evaluation results, your safety testing, and your output formatting were all validated against something that no longer exists.
Pin to explicit, immutable model IDs. Use "claude-opus-4-8", not "latest." Opus 4.8 shipped on May 28, roughly 41 days after 4.7. That cadence is the point. Frontier models now move on something close to a six-week rhythm, so your pinned version will drift from the frontier quickly.
Pinning is only half the control. The other half is a re-validation cadence. Put a standing item on the calendar, every quarter at minimum, to run the new candidate model against your evaluation set, your prompt-injection tests, and your cost profile before you promote it. Pinning without re-validation is how you wake up two years behind on capability and price. Re-validation without pinning is how you ship untested behavior. You need both, and you need the schedule written down where an auditor can see it.
Repatriate the inference that legally cannot leave
Some data cannot go to a third-party API at all. Not "should not." Cannot. If you operate under HIPAA, under SOC 2 commitments you actually made to customers, or under the kind of financial-services controls the US Treasury laid out in February with 230 control objectives across seven domains, then for certain data classes the right architecture is to keep the model inside your perimeter.
This is now realistic. Gemma 4 open-weight models released on May 1, which gives you a self-hosting path for the data that cannot leave. The pattern on AWS is straightforward. Run the open-weight model on inference instances inside a controlled VPC. No internet egress. Private subnets. Endpoint policies that deny anything you did not explicitly allow. The same data-loss controls you would put around a production database. The model weights live in your account. The prompts and completions never cross a vendor boundary.
You will not run your whole workload this way. Open weights at a given size will not match a frontier model on the hardest tasks. That is fine. This is a routing decision, not a religion. Regulated, perimeter-bound data goes to the in-VPC open-weight model. Everything else can go to the gateway and out to Bedrock or a second provider. Draw that line explicitly in your data classification, because the moment the line is implicit, someone will send protected data to a public endpoint and call it a feature.
Route to the cheapest model that still passes
Continuity and cost are the same architecture problem viewed from two angles, and the gateway is where both get solved. Once you have an abstraction layer, you can do minimum effective intelligence routing. Send each task to the cheapest model that still yields an accepted result, and only escalate to the expensive model when the cheap one fails your acceptance check.
That requires measurement. Bedrock added request-level usage attribution on May 20, which means you can finally tie spend to a tenant, a feature, or a team instead of getting one undifferentiated bill. Wire that into per-request cost attribution and hard budget caps at the gateway. The FinOps Foundation named AI cost management the top wanted skill for 2026, with about 98 percent of organizations now managing AI spend, and the reason is simple. Without attribution you cannot route on cost, and without caps a runaway agent loop can spend a quarter's budget over a weekend.
The continuity dividend is that the same routing table that picks the cheapest passing model is the table you flip when a provider goes offline. You already built the muscle. Failover is just routing with a different trigger.
What to actually do Monday
Start with one honest inventory. List every production path that calls a model, and for each one write down the exact model ID, whether it is pinned or floating, what happens if that endpoint returns errors for an hour, and what data class flows through it. Most teams cannot answer the fourth column, and that gap is the real finding.
Then sequence the work. Stand up the gateway with one primary and one fallback. Replace every floating alias with a pinned ID and put the re-validation date on the calendar. Classify your data and move the perimeter-bound classes to an in-VPC open-weight deployment. Turn on request-level cost attribution and set budget caps. None of this is exotic. It is the same discipline we already apply to databases, regions, and credentials, finally pointed at the model layer.
The models will keep getting better, and they will keep getting pulled, deprecated, repriced, and re-aliased. Design for the version that disappears, and the upgrades take care of themselves.
How are you handling model continuity right now? I am curious whether anyone has actually rehearsed a provider-down failover, or whether, like most of us, you found out it worked the hard way. Tell me in the comments.
