One Model, Run by Us, for Every Seat

This weekend we flipped the switch. The assistant, its calendar specialist and the agent that reads scanned documents — the three a clinic actually talks to — now run on one model, an open-weights model we host ourselves on an endpoint we own, inside our own cloud project. The provider's hosted models remain for the jobs the open model cannot do — turning text into the vectors that make search work, transcribing dictation, the phone line — and for the background roles that are still moving over, one at a time. Everything a clinic talks to, every day, runs on the one we run.

I wrote in April that the right model for the right job is an architectural decision, not a shopping one. This is that decision carried to its end. I want to describe what it buys, what it costs, and the one sentence that decided it.

What it buys

A hosted model is a service you call. You send the prompt, you get the answer, and what happens in between — what is cached, what is logged, what is retained to improve the service — is governed by someone else's settings, which you can read and which can change. For a clinic's assistant, the prompt is the clinical context. We had already turned every provider-side switch to its quietest setting. That is a reasonable posture. It is still a posture about someone else's machine.

A model you run yourself has no such switches, because there is no one on the other side. The endpoint is ours. Its whole configuration is declared in our code and applied from it. There is no provider-side cache because there is no provider side. There is no log of prompts because nothing we did not write is listening. The nightly check wakes the model and runs the product against it the way it runs everything else. When that check is green, the claim it supports is not “the vendor says they keep nothing” but “there is nothing there to keep it.”

What it costs

The endpoint scales to nothing when nobody is using it, which is most of the hours in a clinic's week. That is what makes it affordable. It also means that the first request after a quiet stretch has to wake it, and waking a model of this size takes about ten minutes. The first wake we measured took just under ten. We measure every one, because our nightly check is the first thing to use the model each day, and I will publish the range once there is a month of them.

We did not hide that. The honest version of a sleeping model is a product that tells you it is waking up. Open the assistant during a wake and the dashboard says the lane is warming and gives you an estimate; it retries on its own and answers when the model does. The alternative — answering from a rented model while the real one wakes — was the fallback we decided not to have.

For a working clinic the right move is obvious and cheap: keep the model warm through clinic hours and let it sleep at night. That is a dial, and the nightly check will keep measuring the wakes either way, so that the dial is set on numbers.

What we will not claim

A self-hosted open model is not automatically better at the work than a frontier hosted one. For the jobs our assistants do — reading a clinic's own documents, drafting, scheduling, filing — the gap closed enough this year that the trade was worth making, and the nightly run proves every night that the work still gets done. For the jobs where the gap has not closed, we still rent, and we say which. The phone line is one of them by design, and it is a post of its own.

Loyal to one, I wrote last month. This is what that looks like on the inside: one model, run by us, for every seat, and a product honest enough to tell you when it is still waking up.