The decision
Three places a model can run, and the data that dictates each
The choice is not ideological; it follows from what the data is and what your obligations say about where it may go. Most estates end up using more than one of these at once.
| Option | What it asks of you | When it is the honest choice | Best for |
|---|---|---|---|
| A hosted endpoint in a compliant region | Contractual review — where the data is processed, whether it trains anything, how long logs persist — and the discipline to hold vendors to it. | When residency and processing terms can be met contractually and verified. For a large share of enterprise workloads this is enough, and pretending otherwise buys hardware, not safety. | Workloads whose constraints are contractual, not physical. |
| Open weights inside your boundary | Accelerator hardware, an inference stack, and — the part underestimated everywhere — a standing operations practice for updates, evaluation and capacity. | When the data may not leave your infrastructure under any contract — classified material, some regulated records, data whose exposure ends the business. The constraint is physical, so the answer is too. | The narrow set of workloads where no contract is enough. |
| A hybrid split by sensitivity | A classification of workloads and a routing layer that sends each request to the venue its data class allows — which is an architecture question before it is a hardware one. | When only a slice of the estate carries the hard constraint. The private capacity serves that slice; everything else uses hosted models and keeps their quality and pace. | Most organisations that need private capacity at all. |
Model selection
Choosing an open-weight model when you are the one who has to run it
Leaderboard rank is the least of it. A model you host is a dependency you operate, and these are the criteria that decide whether it can be operated in your environment at all.
Licence terms
Whether the weights may be used commercially, modified, and deployed at your scale — read by counsel, not assumed from the word "open".
The failure it prevents: A production system on weights you were never licensed to run.
Arabic and domain capability, on your tasks
An evaluation set drawn from your real prompts and documents, in both languages, run before commitment — published benchmarks rarely include your dialect or your paperwork.
The failure it prevents: Discovering after the hardware arrives that the model cannot do the job.
Size against your hardware
The memory the model needs at your context lengths versus the accelerator class you can actually procure and power — including quantised variants and what they trade away.
The failure it prevents: A model that fits the slide and not the server.
Ecosystem and release cadence
Whether the model family is actively maintained, served well by mainstream inference stacks, and likely to have successors — because you will be upgrading, and orphaned weights make every upgrade a migration.
The failure it prevents: Being stranded on an abandoned branch of the ecosystem.
The bill's shape
What moves the cost of a deployment you own
We do not quote figures here, because they change with the market and your suppliers. What holds is which factors move the number, and in which direction — the shape any honest quotation will follow.
| Variable | Effect on price | Why |
|---|---|---|
| The accelerator class and its supply | Sets the floor of the investment | Server-grade accelerators, their hosts, networking, power and cooling are the dominant capital line, and lead times are a planning fact of their own. The class you need follows from the model size and latency you committed to — which is why that decision comes first. |
| Model size and context length | Multiplies the hardware required | A larger model or a longer context window needs more memory per request and serves fewer users per accelerator. Quantisation and smaller model variants pull the other way, at a quality cost your evaluation set — not the vendor's brochure — should measure. |
| Availability and headroom | Doubles what a single box would cost | A deployment the business depends on needs redundancy, capacity for peak rather than average load, and room to grow — which is why sizing from a demo's traffic understates the estate, sometimes severely. |
| The people who run it | The recurring line procurement forgets | Someone patches the inference stack, re-evaluates after every model update, watches capacity and answers at 2am. Whether that is your team, ours, or shared, it is a standing cost that outlives the hardware refresh — and leaving it unbudgeted is how private deployments decay. |
Year two
The second year of a self-hosted model is the real commitment
Standing a model up is the photographed moment; keeping it good is the job. Open-weight families release successors on a rhythm you do not control, and each upgrade is a decision: the new weights may be better on average and worse on the three tasks your business actually depends on, which only your own evaluation set can tell you. Meanwhile the hosted frontier keeps moving, so the gap between what your users get privately and what they can see publicly widens unless someone is actively managing it. A deployment without an owner for this cycle does not fail loudly — it just becomes quietly worse than every alternative while still costing power and floor space.
This is why we scope the operating practice as part of the engagement, not an appendix: the update calendar, the evaluation harness that gates every change, the capacity review, and a named owner on your side trained to run all three. If your organisation cannot staff that owner, the honest advice is usually a hybrid — keep the truly constrained workload small and private, and let everything else ride on hosted models someone else operates.
Our role
What we build inside the boundary and what your operation inherits
- 01
The honest assessment first
We classify the workloads against your actual obligations and say plainly which need self-hosting and which do not — including when the answer is that a compliant endpoint serves you and no hardware should be bought.
- 02
The deployment, sized by evidence
Model selection run against your own evaluation set, hardware sized from measured workload rather than optimism, and the serving stack, isolation and logging built to your security team's standards.
- 03
The operating cycle, handed over running
The update calendar, evaluation harness and capacity reviews are stood up as practice with your named owner running them alongside us before we step back — because a deployment handed over as documentation decays.
What your data may touch is set by your obligations and your counsel's reading of them; we build the deployment that honours that reading and the operating habit that keeps honouring it after the launch photograph.
Common questions
What buyers ask
Does sensitive data always mean we have to self-host the model?+
No, and treating sensitivity as an automatic verdict for hardware is one of the more expensive reflexes in enterprise AI. The real question is what your specific obligations require of the data: many sensitivity and residency constraints are satisfiable contractually, by a hosted endpoint that processes in an approved region, does not train on your inputs, and limits log retention — terms you can negotiate and verify. Self-hosting is for the narrower case where no contract is acceptable because the data may not leave infrastructure you physically control. The assessment we run splits your workloads along exactly that line, and most estates come out mixed: a small genuinely constrained core, and a majority that a compliant endpoint serves better, faster and with stronger models.
What class of hardware does running our own language model involve?+
Server-grade accelerators — the GPU class built for data centres, not workstations — in hosts with the memory, networking, power and cooling to match, usually more than one of everything once availability is taken seriously. We deliberately do not quote models or prices: the market moves quickly, supply varies by region, and a figure written today misleads next quarter. What we can say is how the sizing works: the model you choose and the context lengths you serve set the memory per request, your peak concurrency sets how many requests must be served at once, and redundancy roughly doubles whatever that produces. Sizing runs in that order — model first, workload measured, hardware last — and a quotation that starts from the hardware is running the logic backwards.
Who keeps a self-hosted model current once the project team leaves?+
A named owner on your side, or the deployment should not happen — that is the blunt version of our advice. The recurring work is real: new weights in the model family arrive and must be evaluated against your own test set before adoption, the inference stack needs patching like any other server software, capacity needs reviewing as usage grows, and quality complaints need someone who can tell a model problem from a retrieval problem. We build that cycle as part of the engagement and run it jointly until your owner runs it alone; we can also stay on as the operating partner where you would rather buy the practice than staff it. What we will not do is stand up a deployment, hand over a wiki page and call it operational — that is how a confident launch becomes an embarrassing audit two years later.
In this cluster
Next step
Scope it with us
An assessment that names which of your workloads genuinely require self-hosting, which are served by an in-region endpoint, and what running the private part will demand of your team.
Scope it with us
What happens next
- 1We reply within one business day.
- 2A 30-minute scoping call with a senior consultant.
- 3A written proposal with scope, team, and timeline.
Or reach us directly

