AI / Gemma

When the data genuinely cannot leave.

Sometimes the constraint isn't quality, it's jurisdiction — or a per-request cost that has to round to nothing at millions of calls. Gemma is Google's family of open-weight models, and open weights mean we can run inference inside your own account, network and audit boundary.

Self-hosted Data residency Predictable unit cost Fine-tunable
When we reach for it

Four constraints that point to open weights.

01

The data can't cross a boundary

Regulated records, contractual restrictions, or a security team that will not sign off on outbound inference. Weights come to the data instead of the reverse.

02

Volume makes per-token pricing hurt

Classification, tagging, routing and enrichment running millions of times a day. Once utilisation is high enough, owning the inference beats renting it.

03

Latency has to be tight and stable

Inference next to your application, with no shared-tenant queue and no third-party rate limit deciding your p99 on a busy afternoon.

04

The task is narrow and repetitive

A single well-defined job, done the same way every time. A smaller model tuned on your examples frequently outperforms a large general one here — and costs a fraction.

How we run it

Self-hosting is an infrastructure problem. That's our home turf.

The model is the easy bit. Keeping GPUs busy but not wasted, surviving a node failure mid-request, and rolling out a new checkpoint without a maintenance window — that's ordinary platform engineering, and it's what we do anyway.

  • Deployed on your AWS account — GPU instances or managed endpoints
  • Served through standard inference runtimes with an OpenAI-compatible API surface
  • Autoscaling tuned to your traffic shape, with scale-to-zero for spiky workloads
  • Batching and quantisation applied where quality allows, measured before and after
  • Model versions treated as deployable artefacts, with rollback
  • Optional fine-tuning or adapters on your own labelled data
How we build on AWS
Runs inYour account
Data egressNone
ScalingAutoscale · scale-to-zero
WeightsVersioned artefact
Cost modelPer GPU-hour, not per token
The honest trade-offs

Self-hosting is cheaper per request and more expensive per engineer.

We'd rather you hear this before signing than discover it in month three.

You now own the uptime

No provider absorbs a bad night for you. That's fine if you already run production infrastructure — and a real cost if you don't.

Idle capacity is real money

Below a certain steady utilisation, a hosted API is simply cheaper. We'll model the crossover point with your actual traffic before recommending either.

Frontier capability moves fast

For the hardest reasoning tasks, a large hosted model is usually still ahead. Many systems end up hybrid — small and local for volume, large and hosted for the hard cases.

Gemma

Told you can't send that data anywhere?

Then let's bring the model to it. We'll size the infrastructure, model the cost and show you what quality you'd actually get.

Get in touch