Open-Weight Models
Our model architecture supports open-weight models when deployment control, private inference, specialization, or workload economics justify operating the model directly.
What changes
Open-weight models change where responsibility sits.
Instead of consuming a managed external inference endpoint, we can control more of the serving environment, model version, adaptation path, hardware allocation, and runtime behavior. That control is useful, but it also transfers infrastructure and evaluation burden to us.
Why we use them
Open-weight execution can be appropriate when:
- Workload volume makes external inference economics unattractive
- Sensitive data should remain inside a controlled environment
- A narrow task does not require frontier-level reasoning
- Latency benefits from local execution
- Task-specific adaptation materially improves the workload
- Model and runtime control matter operationally
For recurring extraction, classification, transformation, monitoring, or other bounded workloads, the correct open-weight model can provide a better quality, cost and control tradeoff than sending every request to a larger external model.
The model is only one dependency
Running open weights in production requires more than downloading a checkpoint. The surrounding execution path can involve:
- Model artifact
- Runtime
- Memory allocation
- Quantization / precision choice
- Batching + concurrency
- API layer
- Monitoring
- Version management
Hardware capacity, VRAM, context length, throughput, and concurrency all affect the feasible deployment envelope.
Model quality also remains workload-specific. An open model that performs well on a public benchmark may still be inferior on the company’s actual classification, retrieval, tool-use, or reasoning task.
Operating tradeoff
The primary advantage is control. The primary cost is also control.
Local serving introduces responsibility for uptime, deployment, model upgrades, monitoring, inference optimization, hardware utilization, and regression evaluation.
We therefore do not treat open-weight models as inherently more private, cheaper, or better than managed frontier models in every case. They are one execution class inside a model-agnostic architecture, used when the measurable workload benefits justify owning more of the stack.
Scope
This page is about model availability and control: what open-weight licensing makes possible, what it costs in capability, and what it gives back in versioning, portability and independence from a single vendor.
Whether a given workload should run privately is a separate decision, covered in Private / Local Models; the serving stack that runs it is Local Model Serving.