Engineering, production, governance, cost, communication · insight-0285
Compare managed and self-hosted ML platforms on total cost including people
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- saas, cross-industry
Summary: situation, action, reason, boundary
Situation
Self-hosting an open-source serving stack looks cheaper than a managed service on the cloud bill.
Action
Estimate total monthly cost for both: compute at realistic utilisation, licence or service fees, engineering and on-call hours at loaded cost, and migration effort; add qualitative risks such as vendor lock-in and data residency.
Reason
Infrastructure bills are only part of operating cost; staff time often dominates for small teams.
Boundary
Choosing serving, training, feature store or vector database infrastructure.
Symptom or risk
Comparisons that ignore engineering time, on-call, upgrades, security patching and idle capacity choose the option that is more expensive in practice.
Exceptions
At very large, steady scale, self-hosting often wins once a platform team already exists.
Trade-offs
Managed services trade control and portability for speed.
Pitfalls
Using peak-month usage for one option and average usage for the other.
Example
Hypothetical: managed fee 3,000 per month versus self-hosted compute 1,200 plus 0.3 of an engineer; at a loaded engineer cost of 12,000, self-hosting costs about 4,800.