We recently implemented new features such as Kubernetes support, a frontend, etc on an open source project I’ve been working on called NBoost.

So I wrote an article to talk about some of the hurdles of building a production-scale domain-specific deployment of SoTA models.

Some of the main features are:

– open-source hosting of finetuned models for domain-specific knowledge

– Kubernetes deployment via Helm

– A frontend for tracking model and network latency