[P] Deploying generalizable deep learning models to production search engines
We recently implemented new features such as Kubernetes support, a frontend, etc on an open source project I’ve been working on called NBoost.
So I wrote an article to talk about some of the hurdles of building a production-scale domain-specific deployment of SoTA models.
Some of the main features are:
– open-source hosting of finetuned models for domain-specific knowledge
– Kubernetes deployment via Helm
– A frontend for tracking model and network latency