Cortex just released V 0.10, which includes their new Predictor Interface for serving models. It lets you take models from any framework and implement them in simple Python, before deploying them with a single terminal command. V 0.10 also still includes out-of-the-box support for TensorFlow Serving and ONNX Runtime.

Repo link: https://github.com/cortexlabs/cortex

Examples:

– Deploying Hugging Face’s DistilGPT-2 – PyTorch

– Deploying a sentiment analyzer with BERT – TensorFlow

– The classic iris classifier – XGBoost/ONNX