Hi guys.

We recently open sourced a project, BentoML that packaging and deploying ML models to production. We would love to hear your thoughts.

Quick pitch: From model in Jupyter notebook to production in 5 minutes.

BentoML is a python library for packaging and deploying ML models. It does two things without any changes to your training workflow: 1. It standardize how to package your models, including preprocessing / feature fetching code, dependencies/env, and configuration. 2. It easily distributes your model as Pypi package, API server(local /docker image), CLI tool or spark UDF.

We think there should be a simple way for data scientists to ship models to production. Our vision is empower them to own models ‘end-to-end’ for production service, just like software engineers do.

Here is a quick look of how it works:

In your notebook, you have trained your scikit-learn model: from sklearn import svm from sklearn import datasets

 clf = svm.SVC(gamma='scale') iris = datasets.load_iris() X, y = iris.data, iris.target clf.fit(X, y) 

To package this model with BentoML, you will need to create a new BentoService by subclassing it, and provides artifacts and env definition for it:

 %%writefile iris_classifier.py from bentoml import BentoService, api, env, artifacts from bentoml.artifact import PickleArtifact from bentoml.handlers import DataframeHandler @artifacts([PickleArtifact('model')]) @env(conda_dependencies=["scikit-learn"]) class IrisClassifier(BentoService): @api(DataframeHandler) def predict(self, df): return self.artifacts.model.predict(df) 

Now, to save your trained model for prodcution use, simply import your BentoService class and pack it with required artifacts: from iris_classifier import IrisClassifier

 svc = IrisClassifier.pack(model=clf) svc.save('./saved_bento', version='v0.0.1') # Saving archive to ./saved_bento/IrisClassifier/v0.0.1/ 

That’s it. Now you have created your first BentoArchive. It’s a directory containing all the source code, data and configurations files required to run this model in production.

Couple ways to use this packaged model archive.

Serve the model as local REST API endpoint:

 bentoml serve --archive-path="./saved_bento/IrisClassifier/v0.0.1/" 

Build API server Docker Image from BentoArchive

 $cd ./saved_bento/IrisClassifier/v0.0.1/ $docker build -t myorg/iris-classifier . # After docker build finish $docker run -p 5000:5000 myorg/iris-classifier 

Load your packaged archive in Python:

 import bentoml bento_svc = bentoml.load('./saved_bento/IrisClassifier/v0.0.1/') bento_svc.predict(X[0]) 

Install BentoArchive as PyPI package

 pip install ./saved_bento/IrisClassifier/v0.0.1/ 

Now you can use it as module in your python code

 from IrisClassifier import IrisClassifier installed_svc = IrisClassifier() installed_svc.predict(X[0]) 

Use archived package as CLI tool:

$pip install ./saved_bento/IrisClassifier/v0.0.1/ $IrisClassifier info $IrisClassifier predict --input='./test.csv' 

Alternatively, you can also use the bentoml cli to load and run the package directly:

bentoml predict ./saved_bento/IrisClassifier/v0.0.1/ --input='./test.csv' 

Feel free to ping me and ask any questions. I love to get you guys feedback and improve this project.

Cheers!

edit: formatting