[P] fastai-Serving: running containerized inference with fastai models
Code: fastai-serving repo
We’ve been experimenting with some Fast AI models recently for our remote sensing work. Unfortunately, we ran into a lot of issues when trying to deploy those models on large-scale inference jobs (specifically running land-classification on big satellite imagery datasets). This fastai-serving repo is meant to solve this in a way that mimics the TF Serving approach/API. Namely, it helps you package a trained model within a small Docker image (running a mini server) so you can make prediction requests via REST POST requests.
We’re working on expanding the functionality (and are very receptive to any help!). For anyone who’s running inference on large image sets, we usually spin up multiple of these these inference-ready images and run large batch predictions with our open chip-n-scale pipeline.