Repo Link: https://github.com/cortexlabs/cortex

We’ve just released a new version of Cortex, our open source platform for deploying trained models from any framework as production APIs on AWS. With this newest version, Cortex now also supports:

• Auto Scaling. If your traffic increases, Cortex will spin up new replicas to handle things. If your traffic decreases, Cortex will reduce replicas to save on cost.
• Spot Instances. Cortex can run on AWS Spot Instances, which can reduce instance costs by as much as 90%.
• More Instance Types. Cortex now supports g3 and g4 instance types.
• Batched Predictions. Cortex can now batch predictions.