I’m working on my AutoML startup since 2016. Recently, I’ve come to the conclusion that AutoML is broken. AutoML should make deploying ML easier and faster, I think it is the opposite. What is more, I think it can do more harm than good.

The benefits of AutoML:

1. AutoML can train (some) model for provided input data. Nice!
2. AutoML can be used for computing batch predictions for provided test data.
3. AutoML can be used for deploying the model behind REST API. User can access model with REST requests. Nice 🙂

The boring code – hidden part needed to make AutoML work:

1. In real-life someone needs to create the input dataset. There is a need to write some code which will fetch data from multiple sources, merge and create a data frame. This code needs to be well written, so all software engineering good practices should be applied.
2. If the final ML model will be used in production, then more code needs to be created, for preparing data for predictions. Whether you are going to use batch predictions or use REST API, you need to add more code.
3. Most of the AutoML solutions I know, do not provide detailed model maintenance mode. There should be monitoring of model and monitoring for input data quality. Based on this monitoring, there should be an automated decision when to update/retrain the model. The feedback of model decisions should be stored somewhere. If you go to production with AutoML, probably there will be a need to write additional code for maintenance and monitoring.

I think there is a lot of boring-code needed to use AutoML in real-life in the production.

The last thing I don’t like in AutoML is complexity. When you apply ML to the problem even a simple model should bring you huge improvement (ROI). I like simple models because they are easy to understand. Applying a complex model as a first iteration for solving a business case with ML is unnecessary over-complication.

If you want to read more, I wrote a post about my thoughts about broken AutoML: https://pplonski.github.io/automatic-machine-learning-is-broken/