For a long time, I have been using old style spreadsheets to log the results of my experiments with columns like “dataset-version”, “git commit” “PARAMS” “results”…etc

However, I find that this became a pain to consistently fill and update those spreadsheets especially when performing hyper-param search.

There are many frameworks built to manage your ML experiments. I have collected a list below (thanks to the comment here). I would like to know if you have a favourite of the ones below or another

​

SACRED https://github.com/IDSIA/sacred

Studiohttps://github.com/studioml/studio
Datmohttps://github.com/datmo/datmo
Lorehttps://github.com/instacart/lore

FORGE https://github.com/akosiorek/forge
Sumatra https://pythonhosted.org/Sumatra/
RandOpthttps://github.com/seba-1511/randopt
Pachydermhttps://github.com/pachyderm/pachyderm
feature Forgehttps://github.com/machinalis/featureforge
Model Chimphttps://github.com/ModelChimp/modelchimp
PolyAxonhttps://github.com/polyaxon/polyaxon
Kubeflowhttps://github.com/kubeflow/kubeflow
Weights and Biaseshttps://www.wandb.com/

​

ps: This maybe a repost but it is worth revisiting this topic since new frameworks are out.