Blog

Learn About Our Meetup

4200+ Members

[D] How do you manage your machine learning experiments?

For a long time, I have been using old style spreadsheets to log the results of my experiments with columns like “dataset-version”, “git commit” “PARAMS” “results”…etc

However, I find that this became a pain to consistently fill and update those spreadsheets especially when performing hyper-param search.

There are many frameworks built to manage your ML experiments. I have collected a list below (thanks to the comment here). I would like to know if you have a favourite of the ones below or another

SACRED https://github.com/IDSIA/sacred

Studiohttps://github.com/studioml/studio
Datmohttps://github.com/datmo/datmo
Lorehttps://github.com/instacart/lore

FORGE https://github.com/akosiorek/forge
Sumatra https://pythonhosted.org/Sumatra/
RandOpthttps://github.com/seba-1511/randopt
Pachydermhttps://github.com/pachyderm/pachyderm
feature Forgehttps://github.com/machinalis/featureforge
Model Chimphttps://github.com/ModelChimp/modelchimp
PolyAxonhttps://github.com/polyaxon/polyaxon
Kubeflowhttps://github.com/kubeflow/kubeflow
Weights and Biaseshttps://www.wandb.com/

ps: This maybe a repost but it is worth revisiting this topic since new frameworks are out.

submitted by /u/pigdogsheep
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.