Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] How do you manage your machine learning experiments?

For a long time, I have been using old style spreadsheets to log the results of my experiments with columns like “dataset-version”, “git commit” “PARAMS” “results”…etc

However, I find that this became a pain to consistently fill and update those spreadsheets especially when performing hyper-param search.

There are many frameworks built to manage your ML experiments. I have collected a list below (thanks to the comment here). I would like to know if you have a favourite of the ones below or another

SACRED https://github.com/IDSIA/sacred

Studiohttps://github.com/studioml/studio
Datmohttps://github.com/datmo/datmo
Lorehttps://github.com/instacart/lore

FORGE https://github.com/akosiorek/forge
Sumatra https://pythonhosted.org/Sumatra/
RandOpthttps://github.com/seba-1511/randopt
Pachydermhttps://github.com/pachyderm/pachyderm
feature Forgehttps://github.com/machinalis/featureforge
Model Chimphttps://github.com/ModelChimp/modelchimp
PolyAxonhttps://github.com/polyaxon/polyaxon
Kubeflowhttps://github.com/kubeflow/kubeflow
Weights and Biaseshttps://www.wandb.com/

ps: This maybe a repost but it is worth revisiting this topic since new frameworks are out.

submitted by /u/pigdogsheep
[link] [comments]