Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[P] Git hook for large files: because who wants to have their 100TB data file committed to Git?

The usual disclaimer – this is not my project, but it is simple and awesome so I wanted to share.

Check out this Git pre-commit hook for large files.

What it does:
Most people working on serious ML projects have probably experienced this issue, where you accidentally do git add .
and after committing (or worse, pushing to the remote), realize that you added your ginormous model / data to the repository.

If you’re a Git expert, you can definitely fix it. But why fix something you can avoid?

It’s super easy to install (only Linux/Mac are currently supported):

curl -L https://gist.github.com/guysmoilov/ddb3329e31b001c1e990e08394a08dc4/raw/install.sh | bash

By default limits files to 5MB max size, but this can be configured with:

GIT_FILE_SIZE_LIMIT=42000000 git commit -m “This commit is allowed file sizes up to 42MB”

The hook itself is based on this Gist – which deserves credit as well.

Thanks to both developers! What other hooks do you use as part of your ML work?

submitted by /u/PhYsIcS-GUY227
[link] [comments]