Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] Testing different linking strategies (hard links vs symlinks vs reflinks) for managing ML projects

Linking data or model files makes it possible to rearrange any amount of files very fast, while avoiding copying, thus saving disk space. Data science teams use symlinks to save space and avoid copying large datasets.

The tutorial start with a strategy of copying files into place, then using hard links and symbolic links, then ending up with a new type of link, reflinks, which implements Copy On Write capabilities available in modern file systems on Linux and Mac OS: Reflinks vs symlinks vs hard links, and how they can help machine learning projects

submitted by /u/cmstrump
[link] [comments]