Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[P] Not sure if it fits here, I made Simple Distributed Job Scheduler with the only SSH required

https://github.com/JeiKeiLim/simple_distribute_job

I posted earlier that I have built a face detector to blur faces for videos. It was so slow that it takes about 10 hours per each video and I have 160 videos to process….. Which takes about more than two months.

I don’t think I can wait for that long. Thankfully, I have a few machines to run in parallel. And Managing which videos has been done processing which is not and merging all videos to one computer seems too much work.

So I made a simple management job management program for multiple servers to do undone work and upload the resulting video to the main server once it is finished.

Then I realized it could be used for machine learning training on multiple machines or experimentations. I know there are good distributed job management systems but it looks like it requires to install too much. All I needed was a simple job scheduler.

I’m posting this hoping someone finds this useful.

Thank you!

BTW, I’ve been running this with blurring faces on video for 2 days and it still says it will be done in 15 days. So.. I got that going for me which is nice

submitted by /u/workout_JK
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.