Join our meetup, learn, connect, share, and get to know your Toronto AI community.
Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.
Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.
I am looking to sparse training; that is, during each training step, only a very small fraction of the weights go through the forward and backwards pass. So, only a small fraction of weights need to be loaded into memory at once.
I was thinking of having the weights saved in sqlite or hdf5, and then having a keras layer which copies the relevant weights stored for the forward pass. And then after the update step, those weights are update into the disk.
I know there are ways to directly setting the weights outside training https://stackoverflow.com/questions/51354186/how-to-update-weights-manually-with-keras
But I am wondering if any framework allows manually updating variables, after they have been already updated in a training step.
submitted by /u/Research2Vec
[link] [comments]