[D] Is there a machine learning framework that allows you to manually change the layer weights after each update step?
I am looking to sparse training; that is, during each training step, only a very small fraction of the weights go through the forward and backwards pass. So, only a small fraction of weights need to be loaded into memory at once.
I was thinking of having the weights saved in sqlite or hdf5, and then having a keras layer which copies the relevant weights stored for the forward pass. And then after the update step, those weights are update into the disk.
I know there are ways to directly setting the weights outside training https://stackoverflow.com/questions/51354186/how-to-update-weights-manually-with-keras
But I am wondering if any framework allows manually updating variables, after they have been already updated in a training step.
submitted by /u/Research2Vec
[link] [comments]