Join our meetup, learn, connect, share, and get to know your Toronto AI community.
Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.
Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.
I was reading Andrej Karpathy’s blog on RNN to get familiarised with working of RNN, both mathematically and intuitively. From my understanding, there are three sets of parameters to optimise.
And we have the rolling hidden state (H) which accumulates all the information from the inputs. And we optimise on the loss calculated from the output to find the best set of above params
What I am not able to visualise and understand is the part in which the so-called sequential memory is stores ?
Is it stored in the vector H (the rolling hidden state) or the weight matrix Whh ?
In either case, could you also give some intuition on how it contains memory in the form of matrix / vector ?
submitted by /u/thehumanlobster
[link] [comments]