I’ve gotten fairly interested in neural networks and machine learning, and wanted to learn more about them. So, what better way to do that than to make some yourself? I didn’t want to use already existing libraries, like TensorFlow, to do it. I wanted to learn how to make one, from the ground up. So, I set out to make a library of my own in C#. Things have gone well. I’ve tried various types of networks, and I’ve gotten them to work. But, recently, I decided that I wanted to try out RNN/LSTM. This is where I’ve gotten a bit stuck.

I understand the concept of it, the math of it is clear to me. However, I am having some difficulties in getting the architecture of it all laid out. Most of the models I’ve seen of a RNN/LSTM use only one input. Or, at least, that’s how I’ve interpreted it, and I may be wrong.

Additionally, how do you size the layers? From what I’ve gathered, each LSTM “gate” is its own layer. If you want to use more than 1 point of data, do you line up several of those gates and connect the output from gate 1 to gate 2, and add the 2nd point of input to gate 2?

Most of the research that I’ve found on neural networks have been pretty straightforward, and I’ve been able to apply it fairly easily. But, when it comes to RNN/LSTM, I haven’t been able to find much. Most of the help out there is related to TensorFlow, and doesn’t tell me the fundamental structure of the network. And the ones that I’ve found on the fundamental structure of the network all seem to contradict each other, so I’m left scratching my head.

I figured if anyone could help me out with this, it would be the good people of this subreddit. Any help would be appreciated!