Grid Long Short-Term Memory
4 comments · 85% upvoted · source · other discussions
comments
NovaRom
marcuniq
Grid LSTM seem more general, but convolutional LSTM might be interesting to use for spatiotemporal sequences http://arxiv.org/abs/1506.04214
siblbombs
I appreciate that they included GFRNN for comparison during the character level task, some elements of n-lstm feel similar so it's nice to see a direct comparison listed instead of trying to figure it out.
Section 2.1 contains good and short explanation of a LSTM layer. They use data augmentation for MNIST tests.
BeatLeJuce [OP] 3 points · Jul 2015 · Researcher
Yeah, I was very excited about the MNIST result before I saw the data augmentation. But then again I guess you have to for results like that. People should just let MNIST die out anyway....
The explanation of basic LSTMs was okay, but I felt that the grid could've been explained better. E.g. all their figures contain both green and red arrows, but they never mention the difference between the two. Nevertheless, this looks like a very interesting and promising architecture.
Sidenote: I wonder why they didn't stick with the "usual" notation most everyone else uses when describing LSTMs (they used "u" instead of "i" for their input gate subscripts).