2+2= 4 That's the truth. No dispute. It can be proven
what prof brainwashed you child?
GRU vs LST vs MRNN : Difference lies in More Flexible control
When we start reading about RNN (Recurrent Neural Net) and its advanced cells, we are introduced with a Memory Unit (in GRU) and then additional Gates (in LSTM).
However, if we analyse the basic architecture of all 3,
Basic architectures of RNN, GRU and LSTM cells
we can easily see, there is no such Memory Unit present in the cells. But increasing number of Mathematical operations of Input(xt) and previous output (ct-1, ht-1).
Simple RNN :- Here there is simple multiplication of Input (xt) and Previous Output (ht-1). Passed through Tanh activation function. No Gates present.
Gated Recurrent Unit (GRU) :- Here a Update gate is introduced, to decide whether to pass Previous O/P (ht-1)to next Cell (as ht) or not. Forget gate is nothing but additional Mathematical Operations with a new set of Weights (Wt).
But, Math operations are performed on same inputs (i.e. xt and ht-1).
Long Short Term Memory Unit (LSTM) :- Here 2 more Gates are introduced (Forget and Output) in addition to Update gate of GRU. And again as above, these are additional Mathematical Operations on same inputs (xt and ht-1). So overall, LSTM has introduced 2 Math operations having 2 new sets of Weights.
BTW NEURAL NETS SUCK
