That's a good point, but in situations with small datasets, deep learning yields poor results. There's nothing sacred about deep learning. It's often useful, but not always. For example, gradient boosting on sequences delivers excellent results despite not being able to "remember" previous samples.