Summarizing https://arxiv.org/pdf/2303.06349.pdf

Here's my try:

ListOps

λj=exp(−exp(νlogj)+iexp(θlogj))γj←(1−|λj|2)1/2magnitudephase

Tanh-RNN

aligned with S4/5

PathFinde%

skip connection

time pool

90%

90%

Stable

Deep RNNs (the S4 way)

NRrm

6tDble

LLn-5NN

MLP/GLU(same for all timestamps)

70%

70%