Matzakos N (2026)
Publication Language: English
Publication Status: Submitted
Publication Type: Unpublished / Preprint
Future Publication Type: Journal article
Publication year: 2026
Open Access Link: https://doi.org/10.48550/arXiv.2604.00543
We prove that activation saturation imposes a structural dynamical limitation on autonomous Neural ODEs h˙=fθ(h) with saturating activations (tanh, sigmoid, etc.): if q hidden layers of the MLP fθ satisfy |σ′|≤δ on a region~U, the input Jacobian is attenuated as $\norm{Df_\theta(x)}\le C(U)$ (for activations with supx|σ′(x)|≤1, e.g.\ tanh and sigmoid, this reduces to CWδq), forcing every Floquet (Lyapunov) exponen along any T-periodic orbit γ⊂U into the interval [−C(U),C(U)]. This is a collapse of the Floquet spectrum: as saturation deepens (δ→0), all exponents are driven to zero, limiting both strong contraction and chaotic sensitivity. The obstruction is structural -- it constrains the learned vector field at inference time, independent of training quality. As a secondary contribution, for activations with σ′>0, a saturation-weighted spectral factorisation yields a refined bound C˜(U)≤C(U) whose improvement is amplified exponentially in~T at the flow level. All results are numerically illustrated on the Stuart--Landau oscillator; the bounds provide a theoretical explanation for the empirically observed failure of tanh-NODEs on the Morris--Lecar neuron model.
APA:
Matzakos, N. (2026). Activation Saturation and Floquet Spectrum Collapse in Neural ODEs. (Unpublished, Submitted).
MLA:
Matzakos, Nikolaos. Activation Saturation and Floquet Spectrum Collapse in Neural ODEs. Unpublished, Submitted. 2026.
BibTeX: Download