WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention
arXiv:2602.22266v1 Announce Type: new Abstract: State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be used to derive stable, memory-efficient dynamical systems that encode the past history of the input signal. However, existing projection-based SSMs often rely on polynomial bases with global temporal support, whose inductive biases are poorly matched to signals exhibiting localized or transient structure. In this work, we introduce \emph{WaveSSM}, a collection of SSMs constructed over wavelet frames. Our key observation is that wavelet frames yield a localized support on the temporal dimension, useful for tasks requiring precise localization. Empirically, we show that on equal conditions, \textit{WaveSSM} outperforms orthogonal counterparts as S4 on real-world datasets with transient dynamics, including physiological signals on the PTB-XL dataset and raw audio
arXiv:2602.22266v1 Announce Type: new Abstract: State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be used to derive stable, memory-efficient dynamical systems that encode the past history of the input signal. However, existing projection-based SSMs often rely on polynomial bases with global temporal support, whose inductive biases are poorly matched to signals exhibiting localized or transient structure. In this work, we introduce \emph{WaveSSM}, a collection of SSMs constructed over wavelet frames. Our key observation is that wavelet frames yield a localized support on the temporal dimension, useful for tasks requiring precise localization. Empirically, we show that on equal conditions, \textit{WaveSSM} outperforms orthogonal counterparts as S4 on real-world datasets with transient dynamics, including physiological signals on the PTB-XL dataset and raw audio on Speech Commands.
Executive Summary
This article introduces WaveSSM, a novel collection of state-space models (SSMs) built upon wavelet frames. WaveSSM addresses the limitations of existing projection-based SSMs, which often rely on polynomial bases with global temporal support, by leveraging localized support on the temporal dimension. Empirical results demonstrate that WaveSSM outperforms its orthogonal counterparts on real-world datasets with transient dynamics. The proposed approach has the potential to improve signal attention in various applications, including physiological signal processing and raw audio analysis. The authors' innovative use of wavelet frames to construct SSMs is a significant contribution to the field of sequential modeling.
Key Points
- ▸ WaveSSM is a collection of SSMs constructed over wavelet frames, offering localized support on the temporal dimension.
- ▸ Existing projection-based SSMs often rely on polynomial bases with global temporal support, which can be poorly matched to signals with localized or transient structure.
- ▸ Empirical results show that WaveSSM outperforms its orthogonal counterparts on real-world datasets with transient dynamics.
Merits
Improved Signal Attention
WaveSSM's localized support on the temporal dimension enables precise localization, which is beneficial for tasks requiring accurate signal attention.
Enhanced Stability
The use of wavelet frames in WaveSSM can lead to more stable and memory-efficient dynamical systems compared to existing projection-based SSMs.
Demerits
Computational Complexity
The proposed approach may introduce additional computational complexity due to the use of wavelet frames, which could be a limitation in resource-constrained environments.
Limited Generalizability
The empirical results may not generalize to other domains or applications, which could limit the broader impact of WaveSSM.
Expert Commentary
The introduction of WaveSSM represents a significant advancement in the field of sequential modeling, as it addresses the limitations of existing projection-based SSMs. The use of wavelet frames to construct SSMs is a clever innovation, and the empirical results demonstrate the effectiveness of this approach. However, it is essential to consider the potential computational complexity and limited generalizability of WaveSSM. Further research is needed to fully explore the implications of this work and to develop more scalable and generalizable methods.
Recommendations
- ✓ Future research should focus on developing more efficient algorithms for computing wavelet frames and integrating WaveSSM with other sequential modeling techniques.
- ✓ The authors should investigate the applicability of WaveSSM to other domains and applications to further establish its generalizability.
