HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
arXiv:2602.21009v1 Announce Type: cross Abstract: Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints. While summarizing history via interest centers offers a practical alternative, existing methods struggle to (1) identify user-specific centers at appropriate granularity and (2) accurately assign behaviors, leading to quantization errors and loss of long-tail preferences. To alleviate these issues, we propose Hierarchical Sparse Activation Compression (HiSAC), an efficient framework for personalized sequence modeling. HiSAC encodes interactions into multi-level semantic IDs and constructs a global hierarchical codebook. A hierarchical voting mechanism sparsely activates personalized interest-agents as fine-grained preference centers. Guided by these agents, Soft-Routing Attention aggregates historical signals in semantic space, weighting by simi
arXiv:2602.21009v1 Announce Type: cross Abstract: Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints. While summarizing history via interest centers offers a practical alternative, existing methods struggle to (1) identify user-specific centers at appropriate granularity and (2) accurately assign behaviors, leading to quantization errors and loss of long-tail preferences. To alleviate these issues, we propose Hierarchical Sparse Activation Compression (HiSAC), an efficient framework for personalized sequence modeling. HiSAC encodes interactions into multi-level semantic IDs and constructs a global hierarchical codebook. A hierarchical voting mechanism sparsely activates personalized interest-agents as fine-grained preference centers. Guided by these agents, Soft-Routing Attention aggregates historical signals in semantic space, weighting by similarity to minimize quantization error and retain long-tail behaviors. Deployed on Taobao's "Guess What You Like" homepage, HiSAC achieves significant compression and cost reduction, with online A/B tests showing a consistent 1.65% CTR uplift -- demonstrating its scalability and real-world effectiveness.
Executive Summary
This article proposes HiSAC, a novel framework for ultra-long sequence modeling in recommender systems. HiSAC addresses the limitations of existing methods by employing a hierarchical voting mechanism to identify personalized preference centers and Soft-Routing Attention to aggregate historical signals. The framework achieves significant compression and cost reduction, demonstrating its scalability and real-world effectiveness. The authors deploy HiSAC on Taobao's homepage and report a consistent 1.65% CTR uplift in online A/B tests. The article contributes to the development of efficient and accurate recommender systems, addressing the growing need for personalized recommendations in various industries. The authors' approach has the potential to improve user experiences and business outcomes in e-commerce and other fields.
Key Points
- ▸ HiSAC proposes a hierarchical voting mechanism for identifying personalized preference centers
- ▸ Soft-Routing Attention is used to aggregate historical signals in semantic space
- ▸ The framework achieves significant compression and cost reduction in production environments
Merits
Strength in scalability and real-world effectiveness
HiSAC is deployed on a large-scale recommender system and demonstrates its ability to improve user experiences and business outcomes in production environments.
Efficient compression and cost reduction
The framework achieves significant compression and cost reduction, making it suitable for large-scale recommender systems.
Improved accuracy in ultra-long sequence modeling
HiSAC addresses the limitations of existing methods by employing a hierarchical voting mechanism and Soft-Routing Attention.
Demerits
Potential complexity in implementation
The framework requires the development of a hierarchical voting mechanism and Soft-Routing Attention, which may be complex to implement.
Limited evaluation on diverse datasets
The article primarily evaluates HiSAC on a single dataset and does not provide a comprehensive evaluation on diverse datasets.
Expert Commentary
This article makes a significant contribution to the development of recommender systems, addressing the limitations of existing methods in ultra-long sequence modeling. The authors' use of a hierarchical voting mechanism and Soft-Routing Attention provides a novel framework for personalized recommendation. However, the potential complexity in implementation and limited evaluation on diverse datasets remain concerns. Nevertheless, the article's findings on the scalability and real-world effectiveness of HiSAC make it a valuable contribution to the field. Future research should focus on further evaluating HiSAC on diverse datasets and exploring its potential applications in various industries.
Recommendations
- ✓ Future researchers should evaluate HiSAC on diverse datasets to assess its generalizability and robustness.
- ✓ The development of HiSAC should be further explored in various industries, including e-commerce, healthcare, and finance.
