# SparseOrLowRank

- [Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection](/transformer_evolution_paper/mha/sparseorlowrank/001.md)
- [Scatterbrain: Unifying Sparse and Low-rank Attention Approximation](/transformer_evolution_paper/mha/sparseorlowrank/002.md)
- [Sparse Factorization of Large Square Matrices](/transformer_evolution_paper/mha/sparseorlowrank/003.md)
- [Blockwise Self-Attention for Long Document Understanding](/transformer_evolution_paper/mha/sparseorlowrank/004.md)
- [H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences](/transformer_evolution_paper/mha/sparseorlowrank/005.md)
- [ChunkFormer: Learning Long Time Series with Multi-stage Chunked Transformer](/transformer_evolution_paper/mha/sparseorlowrank/006.md)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](/transformer_evolution_paper/mha/sparseorlowrank/007.md)
- [Fast Transformers with Clustered Attention](/transformer_evolution_paper/mha/sparseorlowrank/008.md)
- [Long-Short Transformer: Efficient Transformers for Language and Vision](/transformer_evolution_paper/mha/sparseorlowrank/009.md)
- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](/transformer_evolution_paper/mha/sparseorlowrank/010.md)
- [Luna: Linear Unified Nested Attention](/transformer_evolution_paper/mha/sparseorlowrank/011.md)
- [Memory-efficient Transformers via Top-k Attention](/transformer_evolution_paper/mha/sparseorlowrank/012.md)
- [Separable Self-attention for Mobile Vision Transformers](/transformer_evolution_paper/mha/sparseorlowrank/013.md)
- [Simple Local Attentions Remain Competitive for Long-Context Tasks](/transformer_evolution_paper/mha/sparseorlowrank/014.md)
- [You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling](/transformer_evolution_paper/mha/sparseorlowrank/015.md)
