Topic: "linear-attention"
BlinkDL/RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Language: Python - Size: 22 MB - Last synced at: 1 day ago - Pushed at: 17 days ago - Stars: 13,533 - Forks: 913
happinesslz/LION
[NeurIPS 2024] Official code of ”LION: Linear Group RNN for 3D Object Detection in Point Clouds“
Language: Python - Size: 1.63 MB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 153 - Forks: 10
lucidrains/taylor-series-linear-attention
Explorations into the recently proposed Taylor Series Linear Attention
Language: Python - Size: 779 KB - Last synced at: 15 days ago - Pushed at: 8 months ago - Stars: 97 - Forks: 3
lucidrains/agent-attention-pytorch
Implementation of Agent Attention in Pytorch
Language: Python - Size: 516 KB - Last synced at: 28 days ago - Pushed at: 10 months ago - Stars: 90 - Forks: 5
lironui/Multi-Attention-Network
The semantic segmentation of remote sensing images
Language: Python - Size: 2.41 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 52 - Forks: 7
lucidrains/autoregressive-linear-attention-cuda
CUDA implementation of autoregressive linear attention, with all the latest research findings
Language: Python - Size: 5.86 KB - Last synced at: 24 days ago - Pushed at: almost 2 years ago - Stars: 44 - Forks: 3
lironui/MAResU-Net
The semantic segmentation of remote sensing images
Language: Python - Size: 6.6 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 40 - Forks: 2
BICLab/MetaLA
Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)
Language: Python - Size: 211 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 17 - Forks: 0
robflynnyh/hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
Language: Python - Size: 8.79 KB - Last synced at: 5 months ago - Pushed at: over 2 years ago - Stars: 11 - Forks: 0
OSU-STARLAB/LeaPformer
[ICML 2024] Official implementation of "LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions."
Language: Python - Size: 20.1 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 9 - Forks: 1
gmlwns2000/sea-attention
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
Language: Python - Size: 372 MB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 7 - Forks: 1
mtanghu/LEAP
LEAP: Linear Explainable Attention in Parallel for causal language modeling with O(1) path length, and O(1) inference
Language: Jupyter Notebook - Size: 3 MB - Last synced at: 18 days ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 0
RWKV-Wiki/rwkv-wiki.github.io
RWKV Wiki website (under heavy construction yet)
Language: Shell - Size: 700 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0
fattorib/flashy_linear_attention
Flash linear attention kernels in Triton
Language: Python - Size: 258 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
Rushi314/Transformers-for-high-resolution-image-synthesis Fork of CompVis/taming-transformers
Taming Transformers for High-Resolution Image Synthesis
Language: Jupyter Notebook - Size: 400 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
