linear-attention

#大语言模型#RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN a...

[NeurIPS 2024] Official code of ”LION: Linear Group RNN for 3D Object Detection in Point Clouds“

#计算机科学#Explorations into the recently proposed Taylor Series Linear Attention

#计算机科学#Implementation of Agent Attention in Pytorch

The semantic segmentation of remote sensing images

The semantic segmentation of remote sensing images

#计算机科学#CUDA implementation of autoregressive linear attention, with all the latest research findings

#大语言模型#Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)

Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)

Code for the paper "Cottention: Linear Transformers With Cosine Attention"

#计算机科学#Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)

#计算机科学#RWKV Wiki website (archived, please visit official wiki)

[ICML 2024] Official implementation of "LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions."

Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)

#计算机科学#LEAP: Linear Explainable Attention in Parallel for causal language modeling with O(1) path length, and O(1) inference