linformer

list of efficient attention modules

#计算机科学#My take on a practical implementation of Linformer for Pytorch.

Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)

#计算机科学#Vision Xformers

#自然语言处理#linformer implementation and comparison with vanilla transformers

#大语言模型#Lumenspark is a lightweight Linformer-based Language Model Trained from Scratch