cutlass

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

#大语言模型#🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

Examples of CUDA implementations by Cutlass CuTe

CUTLASS and CuTe Examples

#大语言模型#Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

#计算机科学#GEMM and Winograd based convolutions using CUTLASS

study of cutlass

#大语言模型#Multiple GEMM operators are constructed with cutlass to support LLM inference.

Kernel Library Wheel for SGLang

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

#大语言模型#Reading notes on the open source code of AI infrastructure (sglang, llm, cutlass, hpc, etc.)

pytorch implements block sparse

Docs

Lightweight and production level C++ Open source Library

This repository showcases common optimization techniques for kernels.