flashinfer-ai / flashinfer

大模型Grok-1开源

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

Open-Sora： 完全开源的高效复现类Sora视频生成方案

#大语言模型#SGLang is a fast serving framework for large language models and vision language models.

#大语言模型#vLLM 是一个高效的开源库，用于加速大语言模型推理，通过优化内存管理和分布式处理实现高吞吐量和低延迟。

#大语言模型#Code examples and resources for DBRX, a large language model developed by Databricks

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

Devika is an Agentic AI Software Engineer that can understand high-level human instructions, break them down into steps, research relevant information, and write code to achieve the given objective. D...

#自然语言处理#LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Training LLMs with QLoRA + FSDP

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

Fast and memory-efficient exact attention

Flash Attention in ~100 lines of CUDA (forward pass only)

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-L...

#大语言模型#AICI: Prompts as (Wasm) Programs

#计算机科学#Puzzles for learning Triton

#大语言模型#🙌 OpenHands: Code Less, Make More