SqueezeAILab / SqueezeLLM

#自然语言处理#[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

大模型Grok-1开源

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

#大语言模型#🙌 OpenHands: Code Less, Make More

Open-Sora： 完全开源的高效复现类Sora视频生成方案

#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

#自然语言处理#[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Training LLMs with QLoRA + FSDP

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

#自然语言处理#LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

#大语言模型#Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

#大语言模型#SGLang is a fast serving framework for large language models and vision language models.

GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

#大语言模型#HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

An easy-to-use package for implementing SmoothQuant for LLMs

#大语言模型#Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

#大语言模型#AICI: Prompts as (Wasm) Programs