smoothquant

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

#大语言模型#A powerful toolkit for compressing large models including LLM, VLM, and video generation models.