llm-inference
#大语言模型#Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
#大语言模型#Gitleaks 是一个开源SAST（静态应用安全测试）命令行工具，用于检测Git 仓库以防止把密码、API 密钥和访问令牌等机密信息硬编码到代码中
#大语言模型#本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
#大语言模型#20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
#大语言模型#Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
#大语言模型#Official inference library for Mistral models
#自然语言处理#OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
#大语言模型#PowerInfer 是一个快速的、可运行在消费级GPU、个人电脑上的大模型服务
#大语言模型#The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
🚀 全网效果最好的移动端【实时对话数字人】。 支持本地部署、多模态交互（语音、文本、表情），响应速度低于 1.5 秒，适用于直播、教学、客服、金融、政务等对隐私与实时性要求极高的场景。开箱即用，开发者友好。
#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
#向量搜索引擎#Superduper: End-to-end framework for building custom AI applications and agents.
Eko (Eko Keeps Operating) - Build Production-ready Agentic Workflow with Natural Language - eko.fellou.ai
#计算机科学#Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
Open-source implementation of AlphaEvolve
FlashInfer: Kernel Library for LLM Serving
#大语言模型#Simple, scalable AI model deployment on GPU clusters
The smart edge and AI gateway for agents. Arch is a high-performance proxy server that handles the low-level work in building agents: like applying guardrails, routing prompts to the right agent, and ...
- Website
- Wikipedia