vllm · GitHub Topics

#大语言模型#Llama 2 微调/推理方法和示例

人工智能 finetuning langchain llama llama2 大语言模型机器学习 Python PyTorch vllm

Jupyter Notebook 17.48 k

2 天前

#大语言模型#Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any...

ggml PyTorch chatglm 部署 flan-t5 大语言模型 wizardlm 人工智能机器学习 Whisper inference openai-api mistral gemma llama llamacpp vllm qwen llama3 glm4

Python 8.03 k

13 小时前

OpenRLHF / OpenRLHF

An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & Ray & Dynamic Sampling & Async Agent RL)

transformers vllm large-language-models raylib reinforcement-learning-from-human-feedback reinforcement-learning openai-o1 proximal-policy-optimization

Python 7.08 k

2 天前

katanaml / sparrow

#大语言模型#Data processing and instruction calling with ML, LLM and Vision LLM

机器学习 huggingface-transformers 自然语言处理机器视觉 gpt 大语言模型 rag vllm

Python 4.57 k

3 天前

xlite-dev / Awesome-LLM-Inference

📚A curated list of Awesome LLM Inference Papers with Codes.

flash-attention tensorrt-llm vllm llm-inference deepseek deepseek-v3 deepseek-r1 qwen3

Python 4.12 k

7 天前

gpustack / gpustack

#大语言模型#Simple, scalable AI model deployment on GPU clusters

ascend CUDA deepseek distributed-inference genai inference llama llamacpp 大语言模型 maas metal openai qwen rocm vllm mindie llm-inference llm-serving local-ai heterogeneous-cluster

Python 2.92 k

4 天前

containers / ramalama

#大语言模型#RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of con...

人工智能 containers CUDA Docker hip inference-server intel llamacpp 大语言模型 podman vllm

Python 1.76 k

5 天前

bricks-cloud / BricksLLM

#大语言模型#🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI...

Go 大语言模型 openai 人工智能 anthropic Azure gpt PostgreSQL REST API ycombinator API Docker 隐私安全 generative-ai Open Source 自托管 vllm

Go 1.06 k

5 个月前

substratusai / kubeai

#大语言模型#AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

Kubernetes 大语言模型 openai-api autoscaler ollama vllm ollama-operator vllm-operator 人工智能 Whisper faster-whisper

Go 987

4 天前

prometheus-eval / prometheus-eval

#大语言模型#Evaluate your LLM's response with Prometheus and GPT4 💯

evaluation 大语言模型 llmops Python vllm gpt4 llm-as-a-judge

Python 954

2 个月前

mostlygeek / llama-swap

Model swapping for llama.cpp (or any local OpenAPI compatible server)

Go llama llamacpp localllama localllm openai openai-api vllm

Go 876

7 天前

harleyszhang / llm_note

#大语言模型#LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

大语言模型 llm-inference vllm cuda-programming kv-cache transformer-models

Python 778

5 天前

apconw / sanic-web

#大语言模型#一个轻量级、支持全链路且易于二次开发的大模型应用项目(Large Model Data Assistant) 支持DeepSeek/Qwen2.5等大模型基于 Dify 、Ollama&Vllm、Sanic 和 Text2SQL 📊 等技术构建的一站式大模型应用开发项目，采用 Vue3、TypeScript 和 Vite 5 打造现代UI。它支持通过 ECharts 📈 实现基于大模型的数据...

人工智能 bigdata ChatGPT dify ollama rag vllm chat 大语言模型 qwen echarts sanic text2sql Vue.js Python deepseek-r1 mcp

JavaScript 773

5 天前

vllm-project / vllm-ascend

#大语言模型#Community maintained hardware plugin for vLLM on Ascend

ascend inference 大语言模型 llm-serving llmops mlops model-serving transformer vllm

Python 758

1 天前

ModelCloud / GPTQModel

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

gptq peft quantization transformers vllm

Python 606

18 天前

pgalko / BambooAI

#大语言模型#A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

大语言模型 openai-api pandas Python 人工智能数据分析数据科学 ai-agents vector-database pinecone gemini groq mistral ollama anthropic vllm Docker

Python 605

6 天前

mustafaaljadery / llama3v

A SOTA vision model built on top of llama3 8B.

llama llama3 vllm

Python 586

1 年前

jakobdylanc / llmcord

#前端开发#Make Discord your LLM frontend ● Supports any OpenAI compatible API (Ollama, LM Studio, vLLM, OpenRouter, xAI, Mistral, Groq and more)

gpt openai gpt-4 Discord 聊天机器人大语言模型 Bot ollama llama3 llama mistral groq xai grok vllm 前端 chat llama4 gemini

Python 580

3 天前

ModelTC / llmc

#大语言模型#[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

部署大语言模型 pruning quantization 工具 benchmark evaluation large-language-models internlm2 llama3 smoothquant post-training-quantization mixtral vllm

Python 486

6 天前

varunvasudeva1 / llm-server-docs

#大语言模型#Documentation on setting up an LLM server on Debian from scratch, using Ollama/vLLM, Open WebUI, OpenedAI Speech/Kokoro FastAPI, and ComfyUI.

Linux 大语言模型 ollama Server open-webui Debian comfyui vllm

472

2 个月前