model-inference-service · GitHub Topics

#大语言模型#The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

model-serving mlops llmops generative-ai llm-inference model-inference-service inference-platform 深度学习 llm-serving 机器学习 Python multimodal ml-engineering 大语言模型 ai-inference

Python 7.87 k

2 天前

bentoml / CLIP-API-service

CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

ai-applications clip cloud-native mlops model-inference model-inference-service model-serving

Jupyter Notebook 65

1 年前

bentoml / transformers-nlp-service

#自然语言处理#Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more

大语言模型 mlops model-deployment model-inference-service model-serving 自然语言处理 transformer llmops

Python 44

1 年前

ksm26 / Efficiently-Serving-LLMs

Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Pred...

batch-processing inference-optimization machine-learning-operations model-inference-service model-serving text-generation

Jupyter Notebook 17

1 年前