llm-observability · GitHub Topics

#大语言模型#🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

analytics 大语言模型 llmops large-language-models openai 自托管 ycombinator 监控 observability Open Source langchain llama-index evaluation prompt-engineering prompt-management playground llm-evaluation llm-observability autogen

TypeScript 12.61 k

3 小时前

comet-ml / opik

#大语言模型#Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Open Source langchain openai playground prompt-engineering llama-index 大语言模型 llm-evaluation llm-observability llmops

Python 9.76 k

7 小时前

Helicone / helicone

#大语言模型#🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

large-language-models prompt-engineering agent-monitoring analytics evaluation gpt langchain llama-index 大语言模型 llm-cost llm-evaluation llm-observability llmops 监控 Open Source openai playground prompt-management ycombinator

TypeScript 3.92 k

19 小时前

Agenta-AI / agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

llm-tools prompt-engineering prompt-management llm-evaluation llm-framework rag-evaluation llm-observability llm-as-a-judge llm-monitoring llm-platform llm-playground llmops-platform

Python 2.83 k

2 天前

lmnr-ai / lmnr

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

aiops developer-tools observability agents 人工智能 Rust analytics llm-evaluation llm-observability 监控 Open Source 自托管 ai-observability llmops evals evaluation TypeScript ts

TypeScript 2.07 k

2 天前

VoltAgent / voltagent

#大语言模型#Open Source TypeScript AI Agent Framework

agents 人工智能 chatbots 大语言模型 mcp Node.js TypeScript JavaScript tts 框架 autogen llm-observability observability

TypeScript 2.04 k

4 天前

palico-ai / palico-ai

#大语言模型#Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework

langchain langchain-js llamaindex 大语言模型 llm-agent llm-evaluation llm-framework rag JavaScript TypeScript llm-observability autogen anthropic openai 人工智能 Node.js full-stack Docker

TypeScript 340

7 个月前

langfuse / oss-llmops-stack

#大语言模型#Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability,...

ai-gateway gateway 大语言模型 llm-evaluation llm-gateway llm-observability llmops 监控 openai-proxy prompt-management 自托管 ycombinator Open Source

104

4 个月前

JudgmentLabs / judgeval

#大语言模型#Rigorously test, monitor, and optimize agent systems—grounded in research from Stanford and Berkeley AI Labs.

langchain langgraph llama-index 大语言模型 llm-evaluation llm-observability Open Source openai prompt-engineering agent agentic-ai agents

Python 84

5 天前

cyberark / agentwatch

#大语言模型#A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with m...

agent 人工智能 Cybersecurity large-language-models 大语言模型机器学习监控 observability 安全 agentic agentic-ai llm-framework llm-observability llm-tools

Python 83

1 个月前

radicalbit / radicalbit-ai-monitoring

#计算机科学#A comprehensive solution for monitoring your AI models in production

人工智能 ai-monitoring data-drift 数据科学机器学习 machine-learning-engineering 监控 observability ai-observability ml-observability llm-observability

Python 81

12 天前

myscale / myscale-telemetry

Open-source observability for your LLM application.

callback langchain llm-observability 监控 Python

Python 52

5 个月前

teilomillet / hapax

#大语言模型#The reliability layer between your code and LLM providers.

anthropic infrastructure Infrastructure as code 大语言模型监控 observability openai llm-observability llmops Open Source llm-gateway

Go 18

5 个月前

langfuse / langfuse-java

#大语言模型#🪢 Auto-generated Java Client for Langfuse API

analytics Java large-language-models 大语言模型 llm-observability llmops 监控 observability Open Source openai prompt-engineering sdk-java 自托管 ycombinator

Java 12

3 个月前

AndrMoura / streamlit-chatbot-analytics

#大语言模型#Streamlit-based chatbot leveraging Ollama via LangChain and PostHog-LLM for advanced logging and monitoring

analytics 聊天机器人 chatbot-application chatbots langchain llama3 大语言模型 llm-ops ollama Streamlit llm-observability

Python 6

1 年前

matdev83 / llm-accounting

A Python package for tracking and analyzing LLM usage across different models and applications. It is primarily designed as a library for integration into development process of LLM-based agentic work...

agentic-ai agentic-ai-development agentic-workflow 大语言模型 mlops llm-observability llm-ops

Python 2

5 天前

modelmetry / modelmetry-sdk-js

#大语言模型#The Modelmetry JS/TS SDK allows developers to easily integrate Modelmetry’s advanced guardrails and monitoring capabilities into their LLM-powered applications.

ai-observability evals guardrails large-language-models 大语言模型 llm-observability 监控 observability tracing

TypeScript 1

5 个月前