llm-observability · GitHub Topics

#大语言模型#🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

analytics 大语言模型 llmops large-language-models openai 自托管 ycombinator 监控 observability Open Source langchain llama-index evaluation prompt-engineering prompt-management playground llm-evaluation llm-observability autogen

TypeScript 14.47 k

1 小时前

comet-ml / opik

#大语言模型#Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Open Source langchain openai playground prompt-engineering llama-index 大语言模型 llm-evaluation llm-observability llmops

Python 11.93 k

10 小时前

Helicone / helicone

#大语言模型#🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

large-language-models prompt-engineering agent-monitoring analytics evaluation gpt langchain llama-index 大语言模型 llm-cost llm-evaluation llm-observability llmops 监控 Open Source openai playground prompt-management ycombinator

TypeScript 4.25 k

10 小时前

coze-dev / coze-loop

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to...

agent 人工智能 agentops evaluation langchain llmops 监控 observability Open Source openai playground prompt-management llm-observability coze

Go 4.24 k

1 天前

Agenta-AI / agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

llm-tools prompt-engineering prompt-management llm-evaluation llm-framework rag-evaluation llm-observability llm-as-a-judge llm-monitoring llm-platform llm-playground llmops-platform

Python 3.03 k

1 天前

VoltAgent / voltagent

#大语言模型#Open Source TypeScript AI Agent Framework

agents 人工智能 chatbots 大语言模型 mcp Node.js TypeScript JavaScript tts 框架 autogen llm-observability observability

TypeScript 2.55 k

21 小时前

lmnr-ai / lmnr

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

aiops developer-tools observability agents 人工智能 Rust analytics llm-evaluation llm-observability 监控 Open Source 自托管 ai-observability llmops evals evaluation TypeScript ts

TypeScript 2.2 k

19 小时前

JudgmentLabs / judgeval

#大语言模型#The open source post-building layer for agents. Our traces + evals power agent post-training (RL, SFT), monitoring, and regression testing.

langchain langgraph llama-index 大语言模型 llm-evaluation llm-observability Open Source openai prompt-engineering agent agentic-ai agents

Python 619

1 天前

palico-ai / palico-ai

#大语言模型#Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework

langchain langchain-js llamaindex 大语言模型 llm-agent llm-evaluation llm-framework rag JavaScript TypeScript llm-observability autogen anthropic openai 人工智能 Node.js full-stack Docker

TypeScript 342

8 个月前

langfuse / oss-llmops-stack

#大语言模型#Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability,...

ai-gateway gateway 大语言模型 llm-evaluation llm-gateway llm-observability llmops 监控 openai-proxy prompt-management 自托管 ycombinator Open Source

107

5 个月前

cyberark / agentwatch

#大语言模型#A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with m...

agent 人工智能 Cybersecurity large-language-models 大语言模型机器学习监控 observability 安全 agentic agentic-ai llm-framework llm-observability llm-tools

Python 90

3 个月前

radicalbit / radicalbit-ai-monitoring

#计算机科学#A comprehensive solution for monitoring your AI models in production

人工智能 ai-monitoring data-drift 数据科学机器学习 machine-learning-engineering 监控 observability ai-observability ml-observability llm-observability

Python 83

1 个月前

myscale / myscale-telemetry

Open-source observability for your LLM application.

callback langchain llm-observability 监控 Python

Python 53

7 个月前

langfuse / langfuse-java

#大语言模型#🪢 Auto-generated Java Client for Langfuse API

analytics Java large-language-models 大语言模型 llm-observability llmops 监控 observability Open Source openai prompt-engineering sdk-java 自托管 ycombinator

Java 21

1 个月前

teilomillet / hapax

#大语言模型#The reliability layer between your code and LLM providers.

anthropic infrastructure Infrastructure as code 大语言模型监控 observability openai llm-observability llmops Open Source llm-gateway

Go 19

7 个月前

AndrMoura / streamlit-chatbot-analytics

#大语言模型#Streamlit-based chatbot leveraging Ollama via LangChain and PostHog-LLM for advanced logging and monitoring

analytics 聊天机器人 chatbot-application chatbots langchain llama3 大语言模型 llm-ops ollama Streamlit llm-observability

Python 7

1 年前

matdev83 / llm-accounting

A Python package for tracking and analyzing LLM usage across different models and applications. It is primarily designed as a library for integration into development process of LLM-based agentic work...

agentic-ai agentic-ai-development agentic-workflow 大语言模型 mlops llm-observability llm-ops

Python 4

24 天前

modelmetry / modelmetry-sdk-js

#大语言模型#The Modelmetry JS/TS SDK allows developers to easily integrate Modelmetry’s advanced guardrails and monitoring capabilities into their LLM-powered applications.

ai-observability evals guardrails large-language-models 大语言模型 llm-observability 监控 observability tracing

TypeScript 1

6 个月前

Dennauss / volta

#大语言模型#Volta is a Go library for multi-tenant encryption and secrets management, offering embedded data protection without external dependencies. 🔒🚀

agents autogen cloud energy-monitor file-sharing 框架 JavaScript 大语言模型 llm-observability LLVM mcp 移动 observability Python PyTorch storage text2image TypeScript

Go 0

1 个月前

Tonyslebew / evals

#大语言模型#Evaluate and compare AI language models on coding tasks with Evals. Run structured tests, integrate usage rules, and generate detailed reports. 🚀🤖

analytics autogen ccp Edge Internet of things llama-index 大语言模型 llm-evaluation llm-observability llmops 监控 PHP rag 自托管 ycombinator

Elixir 0

1 个月前