llm-evaluation

#计算机科学#MLflow 是一个开源框架，旨在管理整个机器学习生命周期。 它可以在不同的平台上训练模型并为模型提供服务，让你能够使用相同的一组工具，而不管试验是在计算机本地、远程计算目标上、虚拟机上

#大语言模型#🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

#大语言模型#Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

DeepEval 是大语言模型评估框架，专为评估和测试大语言模型系统而设计。它类似于 Pytest，但专注于对 LLM 输出进行单元测试。

#大语言模型#Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with comma...

#数据仓库#AI Observability & Evaluation

the LLM vulnerability scanner

#大语言模型#🐢 Open-Source Evaluation & Testing library for LLM Agents

ReLE中文大模型能力评测（持续更新）：目前已囊括291个大模型，覆盖chatgpt、gpt-5、o4-mini、谷歌gemini-2.5、Claude4、智谱GLM-Z1、文心一言、qwen-max、百川、讯飞星火、商汤senseChat、minimax等商用模型， 以及kimi-k2、ernie4.5、minimax-M1、DeepSeek-R1-0528、deepseek-v3.1、qwen...

#大语言模型#🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

#大语言模型#AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

#大语言模型#The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

#计算机科学#Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

Build, enrich, and transform datasets using AI models with no code

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandab...

#大语言模型#UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

#大语言模型#The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.