local-inference

#大语言模型#PowerInfer 是一个快速的、可运行在消费级GPU、个人电脑上的大模型服务

#大语言模型#[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

#大语言模型#Tool for test diferents large language models without code.

Local LLM Inference Library

#自然语言处理#LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).

Lightweight 6GB VRAM Gradio web app with auto-installer for running AuraFlow locally — no cloud, no clutter.

#网络爬虫#Verify claims using AI agents that debate using scraped evidence and local language models.

#大语言模型#script which performs RAG and use a local LLM for Q&A

Script which takes a .wav audio file, performs speech-to-text using OpenAI/Whisper, and then, using Llama3, summarization and action point from the transcript generated