ai-infra

🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys...

#大语言模型#A.I.G (AI-Infra-Guard) is a comprehensive, intelligent, and easy-to-use AI Red Teaming platform developed by Tencent Zhuque Lab.

#大语言模型#SpargeAttention: A training-free sparse attention that can accelerate any model inference.

AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识

RLinf is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning.

High performance distributed cache system. Built by Rust.

#计算机科学#Transform your pythonic research to an artifact that engineers can deploy easily.

This is a landscape of the infrastructure that powers the generative AI ecosystem

#计算机科学#Cloud Native ML/DL Platform

Triton multi-level runner, include cubin, ptx, ttgir etc.

#大语言模型#hpc 教程，包含集合通信(mpi、nccl)、cuda 编程、向量化 SIMD、RDMA 通信等

#大语言模型#TME: Structured memory engine for LLM agents to plan, rollback, and reason across multi-step tasks.

vgpu.rs is the fractional GPU & vgpu-hypervisor implementation written in Rust

#Awesome#A curated list of awesome tools, frameworks, platforms, and resources for building scalable and efficient AI infrastructure, including distributed training, model serving, MLOps, and deployment.

Triton for OpenCL backend, and use mlir-translate to get source OpenCL code

OriginDL: A distributed deep learning framework Built from scratch

#Awesome#This repository contains a list of various service-specific Azure Landing Zone implementation options.

Memory Management Service, a Long Term Memory Solution for AI

#计算机科学#The Lisa programming language.

#大语言模型#TME: Structured memory engine for LLM agents to plan, rollback, and reason across multi-step tasks. DAG upgrade in progress.