distributed-inference · GitHub Topics

#大语言模型#Simple, scalable AI model deployment on GPU clusters

ascend CUDA deepseek distributed-inference genai inference llama llamacpp 大语言模型 maas metal openai qwen rocm vllm mindie llm-inference llm-serving local-ai heterogeneous-cluster

Python 2.92 k

3 天前

Lizonghang / prima.cpp

prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters

distributed-ai llm-inference on-device-llms llama-cpp distributed-inference

C++ 934

9 天前

mzbac / mlx_sharding

Distributed Inference for mlx LLm

MLX distributed-inference

Python 92

10 个月前

ipc-lab / collaborative-inference-oac

#计算机科学#Source code of the paper "Private Collaborative Edge Inference via Over-the-Air Computation".

differential-privacy distributed-inference ensemble-learning 机器学习

Python 3

5 个月前

JiangkaiWu / Attribute_Reid

Official impl. of ACM MM paper "Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds". A distributed inference model for pedestrian attribute recognition with...

distributed-inference edge-computing re-identification

Python 2

5 年前