deepspeed · GitHub Topics

InternLM / lmdeploy

#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

cuda-kernels deepspeed fastertransformer llm-inference turbomind internlm llama 大语言模型 codellama llama2 llama3

Python 6.52 k

2 天前

PKU-Alignment / safe-rlhf

#数据仓库#Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

ai-safety alpaca 数据集 deepspeed large-language-models llama 大语言模型 reinforcement-learning reinforcement-learning-from-human-feedback rlhf transformers vicuna safety gpt transformer beaver

Python 1.49 k

1 年前

zjunlp / KnowLM

#计算机科学#An Open-sourced Knowledgable Large Language Model Framework.

llama large-language-models pre-trained-language-models language-model instruction-following 深度学习中文 english instructions models reasoning gpt-3 deepspeed instruction-tuning lora pre-training pre-trained-model

Python 1.32 k

5 个月前

antgroup / glake

#大语言模型#GLake: optimizing GPU memory management and IO transmission.

deepspeed gpu 大语言模型 memory onnx PyTorch

Python 466

3 个月前

Coobiw / MPP-LLaVA

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...

multimodal-large-language-models deepspeed pipeline-parallelism mllm qwen fine-tuning pretraining

Jupyter Notebook 452

3 个月前

shm007g / LLaMA-Cult-and-More

#大语言模型#Large Language Models for All, 🦙 Cult and More, Stay in touch !

alpaca ChatGPT gpt llama ggml gpt4 gptq vicuna PyTorch Tensorflow transformers deepspeed 大语言模型

HTML 446

2 年前

Xirider / finetune-gpt2xl

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and finetune GPT-NEO (2.7 B) on a single GPU with Huggingface Transformers using DeepSpeed

huggingface huggingface-transformers deepspeed gpt2 gpt3 finetuning gpt-neo

Python 437

2 年前

LambdaLabsML / distributed-training-guide

Best practices & guides on how to write distributed pytorch training code

CUDA deepspeed distributed-training gpu gpu-cluster kuberentes nccl PyTorch slurm cluster mpi sharding

Python 435

4 个月前

OpenMOSS / CoLLiE

#自然语言处理#Collaborative Training of Large Language Models in an Efficient Way

深度学习 deepspeed 自然语言处理 PyTorch

Python 415

10 个月前

openpsi-project / ReaLHF

#大语言模型#Super-Efficient RLHF Training of LLMs with Parameter Reallocation

大语言模型 llm-training reinforcement-learning-from-human-feedback reinforcement-learning distributed-systems distributed-computing large-language-models llm-framework deepspeed transformers

Python 300

2 个月前

sunzeyeah / RLHF

#自然语言处理#Implementation of Chinese ChatGPT

ChatGPT 深度学习 deepspeed glm 自然语言处理 PyTorch

Python 286

2 年前

stanleylsx / llms_tool

一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测，低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。

baichuan bloom chatglm falcon internlm llama llama2 moss qwen chatglm2 PyTorch deepspeed baichuan2 mistral chatglm3

Python 217

2 年前

git-cloner / llama2-lora-fine-tuning

llama2 finetuning with deepspeed and lora

deepspeed finetuning llama2 lora

Python 174

2 年前

bobo0810 / LearnDeepSpeed

DeepSpeed教程 & 示例注释 & 学习笔记（大模型高效训练）

deepspeed Example large-language-models

Python 164

2 年前

jackaduma / ChatGLM-LoRA-RLHF-PyTorch

#大语言模型#A full pipeline to finetune ChatGLM LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the ChatGLM architecture. Basically ChatG...

lora chatglm chatglm-6b ChatGPT finetune gpt 大语言模型 PyTorch rlhf llama deepspeed peft ppo

Python 136

2 年前

HomebrewML / revlib

#计算机科学#Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload

PyTorch 深度学习 deepspeed xla tpu

Python 127

3 年前

CoinCheung / gdGPT

#自然语言处理#Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

deepspeed 大语言模型 pipeline 自然语言处理 PyTorch bloom flash-attention baichuan2-7b mixtral-8x7b llama2

Python 96

1 年前

OpenCSGs / llm-inference

llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource...

deepspeed llama-cpp llm-inference ray transformer vllm

Python 81

1 年前