model-parallelism 11 pytorch 11 deep-learning 9 data-parallelism 8 machine-learning 6 tensor-parallelism 4 inference 4 large-scale 4 distributed-training 4 gpipe 3 transformer 3 distributed-systems 3 nlp 3 gpt 2 serving 2 tensorrt 2 gpu 2 large-language-models 2 llama 2 mixture-of-experts 2 sequence-parallelism 2 fine-tuning 2 pretraining 2 self-supervised-learning 2 deepspeed 2 transformers 2 neural-networks 2 deployment 2 transformers-models 1 distribution-strategy-planner 1 ring-attention 1 ray 1 multi-modal 1 llm-training 1 llm-framework 1 serve 1 triton-inference-server 1 llava 1 llama3 1 internlm2 1 internlm 1 gemma 1 flash-attention 1 deepspeed-ulysses 1 910b 1 torch2trt 1 t5 1 deep-neural-networks 1 hybrid-parallelism 1 big-model 1 distributed-computing 1 foundation-models 1 ai 1 zero 1 trillion-parameters 1 compression 1 billion-parameters 1 vision-transformer 1 oneflow 1 python 1 heterogeneous-training 1 hpc 1 pose-estimation 1 object-detection 1 llm-serving 1 multimodal 1 mlops 1 llms 1 image-classification 1 face-detection 1 data-science 1 zero3 1 3d-parallelism 1 volunteer-computing 1 pretrained-models 1 mixtral 1 language-models 1 guanaco 1 falcon 1 chatbot 1 bloom 1 parallelism 1 checkpointing 1 memory-efficient 1 treelstm 1 scheduling 1 reinforcement-learning 1 dynpartition 1 dynamic-neural-network 1 distributed-deep-learning 1 tensorflow 1 rust 1 pipedream 1 high-performance-computing 1 unsupervised-learning 1 paddlepaddle 1 paddlecloud 1 lightning 1 fleet-api 1 elastic 1