pytorch 5 pipeline-parallelism 4 llama 3 sequence-parallelism 3 model-parallelism 3 machine-learning 3 deep-learning 3 distributed-systems 2 llm-training 2 transformer 2 llm 2 distributed-training 2 data-parallelism 2 large-language-models 2 transformers 2 jax 2 gpt 2 zero-1 1 fine-tuning 1 instruction-tuning 1 llm-finetuning 1 moe 1 mixture-of-experts 1 pjit 1 phi-3 1 pre-training 1 910b 1 deepspeed-ulysses 1 flash-attention 1 gemma 1 internlm 1 internlm2 1 llama3 1 llava 1 llm-framework 1 multi-modal 1 ring-attention 1 transformers-models 1 zero3 1 shmap 1 tpu 1 lora 1 fsdp 1 xla 1 zero 1 bloom 1 chatbot 1 falcon 1 guanaco 1 language-models 1 mixtral 1 neural-networks 1 nlp 1 pretrained-models 1 volunteer-computing 1 3d-parallelism 1 distributed-optimizers 1 huggingface-transformers 1 large-scale-language-modeling 1 megatron 1 megatron-lm 1