tensor-parallelism 10 pytorch 5 pipeline-parallelism 5 llama 3 deep-learning 3 machine-learning 3 model-parallelism 3 sequence-parallelism 3 jax 2 transformers 2 transformer 2 data-parallelism 2 distributed-systems 2 distributed-training 2 gpt 2 llm-training 2 llm 2 large-language-models 2 llm-framework 1 multi-modal 1 ring-attention 1 llava 1 llama3 1 internlm2 1 internlm 1 gemma 1 flash-attention 1 chunked-prefill 1 transformers-models 1 zero3 1 fine-tuning 1 instruction-tuning 1 llm-finetuning 1 phi-3 1 pre-training 1 fsdp 1 xla 1 zero 1 lora 1 pjit 1 shmap 1 tpu 1 continuous-batching 1 llm-inference 1 llm-serving 1 pagedattention 1 qwen3 1 token-throttling 1 bloom 1 chatbot 1 falcon 1 guanaco 1 language-models 1 mixtral 1 neural-networks 1 nlp 1 pretrained-models 1 volunteer-computing 1 3d-parallelism 1 distributed-optimizers 1 huggingface-transformers 1 large-scale-language-modeling 1 megatron 1 megatron-lm 1 mixture-of-experts 1 moe 1 zero-1 1 910b 1 deepspeed-ulysses 1