megatron-lm 12 llm 5 llm-training 4 transformers 4 deepspeed 4 pytorch 4 llama 4 megatron 3 huggingface-transformers 2 gpt 2 mixture-of-experts 2 nlp 2 deepseek 1 zero-1 1 tensor-parallelism 1 sequence-parallelism 1 pipeline-parallelism 1 moe 1 model-parallelism 1 large-scale-language-modeling 1 annotated-paper 1 fine-tuning 1 npu 1 pretrain 1 qwen 1 qwen-vl 1 sft 1 simplification 1 transformer 1 wrap 1 wrapper 1 llama2 1 gpt-neox 1 tokenizers 1 bert 1 deep-learning 1 gpt-2 1 machine-learning 1 papers-annotations 1 research-paper 1 xlnet 1 distributed-computing 1 distributed-systems 1 large-language-models 1 large-scale-machine-learning 1 llm-framework 1 reinforcement-learning 1 reinforcement-learning-from-human-feedback 1 distributed-training 1 pretraining 1 gke 1 nvidia 1 nvidia-gpu 1 nvidia-nemo 1 3d-parallelism 1 data-parallelism 1 distributed-optimizers 1