Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / EleutherAI / gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EleutherAI%2Fgpt-neox

Stars: 6,535
Forks: 946
Open Issues: 77

License: apache-2.0
Language: Python
Repo Size: 110 MB
Dependencies: 44

Created: over 3 years ago
Updated: 28 days ago
Last pushed: 30 days ago
Last synced: 29 days ago

Topics: deepspeed-library, gpt-3, language-model, transformers

Files

Loading...

Readme

Loading...

Dependencies

requirements/requirements-dev.txt pypi

autopep8 ==1.5.6 development
clang-format ==13.0.1 development
pre-commit * development
pytest ==6.2.3 development
pytest-cov ==2.11.1 development
pytest-forked ==1.3.0 development
pytest-xdist * development
transformers * development

requirements/requirements-onebitadam.txt pypi

cupy-cuda111 ==8.6.0

requirements/requirements-sparseattention.txt pypi

triton ==0.4.2

requirements/requirements-tensorboard.txt pypi

tensorboard ==2.5.0

requirements/requirements.txt pypi

deepspeed eb7f5cff36678625d23db8a8fe78b4a93e5d2c75
einops ==0.3.0
ftfy ==6.0.1
lm_dataformat ==0.0.20
lm_eval ==0.2.0
mpi4py ==3.0.3
numpy ==1.22.0
pybind11 ==2.6.2
regex *
sentencepiece *
six *
tokenizers ==0.10.2
transformers *
wandb ==0.10.28

.github/workflows/cpu_ci.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/docker_build.yml actions

actions/checkout v2 composite
crazy-max/ghaction-docker-meta v1 composite
docker/build-push-action v2 composite
docker/login-action v1 composite
docker/setup-buildx-action v1 composite
docker/setup-qemu-action v1 composite

.github/workflows/pull_request.yml actions

actions/checkout v2 composite
actions/checkout v3 composite
actions/setup-python v2 composite
pre-commit/action v2.0.3 composite

Dockerfile docker

nvidia/cuda 11.1.1-devel-ubuntu20.04 build

requirements/requirements-flashattention.txt pypi

flash-attn ==0.2.2

megatron/fused_kernels/setup.py pypi

requirements/requirements-s3.txt pypi

boto3 *
hf-transfer >=0.1.3

requirements/requirements-wandb.txt pypi

wandb >=0.10.28

.github/workflows/coverity_scan.yml actions

actions/checkout v2 composite
actions/upload-artifact v3 composite