Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
Package Usage: pypi: lm_dataformat
A utility for storing and reading files for LM training.
20 versions
Latest release: over 2 years ago
2 dependent packages
12,928 downloads last month
View more package details: https://packages.ecosyste.ms/registries/pypi.org/packages/lm_dataformat
View more repository details: https://repos.ecosyste.ms/hosts/GitHub/repositories/leogao2%2Flm_dataformat
Dependent Repos 223
EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of language models.- ==0.0.20 setup.py
Size: 22.6 MB - Last synced: 4 days ago - Pushed: 4 days ago
EleutherAI/the-pile
- * requirements-dev.txt
Size: 259 KB - Last synced: 3 months ago - Pushed: about 1 year ago
CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57- * requirements_scripts.txt
Size: 17.3 MB - Last synced: 25 days ago - Pushed: over 2 years ago
EleutherAI/gpt-neo 📦
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.- * requirements.txt
Size: 1.56 MB - Last synced: 25 days ago - Pushed: about 2 years ago
EleutherAI/openwebtext2
- * requirements.txt
Size: 5.32 MB - Last synced: 3 months ago - Pushed: almost 2 years ago
EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.- ==0.0.20 requirements/requirements.txt
Size: 110 MB - Last synced: about 1 month ago - Pushed: about 1 month ago
pxu-mdsol/gpt-neox
- ==0.0.19 requirements/requirements.txt
Size: 36.1 MB - Last synced: about 1 year ago - Pushed: almost 2 years ago
google-research/t5x
- ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt
Size: 9.47 MB - Last synced: about 2 months ago - Pushed: about 2 months ago
ShivanshuPurohit/docker-image
Docker image that enables EFA and works with gpt-neox- ==0.0.20 helpers/requirements.txt
Size: 4.89 MB - Last synced: 9 months ago - Pushed: over 1 year ago
EleutherAI/DALLE-mtf
Open-AI's DALL-E for large scale training in mesh-tensorflow.- * requirements.txt
Size: 272 KB - Last synced: 29 days ago - Pushed: over 2 years ago
adrwong/gpt-j-tuned-for-fin
A fine-tuned gpt-j model for quotes generation, testing use only- * requirements.txt
Size: 3.23 MB - Last synced: 8 months ago - Pushed: over 2 years ago
igor0/gpt-neox
- ==0.0.19 requirements/requirements.txt
Size: 38.3 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
aishwaryaanegundi/lm-evaluation-harness Fork of OpenGPTX/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.- ==0.0.20 setup.py
Size: 9.32 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
m-abdelgaber/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 126 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
keyanvakil/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Last synced: over 1 year ago
venkatesh-narayan/gpt2-nonpara
- ==0.0.20 lm-evaluation-harness/setup.py
Size: 33.3 MB - Last synced: about 1 year ago - Pushed: almost 2 years ago
EleutherAI/pile-enron-emails
A script for collecting the Enron Emails dataset in a language modelling friendly format.- * requirements.txt
Size: 6.84 KB - Last synced: about 1 year ago - Pushed: about 3 years ago
sdtblck/lm_dataloader
Dataloader tools for language modelling- ==0.0.20 requirements.txt
- ==0.0.20 setup.py
Size: 71.3 KB - Last synced: 24 days ago - Pushed: over 2 years ago
EleutherAI/visual-grounding
Visually ground GPT-Neo 1.3b and 2.7b- * requirements.txt
Size: 124 KB - Last synced: about 1 year ago - Pushed: about 3 years ago
tensorfork/OBST
Your fruity companion for transformers- * requirements.txt
Size: 1.22 MB - Last synced: about 1 year ago - Pushed: about 2 years ago
cfoster0/CLAP 📦
Contrastive Language-Audio Pretraining- * setup.py
Size: 171 KB - Last synced: 12 months ago - Pushed: about 2 years ago
losttech/lm-evaluation-harness Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.- ==0.0.20 setup.py
Size: 9.17 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
Cognation/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Fork for fine tuning- * requirements.txt
Size: 358 KB - Last synced: about 1 month ago - Pushed: over 1 year ago
47-studio-org/mesh-transformer-jax Fork of MarcelRaschke/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 331 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
microsoft/deepspeed-gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.- ==0.0.20 requirements/requirements.txt
Size: 59.3 MB - Last synced: about 2 months ago - Pushed: over 1 year ago
47-studio-org/mesh-transformer-jax-1 Fork of 47-studio-org/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 324 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
Lisennlp/gpt-neox-j
基于gpt-neox修改为支持huggingface的gptj训练- ==0.0.20 requirements/requirements.txt
Size: 1.08 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
speakleash/speakleash
- * requirements.txt
- * setup.py
Size: 108 KB - Last synced: 20 days ago - Pushed: 21 days ago
speakleash/speakleash-extractor-wolne-lektury
- * requirements.txt
Size: 6.84 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
Daasin/codepy-lot Fork of CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57- * requirements_scripts.txt
Size: 17.1 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
Eichhof/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 348 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
SahilJain314/t5x Fork of google-research/t5x
- ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt
Size: 8.89 MB - Last synced: about 1 month ago - Pushed: 9 months ago
Mu-L/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 314 KB - Last synced: about 2 months ago - Pushed: 3 months ago
dumpmemory/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 314 KB - Last synced: about 2 months ago - Pushed: 3 months ago
Xen0byte/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 314 KB - Last synced: about 1 month ago - Pushed: 3 months ago
stungkit/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 319 KB - Last synced: about 1 month ago - Pushed: over 1 year ago
ghomasHudson/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Training GPT- * requirements.txt
Size: 379 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
minipilot-team/minipilot-gpt-j Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 348 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
OpenGPTX/lm-evaluation-harness Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.- ==0.0.20 setup.py
Size: 21.6 MB - Last synced: 29 days ago - Pushed: about 1 month ago
kokizzu/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 318 KB - Last synced: about 1 month ago - Pushed: over 1 year ago
lipovsek/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.- ==0.0.19 requirements/requirements.txt
Size: 36.5 MB - Last synced: about 1 year ago - Pushed: about 1 year ago
Tiamat-Tech/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.- ==0.0.20 requirements/requirements.txt
Size: 36.3 MB - Last synced: about 1 month ago - Pushed: about 2 months ago
kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 318 KB - Last synced: 5 days ago - Pushed: over 1 year ago
mathemakitten/t5x Fork of google-research/t5x
t5x, but outside of google- ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt
Size: 7.18 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
bmosaicml/lm-evaluation-harness-composergpt-integration Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.- ==0.0.20 setup.py
Size: 9.28 MB - Last synced: 29 days ago - Pushed: over 1 year ago
Napkin-DL/gpt-neox-on-sagemaker
- ==0.0.20 gpt-neox/requirements/requirements.txt
- ==0.0.20 gpt-neox/requirements.txt
Size: 223 KB - Last synced: about 1 year ago - Pushed: about 2 years ago
mgrankin/mesh-transformer-jax-clip
A fork of kingoflolz/mesh-transformer-jax to train GPT with CLIP supervision- * requirements.txt
Size: 274 KB - Last synced: about 1 year ago - Pushed: over 2 years ago
mgrankin/gpt-c
Train GPT model helped by CLIP- * environment.yml
Size: 1.35 MB - Last synced: about 1 year ago - Pushed: over 2 years ago
okbuddyhololive/mesh-transformer-jax
A fork of a library in JAX for model parallelism of transformers, for "Project Cybertard"- * requirements.txt
Size: 88.9 KB - Last synced: about 1 year ago - Pushed: about 2 years ago
lcw99/gpt-neo-ko
- * requirements.txt
Size: 2.7 MB - Last synced: 29 days ago - Pushed: over 1 year ago
Linyxus/code-gpt-neox
Faster fine-tuning GPT for code generation.- ==0.0.19 requirements/requirements.txt
Size: 194 KB - Last synced: about 1 year ago - Pushed: over 2 years ago
Dahoas/gpt-neox-finetuning
- ==0.0.19 requirements/requirements.txt
Size: 36.7 MB - Last synced: about 1 year ago - Pushed: about 2 years ago
HomebrewNLP/HomebrewNLP-MTF 📦
HomebrewNLP in Mesh-TensorFlow flavour for distributed TPU training- * requirements.txt
Size: 1.29 MB - Last synced: about 1 month ago - Pushed: over 1 year ago
mosmos6/mesh-transformer-jax3
test for v3- * requirements.txt
- * requirements_before_20230113.txt
- * requirements_original.txt
Size: 165 KB - Last synced: about 1 year ago - Pushed: about 1 year ago
mosmos6/mesh-transformer-jax7
finetune test of jax4 repo- * requirements.txt
- * requirements_before_20230113.txt
- * requirements_original.txt
Size: 236 KB - Last synced: about 1 year ago - Pushed: about 1 year ago
mosmos6/Large-MTJ
This is a modified version of https://github.com/VE-FORBRYDERNE/mesh-transformer-jax/tree/ck to be adapted to JAX 0.3.25 so this runs on colab with TPU_driver0.2- * requirements.txt
Size: 245 KB - Last synced: about 1 year ago - Pushed: about 1 year ago
86885974647565434342125458569795756674/prompt-tuning
- ==0.0.20 prompt-tuning/pile_requirements.txt
Size: 156 MB - Last synced: 9 months ago - Pushed: 9 months ago
leogao2/commoncrawl_downloader
- ==0.0.8 requirements.txt
Size: 36.1 KB - Last synced: about 1 year ago - Pushed: about 3 years ago
hyang5916/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.- * requirements.txt
Size: 1.41 MB - Last synced: 11 months ago - Pushed: almost 3 years ago
rbleattler/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 22.4 MB - Last synced: about 1 month ago - Pushed: about 1 month ago
Ralf12358/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 9.54 MB - Last synced: about 1 month ago - Pushed: about 1 year ago
pterameta/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 18.3 MB - Last synced: about 1 month ago - Pushed: 11 months ago
stancx1/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 18.9 MB - Last synced: 11 months ago - Pushed: 11 months ago
Gurix525/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.- * requirements.txt
Size: 1.56 MB - Last synced: 11 months ago - Pushed: about 2 years ago
sunilravilla/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 18.3 MB - Last synced: 11 months ago - Pushed: 11 months ago
tiger322/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 15.7 MB - Last synced: 11 months ago - Pushed: 11 months ago
iamianM/mesh-transformer-jax
- * requirements.txt
Size: 1.77 MB - Last synced: about 1 year ago - Pushed: almost 3 years ago
incredimo/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 15.5 MB - Last synced: 11 months ago - Pushed: 12 months ago
jllllll/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 19.3 MB - Last synced: 11 months ago - Pushed: 11 months ago
crjaensch/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 19.5 MB - Last synced: about 1 month ago - Pushed: 11 months ago
McClentyS/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 19.2 MB - Last synced: 11 months ago - Pushed: 11 months ago
qqq-tech/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT2& GPT3-like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library.- * requirements.txt
Size: 1.48 MB - Last synced: about 2 months ago - Pushed: about 3 years ago
qqq-tech/gpt-code-clippy Fork of CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57- * requirements_scripts.txt
Size: 17.3 MB - Last synced: about 2 months ago - Pushed: over 2 years ago
qqq-tech/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.- ==0.0.19 requirements/requirements.txt
Size: 59.5 MB - Last synced: about 2 months ago - Pushed: over 2 years ago
qqq-tech/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 10.1 MB - Last synced: about 2 months ago - Pushed: about 1 year ago
jialii/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 18.5 MB - Last synced: 10 months ago - Pushed: 10 months ago
csqr/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 9.55 MB - Last synced: 10 months ago - Pushed: about 1 year ago
jesusoctavioas/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 16.2 MB - Last synced: about 1 month ago - Pushed: 11 months ago
gchenfly/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT2& GPT3-like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library.- * requirements.txt
Size: 1.17 MB - Last synced: about 2 months ago - Pushed: over 3 years ago
gchenfly/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf- ==0.0.20 requirements.txt
- ==0.0.20 spaces/chatbot/requirements.txt
Size: 15.6 MB - Last synced: about 2 months ago - Pushed: 11 months ago
EC3-Gang/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku- * requirements.txt
Size: 420 KB - Last synced: 10 months ago - Pushed: 10 months ago
cch230/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai- ==0.0.20 requirements.txt
Size: 30.1 MB - Last synced: 8 months ago - Pushed: 8 months ago
sergefedorow/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 21.2 MB - Last synced: 10 months ago - Pushed: 10 months ago
mengyanyuan/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.- * requirements.txt
Size: 1.56 MB - Last synced: 10 months ago - Pushed: about 2 years ago
BiaoWu/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/- ==0.0.20 requirements.txt
Size: 20.8 MB - Last synced: 10 months ago - Pushed: 10 months ago
ozmode/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.- * requirements.txt
Size: 1.56 MB - Last synced: 10 months ago - Pushed: about 2 years ago