Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

Package Usage: pypi: lm_dataformat

A utility for storing and reading files for LM training.
20 versions
Latest release: over 2 years ago
2 dependent packages
12,928 downloads last month

View more package details: https://packages.ecosyste.ms/registries/pypi.org/packages/lm_dataformat

View more repository details: https://repos.ecosyste.ms/hosts/GitHub/repositories/leogao2%2Flm_dataformat

Dependent Repos 223

EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of language models.
  • ==0.0.20 setup.py

Size: 22.6 MB - Last synced: 4 days ago - Pushed: 4 days ago

EleutherAI/the-pile
  • * requirements-dev.txt

Size: 259 KB - Last synced: 3 months ago - Pushed: about 1 year ago

CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57
  • * requirements_scripts.txt

Size: 17.3 MB - Last synced: 25 days ago - Pushed: over 2 years ago

EleutherAI/gpt-neo 📦
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.56 MB - Last synced: 25 days ago - Pushed: about 2 years ago

EleutherAI/openwebtext2
  • * requirements.txt

Size: 5.32 MB - Last synced: 3 months ago - Pushed: almost 2 years ago

EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
  • ==0.0.20 requirements/requirements.txt

Size: 110 MB - Last synced: about 1 month ago - Pushed: about 1 month ago

pxu-mdsol/gpt-neox
  • ==0.0.19 requirements/requirements.txt

Size: 36.1 MB - Last synced: about 1 year ago - Pushed: almost 2 years ago

google-research/t5x
  • ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt

Size: 9.47 MB - Last synced: about 2 months ago - Pushed: about 2 months ago

ShivanshuPurohit/docker-image
Docker image that enables EFA and works with gpt-neox
  • ==0.0.20 helpers/requirements.txt

Size: 4.89 MB - Last synced: 9 months ago - Pushed: over 1 year ago

EleutherAI/DALLE-mtf
Open-AI's DALL-E for large scale training in mesh-tensorflow.
  • * requirements.txt

Size: 272 KB - Last synced: 29 days ago - Pushed: over 2 years ago

adrwong/gpt-j-tuned-for-fin
A fine-tuned gpt-j model for quotes generation, testing use only
  • * requirements.txt

Size: 3.23 MB - Last synced: 8 months ago - Pushed: over 2 years ago

igor0/gpt-neox
  • ==0.0.19 requirements/requirements.txt

Size: 38.3 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

aishwaryaanegundi/lm-evaluation-harness Fork of OpenGPTX/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
  • ==0.0.20 setup.py

Size: 9.32 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

m-abdelgaber/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 126 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

keyanvakil/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Last synced: over 1 year ago

venkatesh-narayan/gpt2-nonpara
  • ==0.0.20 lm-evaluation-harness/setup.py

Size: 33.3 MB - Last synced: about 1 year ago - Pushed: almost 2 years ago

EleutherAI/pile-enron-emails
A script for collecting the Enron Emails dataset in a language modelling friendly format.
  • * requirements.txt

Size: 6.84 KB - Last synced: about 1 year ago - Pushed: about 3 years ago

sdtblck/lm_dataloader
Dataloader tools for language modelling
  • ==0.0.20 requirements.txt
  • ==0.0.20 setup.py

Size: 71.3 KB - Last synced: 24 days ago - Pushed: over 2 years ago

EleutherAI/visual-grounding
Visually ground GPT-Neo 1.3b and 2.7b
  • * requirements.txt

Size: 124 KB - Last synced: about 1 year ago - Pushed: about 3 years ago

tensorfork/OBST
Your fruity companion for transformers
  • * requirements.txt

Size: 1.22 MB - Last synced: about 1 year ago - Pushed: about 2 years ago

cfoster0/CLAP 📦
Contrastive Language-Audio Pretraining
  • * setup.py

Size: 171 KB - Last synced: 12 months ago - Pushed: about 2 years ago

losttech/lm-evaluation-harness Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
  • ==0.0.20 setup.py

Size: 9.17 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

gb302/health-cascade-january-deliverable
  • * requirements2.txt

Last synced: over 1 year ago

Cognation/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Fork for fine tuning
  • * requirements.txt

Size: 358 KB - Last synced: about 1 month ago - Pushed: over 1 year ago

47-studio-org/mesh-transformer-jax Fork of MarcelRaschke/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 331 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

microsoft/deepspeed-gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
  • ==0.0.20 requirements/requirements.txt

Size: 59.3 MB - Last synced: about 2 months ago - Pushed: over 1 year ago

47-studio-org/mesh-transformer-jax-1 Fork of 47-studio-org/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 324 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

Lisennlp/gpt-neox-j
基于gpt-neox修改为支持huggingface的gptj训练
  • ==0.0.20 requirements/requirements.txt

Size: 1.08 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

speakleash/speakleash
  • * requirements.txt
  • * setup.py

Size: 108 KB - Last synced: 20 days ago - Pushed: 21 days ago

speakleash/speakleash-extractor-wolne-lektury
  • * requirements.txt

Size: 6.84 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

Daasin/codepy-lot Fork of CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57
  • * requirements_scripts.txt

Size: 17.1 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

Eichhof/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 348 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

SahilJain314/t5x Fork of google-research/t5x
  • ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt

Size: 8.89 MB - Last synced: about 1 month ago - Pushed: 9 months ago

Mu-L/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 314 KB - Last synced: about 2 months ago - Pushed: 3 months ago

dumpmemory/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 314 KB - Last synced: about 2 months ago - Pushed: 3 months ago

Xen0byte/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 314 KB - Last synced: about 1 month ago - Pushed: 3 months ago

stungkit/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 319 KB - Last synced: about 1 month ago - Pushed: over 1 year ago

ghomasHudson/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Training GPT
  • * requirements.txt

Size: 379 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

minipilot-team/minipilot-gpt-j Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 348 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

OpenGPTX/lm-evaluation-harness Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
  • ==0.0.20 setup.py

Size: 21.6 MB - Last synced: 29 days ago - Pushed: about 1 month ago

kokizzu/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 318 KB - Last synced: about 1 month ago - Pushed: over 1 year ago

lipovsek/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
  • ==0.0.19 requirements/requirements.txt

Size: 36.5 MB - Last synced: about 1 year ago - Pushed: about 1 year ago

Tiamat-Tech/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.
  • ==0.0.20 requirements/requirements.txt

Size: 36.3 MB - Last synced: about 1 month ago - Pushed: about 2 months ago

kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 318 KB - Last synced: 5 days ago - Pushed: over 1 year ago

mathemakitten/t5x Fork of google-research/t5x
t5x, but outside of google
  • ==0.0.20 t5x/contrib/gpu/scripts_gpu/pile_requirements.txt

Size: 7.18 MB - Last synced: about 1 year ago - Pushed: over 1 year ago

bmosaicml/lm-evaluation-harness-composergpt-integration Fork of EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
  • ==0.0.20 setup.py

Size: 9.28 MB - Last synced: 29 days ago - Pushed: over 1 year ago

Napkin-DL/gpt-neox-on-sagemaker
  • ==0.0.20 gpt-neox/requirements/requirements.txt
  • ==0.0.20 gpt-neox/requirements.txt

Size: 223 KB - Last synced: about 1 year ago - Pushed: about 2 years ago

mgrankin/mesh-transformer-jax-clip
A fork of kingoflolz/mesh-transformer-jax to train GPT with CLIP supervision
  • * requirements.txt

Size: 274 KB - Last synced: about 1 year ago - Pushed: over 2 years ago

mgrankin/gpt-c
Train GPT model helped by CLIP
  • * environment.yml

Size: 1.35 MB - Last synced: about 1 year ago - Pushed: over 2 years ago

okbuddyhololive/mesh-transformer-jax
A fork of a library in JAX for model parallelism of transformers, for "Project Cybertard"
  • * requirements.txt

Size: 88.9 KB - Last synced: about 1 year ago - Pushed: about 2 years ago

lcw99/gpt-neo-ko
  • * requirements.txt

Size: 2.7 MB - Last synced: 29 days ago - Pushed: over 1 year ago

Linyxus/code-gpt-neox
Faster fine-tuning GPT for code generation.
  • ==0.0.19 requirements/requirements.txt

Size: 194 KB - Last synced: about 1 year ago - Pushed: over 2 years ago

Dahoas/gpt-neox-finetuning
  • ==0.0.19 requirements/requirements.txt

Size: 36.7 MB - Last synced: about 1 year ago - Pushed: about 2 years ago

HomebrewNLP/HomebrewNLP-MTF 📦
HomebrewNLP in Mesh-TensorFlow flavour for distributed TPU training
  • * requirements.txt

Size: 1.29 MB - Last synced: about 1 month ago - Pushed: over 1 year ago

mosmos6/mesh-transformer-jax3
test for v3
  • * requirements.txt
  • * requirements_before_20230113.txt
  • * requirements_original.txt

Size: 165 KB - Last synced: about 1 year ago - Pushed: about 1 year ago

mosmos6/mesh-transformer-jax7
finetune test of jax4 repo
  • * requirements.txt
  • * requirements_before_20230113.txt
  • * requirements_original.txt

Size: 236 KB - Last synced: about 1 year ago - Pushed: about 1 year ago

mosmos6/Large-MTJ
This is a modified version of https://github.com/VE-FORBRYDERNE/mesh-transformer-jax/tree/ck to be adapted to JAX 0.3.25 so this runs on colab with TPU_driver0.2
  • * requirements.txt

Size: 245 KB - Last synced: about 1 year ago - Pushed: about 1 year ago

86885974647565434342125458569795756674/prompt-tuning
  • ==0.0.20 prompt-tuning/pile_requirements.txt

Size: 156 MB - Last synced: 9 months ago - Pushed: 9 months ago

leogao2/commoncrawl_downloader
  • ==0.0.8 requirements.txt

Size: 36.1 KB - Last synced: about 1 year ago - Pushed: about 3 years ago

hyang5916/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.41 MB - Last synced: 11 months ago - Pushed: almost 3 years ago

rbleattler/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 22.4 MB - Last synced: about 1 month ago - Pushed: about 1 month ago

Ralf12358/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 9.54 MB - Last synced: about 1 month ago - Pushed: about 1 year ago

pterameta/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 18.3 MB - Last synced: about 1 month ago - Pushed: 11 months ago

stancx1/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 18.9 MB - Last synced: 11 months ago - Pushed: 11 months ago

Gurix525/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.56 MB - Last synced: 11 months ago - Pushed: about 2 years ago

sunilravilla/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 18.3 MB - Last synced: 11 months ago - Pushed: 11 months ago

tiger322/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 15.7 MB - Last synced: 11 months ago - Pushed: 11 months ago

iamianM/mesh-transformer-jax
  • * requirements.txt

Size: 1.77 MB - Last synced: about 1 year ago - Pushed: almost 3 years ago

incredimo/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 15.5 MB - Last synced: 11 months ago - Pushed: 12 months ago

jllllll/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 19.3 MB - Last synced: 11 months ago - Pushed: 11 months ago

crjaensch/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 19.5 MB - Last synced: about 1 month ago - Pushed: 11 months ago

McClentyS/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 19.2 MB - Last synced: 11 months ago - Pushed: 11 months ago

qqq-tech/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT2& GPT3-like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.48 MB - Last synced: about 2 months ago - Pushed: about 3 years ago

qqq-tech/gpt-code-clippy Fork of CodedotAl/gpt-code-clippy
Full description can be found here: https://discuss.huggingface.co/t/pretrain-gpt-neo-for-open-source-github-copilot-model/7678?u=ncoop57
  • * requirements_scripts.txt

Size: 17.3 MB - Last synced: about 2 months ago - Pushed: over 2 years ago

qqq-tech/gpt-neox Fork of EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
  • ==0.0.19 requirements/requirements.txt

Size: 59.5 MB - Last synced: about 2 months ago - Pushed: over 2 years ago

qqq-tech/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 10.1 MB - Last synced: about 2 months ago - Pushed: about 1 year ago

jialii/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 18.5 MB - Last synced: 10 months ago - Pushed: 10 months ago

csqr/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 9.55 MB - Last synced: 10 months ago - Pushed: about 1 year ago

jesusoctavioas/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 16.2 MB - Last synced: about 1 month ago - Pushed: 11 months ago

gchenfly/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT2& GPT3-like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.17 MB - Last synced: about 2 months ago - Pushed: over 3 years ago

gchenfly/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf
  • ==0.0.20 requirements.txt
  • ==0.0.20 spaces/chatbot/requirements.txt

Size: 15.6 MB - Last synced: about 2 months ago - Pushed: 11 months ago

EC3-Gang/mesh-transformer-jax Fork of kingoflolz/mesh-transformer-jax
Model parallel transformers in JAX and Haiku
  • * requirements.txt

Size: 420 KB - Last synced: 10 months ago - Pushed: 10 months ago

cch230/h2ogpt Fork of h2oai/h2ogpt
Come join the movement to make the world's best open source GPT led by H2O.ai
  • ==0.0.20 requirements.txt

Size: 30.1 MB - Last synced: 8 months ago - Pushed: 8 months ago

sergefedorow/h2ogpt Fork of h2oai/h2ogpt
Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 21.2 MB - Last synced: 10 months ago - Pushed: 10 months ago

mengyanyuan/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.56 MB - Last synced: 10 months ago - Pushed: about 2 years ago

BiaoWu/h2ogpt Fork of h2oai/h2ogpt
Join us at H2O.ai to make the world's best open-source GPT with document and image Q&A, 100% private chat, no data leaks, Apache 2.0 https://arxiv.org/pdf/2306.08161.pdf Live Demo: https://gpt.h2o.ai/
  • ==0.0.20 requirements.txt

Size: 20.8 MB - Last synced: 10 months ago - Pushed: 10 months ago

ozmode/gpt-neo Fork of EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
  • * requirements.txt

Size: 1.56 MB - Last synced: 10 months ago - Pushed: about 2 years ago