GitHub / endomorphosis / ipfs_datasets_py
a model manager for downloading huggingface datasets over s3 / ipfs
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endomorphosis%2Fipfs_datasets_py
Stars: 5
Forks: 1
Open issues: 0
License: agpl-3.0
Language: HTML
Size: 13.8 MB
Dependencies parsed at:
45
Created at: about 1 year ago
Updated at: 16 days ago
Pushed at: 16 days ago
Last synced at: 11 days ago
package.json
npm
- @aws-sdk/client-s3 ^3.315.0
- @aws-sdk/lib-storage ^3.315.0
- @chainsafe/libp2p-gossipsub ^13.0.0
- @orbitdb/core ^2.1.0
- @orbitdb/quickstart ^1.0.0
- commander ^11.0.0
- crypto ^1.0.1
- deasync ^0.1.29
- debug ^4.3.4
- helia ^4.1.0
- ipfs-cluster-api ^0.0.9
- is-stream ^3.0.0
- nodejs ^0.0.0
- prompt-sync ^4.2.0
- prompt-sync-history ^1.0.1
- shutil ^0.0.1
- subprocess ^0.3.0
- sync-request 6.1.0
setup.py
pypi
- boto3 *
- requests *
- transformers *
- urllib3 *
- actions/checkout v4 composite
- actions/download-artifact v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact v3 composite
- pypa/gh-action-pypi-publish release/v1 composite
pyproject.toml
pypi
- FlagEmbedding *
- InstructorEmbedding *
- accelerate *
- faiss-cpu *
- faiss-gpu *
- gguf *
- hsnwlib *
- llama-cpp-python *
- numpy <=1.25.0
- openai *
- pandas *
- protobuf <=3.20
- scipy *
- sentence_transformers *
- sentencepiece *
- torch ==2.0.0
- transformers *
- w3storage *