Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / NVIDIA / TensorRT-LLM
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2FTensorRT-LLM
Stars: 7,030
Forks: 751
Open Issues: 625
License: apache-2.0
Language: C++
Repo Size: 272 MB
Dependencies:
206
Created: 10 months ago
Updated: about 22 hours ago
Last pushed: about 18 hours ago
Last synced: about 18 hours ago
Files
Dependencies
- accelerate ==0.25.0
- transformers ==4.36.0
- breathe *
- myst_parser *
- pygit2 *
- sphinx >=7.0
- sphinx-rtd-theme *
- datasets *
- rouge_score *
- sentencepiece *
- tqdm *
- transformers >=4.31.0
- typing-extensions ==4.5.0
- datasets ==2.14.5
- rouge_score *
- sentencepiece *
- datasets >=2.14.4
- nemo-toolkit <=1.20.0,>=1.18.0
- rouge_score *
- accelerate ==0.20.3 development
- colored * development
- cuda-python ==12.2.0 development
- diffusers ==0.15.0 development
- einops * development
- graphviz * development
- mpi4py * development
- mypy * development
- numpy * development
- onnx ==1.12.0 development
- parameterized * development
- polygraphy * development
- pre-commit * development
- pytest-cov * development
- pytest-forked * development
- pytest-xdist * development
- pywin32 * development
- tokenizers ==0.13.3 development
- torch ==2.1.0.dev20230828 development
- torchdata ==0.7.0.dev20230828 development
- torchtext ==0.16.0.dev20230828 development
- torchvision ==0.16.0.dev20230828 development
- transformers ==4.31.0 development
- accelerate ==0.20.3 development
- colored * development
- cuda-python ==12.2.0 development
- diffusers ==0.15.0 development
- einops * development
- graphviz * development
- mpi4py * development
- mypy * development
- numpy * development
- onnx ==1.12.0 development
- parameterized * development
- polygraphy * development
- pre-commit * development
- pytest-cov * development
- pytest-forked * development
- pytest-xdist * development
- torch * development
- transformers ==4.31.0 development
- accelerate ==0.20.3
- build *
- colored *
- cuda-python ==12.2.0
- diffusers ==0.15.0
- mpi4py *
- numpy *
- onnx >=1.12.0
- polygraphy *
- pywin32 *
- sentencepiece >=0.1.99
- tokenizers ==0.13.3
- torch ==2.1.0.dev20230828
- torchdata ==0.7.0.dev20230828
- torchtext ==0.16.0.dev20230828
- torchvision ==0.16.0.dev20230828
- transformers ==4.31.0
- wheel *
- accelerate ==0.20.3
- build *
- colored *
- cuda-python ==12.2.0
- diffusers ==0.15.0
- lark *
- mpi4py *
- numpy *
- onnx >=1.12.0
- polygraphy *
- sentencepiece >=0.1.99
- tensorrt >=8.6.0
- torch *
- transformers ==4.31.0
- wheel *
- mcr.microsoft.com/windows/servercore ltsc2019 build
- datasets *
- evaluate *
- protobuf *
- rouge_score *
- sentencepiece *
- datasets ==2.14.5
- evaluate *
- rouge_score *
- sentencepiece *
- aiohttp_sse_client *
- datasets *
- einops *
- evaluate *
- gradio ==3.40.1
- mdtex2html *
- openai *
- rouge_score *
- sentencepiece *
- sse_starlette *
- tiktoken *
- transformers *
- transformers-stream-generator *
- datasets *
- evaluate *
- rouge_score *
- mamba-ssm ==1.1.1
- auto-gptq *
- datasets *
- einops *
- evaluate *
- rouge_score *
- sentencepiece *
- tiktoken *
- transformers *
- transformers-stream-generator *
- datasets *
- evaluate *
- rouge_score *
- sentencepiece >=0.1.99
- easydict *
- flax *
- h5py *
- jax *
- nltk *
- rouge_score *
- safetensors *
- sentencepiece *
- tensorrt_llm ==0.9.0.dev2024031900
- datasets ==2.14.6
- evaluate *
- rouge_score *
- sentencepiece *
- tensorrt_llm ==0.9.0.dev2024040200
- aenum ==3.1.15
- click-option-group ==0.5.6
- pydantic >=2.2.1
- datasets *
- evaluate *
- rouge_score *
- tensorrt_llm ==0.9.0.dev2024040900
- tiktoken ==0.6.0
- actions/stale v9 composite
- tensorrt_llm ==0.10.0.dev2024043000
- datasets *
- evaluate *
- rouge_score *
- tensorrt_llm ==0.11.0.dev2024051400
- datasets *
- evaluate *
- rouge_score *
- tensorrt_llm ==0.11.0.dev2024060400
- datasets ==2.14.6
- dm_haiku ==0.0.12
- evaluate *
- jax ==0.4.28
- jaxlib ==0.4.28
- rouge_score *
- sentencepiece ==0.2.0
- tensorrt_llm ==0.11.0.dev2024060400
- SentencePiece *
- datasets *
- evaluate *
- rouge_score *
- tensorrt_llm ==0.11.0.dev2024051400
- transformers ==4.36.2