GitHub / 0-mostafa-rezaee-0 / Batch_LLM_Inference_with_Ray_Data_LLM
Batch LLM Inference with Ray Data LLM: From Simple to Advanced
Stars: 3
Forks: 1
Open issues: 0
License: mit
Language: Jupyter Notebook
Size: 1.63 MB
Dependencies parsed at:
15
Created at: about 2 months ago
Updated at: about 2 months ago
Pushed at: about 2 months ago
Last synced at: about 2 months ago
Topics: batch-inference, distributed-computing, large-language-models, llm, llm-api, nlp, parallel-processing, ray, ray-data, ray-serve, vllm
docker/Dockerfile
docker
- rayproject/ray 2.9.3-py310 build
docker/docker-compose.yml
docker
requirements.txt
pypi
- datasets >=2.12.0
- huggingface_hub >=0.16.0
- jupyter >=1.0.0
- jupyterlab >=4.0.0
- matplotlib >=3.7.0
- numpy >=1.23.0
- openai >=1.0.0
- pandas >=2.0.0
- ray >=2.9.0
- scikit-learn >=1.2.0
- torch >=2.0.0
- tqdm >=4.65.0
- transformers >=4.30.0
- vllm >=0.2.0