prima.cpp

prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lizonghang%2Fprima.cpp
PURL: pkg:github/Lizonghang/prima.cpp

Stars: 979
Forks: 67
Open issues: 5

License: mit
Language: C++
Size: 54.8 MB
Dependencies parsed at: 86

Created at: 11 months ago
Updated at: about 2 months ago
Pushed at: about 2 months ago
Last synced at: about 2 months ago

Topics: distributed-ai, distributed-inference, llama-cpp, llm-inference, on-device-llms

examples/batched.swift/Package.swift swiftpm

examples/llama.android/build.gradle.kts maven

examples/llama.android/llama/build.gradle.kts maven

androidx.appcompat:appcompat 1.6.1 implementation
androidx.core:core-ktx 1.12.0 implementation
com.google.android.material:material 1.11.0 implementation
junit:junit 4.13.2 testImplementation

examples/gguf-hash/deps/rotate-bits/package.json npm

examples/gguf-hash/deps/sha1/package.json npm

examples/gguf-hash/deps/sha256/package.json npm

jb55/rotate-bits.h 0.1.1

examples/llava/requirements.txt pypi

examples/server/bench/requirements.txt pypi

matplotlib *
requests *

examples/server/tests/requirements.txt pypi

aiohttp * test
behave * test
huggingface_hub * test
numpy * test
openai * test
prometheus-client * test
requests * test

gguf-py/pyproject.toml pypi

pytest ^5.2 develop
numpy >=1.17
python >=3.8
pyyaml >=5.1
sentencepiece >=0.1.98,<=0.2.0
tqdm >=4.27

examples/llama.android/app/build.gradle.kts maven

androidx.activity:activity-compose 1.8.2 implementation
androidx.compose.material3:material3 * implementation
androidx.compose.ui:ui * implementation
androidx.compose.ui:ui-graphics * implementation
androidx.compose.ui:ui-tooling-preview * implementation
androidx.core:core-ktx 1.12.0 implementation
androidx.lifecycle:lifecycle-runtime-ktx 2.6.2 implementation
junit:junit 4.13.2 testImplementation

pyproject.toml pypi

pytest ^5.2 develop
gguf *
numpy ^1.25.0
protobuf >=4.21.0,<5.0.0
python >=3.9
sentencepiece >=0.1.98,<=0.2.0
torch ^2.2.0
transformers >=4.35.2,<5.0.0

requirements/requirements-all.txt pypi

requirements/requirements-compare-llama-bench.txt pypi

GitPython *
tabulate *

requirements/requirements-convert_hf_to_gguf.txt pypi

torch *

requirements/requirements-convert_hf_to_gguf_update.txt pypi

torch *

requirements/requirements-convert_legacy_llama.txt pypi

gguf >=0.1.0
numpy *
protobuf >=4.21.0,<5.0.0
sentencepiece *
transformers >=4.45.1,<5.0.0

requirements/requirements-convert_llama_ggml_to_gguf.txt pypi

requirements/requirements-convert_lora_to_gguf.txt pypi

requirements/requirements-pydantic.txt pypi

requirements/requirements-test-tokenizer-random.txt pypi

cffi * test

requirements.txt pypi

Package.swift swiftpm

poetry.lock pypi

atomicwrites 1.4.1
attrs 23.2.0
certifi 2024.2.2
charset-normalizer 3.3.2
colorama 0.4.6
filelock 3.13.1
fsspec 2024.2.0
gguf 0.7.0
huggingface-hub 0.20.3
idna 3.6
jinja2 3.1.3
markupsafe 2.1.5
more-itertools 10.2.0
mpmath 1.3.0
networkx 3.2.1
numpy 1.26.4
packaging 23.2
pluggy 0.13.1
protobuf 4.25.3
py 1.11.0
pytest 5.4.3
pyyaml 6.0.1
regex 2023.12.25
requests 2.31.0
safetensors 0.4.2
sentencepiece 0.1.99
sympy 1.12
tokenizers 0.15.2
torch 2.2.1+cpu
tqdm 4.66.2
transformers 4.38.1
typing-extensions 4.9.0
urllib3 2.2.1
wcwidth 0.2.13

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Repos

GitHub / Lizonghang / prima.cpp