Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / benbrandt / text-splitter

Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benbrandt%2Ftext-splitter

Stars: 135
Forks: 9
Open Issues: 5

License: mit
Language: Rust
Repo Size: 11.5 MB
Dependencies: 225

Created: about 1 year ago
Updated: about 1 month ago
Last pushed: about 1 month ago
Last synced: about 1 month ago

Files

Readme

Dependencies

.github/workflows/ci.yml actions

Swatinem/rust-cache v2 composite
actions-rs/clippy-check v1.0.7 composite
actions/checkout v4 composite
codecov/codecov-action v3 composite
dtolnay/rust-toolchain stable composite
dtolnay/rust-toolchain master composite
obi1kenobi/cargo-semver-checks-action v2 composite
taiki-e/install-action cargo-hack composite
taiki-e/install-action cargo-llvm-cov composite

.github/workflows/python.yml actions

PyO3/maturin-action v1 composite
Swatinem/rust-cache v2 composite
actions/checkout v4 composite
actions/download-artifact v3 composite
actions/setup-python v4 composite
actions/upload-artifact v3 composite
dtolnay/rust-toolchain stable composite

.github/workflows/release.yml actions

actions/checkout v4 composite
dtolnay/rust-toolchain stable composite

.github/workflows/security.yml actions

actions-rs/audit-check v1.2.0 composite
actions-rs/install v0.1.2 composite
actions/checkout v4 composite

Cargo.toml cargo

fake 2.6.1 development
insta 1.31.0 development
more-asserts 0.3.1 development
tokenizers >=0.13.3, <0.15.0 development
auto_enums 0.8.2
either 1.9.0
itertools 0.11.0
once_cell 1.18.0
regex 1.9.3
tiktoken-rs >=0.2.0, <0.6.0
tokenizers >=0.13.3, <0.15.0
unicode-segmentation 1.10.1

bindings/python/Cargo.toml cargo

bindings/python/pyproject.toml pypi

Cargo.lock cargo

192 dependencies