GitHub / shadowy-pycoder / tinybpe
Tiny implementation of the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Ispired by karpathy/minbpe
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadowy-pycoder%2Ftinybpe
PURL: pkg:github/shadowy-pycoder/tinybpe
Stars: 0
Forks: 0
Open issues: 0
License: mit
Language: Go
Size: 5.65 MB
Dependencies parsed at: Pending
Created at: 3 months ago
Updated at: 3 months ago
Pushed at: 3 months ago
Last synced at: 3 months ago
Loading...