GitHub / AnimaVR / TOKENIZER-BytePairEncoderDecoder-ModelTrainer-CSharp
Actual C Sharp Byte Pair Encoder that works. Use bin folder or add your own data to be able to train your own model, this model is then used to encode into train.bin and val.bin binary files to use to train an LLM or similar.
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnimaVR%2FTOKENIZER-BytePairEncoderDecoder-ModelTrainer-CSharp
PURL: pkg:github/AnimaVR/TOKENIZER-BytePairEncoderDecoder-ModelTrainer-CSharp
Stars: 2
Forks: 0
Open issues: 0
License: None
Language: C#
Size: 4.61 MB
Dependencies parsed at: Pending
Created at: about 2 years ago
Updated at: about 1 year ago
Pushed at: about 1 year ago
Last synced at: about 1 year ago
Topics: ai, artificial-intelligence, bytepairencoding, llm