GitHub / taskswithcode / GPTTokenizationTutorial
Notebook to understand how Byte level BPE tokenizer used in GPT models avoids unknown (UNK) token
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskswithcode%2FGPTTokenizationTutorial
PURL: pkg:github/taskswithcode/GPTTokenizationTutorial
Stars: 0
Forks: 0
Open issues: 0
License: mit
Language: Jupyter Notebook
Size: 29.3 KB
Dependencies parsed at: Pending
Created at: over 2 years ago
Updated at: over 1 year ago
Pushed at: over 2 years ago
Last synced at: about 1 year ago
Funding Links https://github.com/sponsors/taskswithcode
Loading...