GitHub / 2003HARSH / Transformer-based-Decoder-only-Language-Model
This repository explores building a character-level transformer decoder in PyTorch, similar to GPT while focusing more on understanding individual components. My goal is to gain deep transformer knowledge and see if character-level learning improves handling of unseen words. The code allows for hyperparameter tuning and experiment customization.
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2003HARSH%2FTransformer-based-Decoder-only-Language-Model
PURL: pkg:github/2003HARSH/Transformer-based-Decoder-only-Language-Model
Stars: 0
Forks: 0
Open issues: 0
License: mit
Language: Jupyter Notebook
Size: 74.3 MB
Dependencies parsed at: Pending
Created at: about 1 year ago
Updated at: 8 months ago
Pushed at: 10 months ago
Last synced at: 4 months ago
Topics: from-scratch-in-python, gpt, language-model, pytorch, transformers