GitHub / 2003HARSH / Transformer-based-Decoder-only-Language-Model

This repository explores building a character-level transformer decoder in PyTorch, similar to GPT while focusing more on understanding individual components. My goal is to gain deep transformer knowledge and see if character-level learning improves handling of unseen words. The code allows for hyperparameter tuning and experiment customization.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2003HARSH%2FTransformer-based-Decoder-only-Language-Model

Topics: from-scratch-in-python, gpt, language-model, pytorch, transformers

• Files
• Readme
• Dependencies