GitHub topics: swiglu
liaoyanqing666/Decoder-only-transformer_Time_Series_Prediction
使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)
Language: Python - Size: 31.3 KB - Last synced at: 18 days ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 1
estnafinema0/russian-jokes-generator
Transformer Models for Humorous Text Generation. Fine-tuned on Russian jokes dataset with ALiBi, RoPE, GQA, and SwiGLU.Plus a custom Byte-level BPE tokenizer.
Language: Jupyter Notebook - Size: 294 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0
andrewhsugithub/min-llama
my llama3 implementation
Language: Python - Size: 12.7 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0
s-chh/PyTorch-Scratch-LLM
Simple and easy to understand PyTorch implementation of Large Language Model (LLM) GPT and LLAMA from scratch with detailed steps. Implemented: Byte-Pair Tokenizer, Rotational Positional Embedding (RoPe), SwishGLU, RMSNorm, Mixture of Experts (MOE). Tested on Taylor Swift song lyrics dataset.
Language: Python - Size: 58.6 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0
fattorib/fusedswiglu
Fused SwiGLU Triton kernels
Language: Python - Size: 188 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
