transformer-architecture | Topic

Topic: "transformer-architecture"

Plachtaa/VALL-E-X

An open source implementation of Microsoft's VALL-E X zero-shot TTS model. Demo is available in https://plachtaa.github.io/vallex/

Language: Python - Size: 56.5 MB - Last synced at: 2 days ago - Pushed at: over 1 year ago - Stars: 7,867 - Forks: 787

cmhungsteve/Awesome-Transformer-Attention

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

Size: 5.65 MB - Last synced at: 2 days ago - Pushed at: 10 months ago - Stars: 4,860 - Forks: 493

tairov/llama2.mojo

Inference Llama 2 in one file of pure 🔥

Language: Mojo - Size: 2.61 MB - Last synced at: about 21 hours ago - Pushed at: 12 months ago - Stars: 2,111 - Forks: 139

nlpodyssey/spago

Self-contained Machine Learning and Natural Language Processing library in Go

Language: Go - Size: 19.5 MB - Last synced at: 2 days ago - Pushed at: about 1 month ago - Stars: 1,794 - Forks: 89

Ma-Lab-Berkeley/CRATE

Code for CRATE (Coding RAte reduction TransformEr).

Language: Python - Size: 55.8 MB - Last synced at: 1 day ago - Pushed at: 7 months ago - Stars: 1,225 - Forks: 96

awslabs/sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

Language: Python - Size: 9.83 MB - Last synced at: 1 day ago - Pushed at: 7 months ago - Stars: 1,215 - Forks: 325

genieincodebottle/generative-ai

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

Language: Jupyter Notebook - Size: 52.3 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 968 - Forks: 255

berniwal/swin-transformer-pytorch

Implementation of the Swin Transformer in PyTorch.

Language: Python - Size: 201 KB - Last synced at: 9 days ago - Pushed at: about 4 years ago - Stars: 827 - Forks: 130

joeynmt/joeynmt

Minimalist NMT for educational purposes

Language: Python - Size: 37.8 MB - Last synced at: 14 days ago - Pushed at: over 1 year ago - Stars: 692 - Forks: 215

cuiziteng/Illumination-Adaptive-Transformer

🌕 [BMVC 2022] You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction. SOTA for low light enhancement, 0.004 seconds try this for pre-processing.

Language: Python - Size: 29.4 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 513 - Forks: 49

fastnlp/CPT

CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

Language: Python - Size: 1.33 MB - Last synced at: 6 days ago - Pushed at: over 2 years ago - Stars: 487 - Forks: 72

wgcban/ChangeFormer

[IGARSS'22]: A Transformer-Based Siamese Network for Change Detection

Language: Python - Size: 13.8 MB - Last synced at: 4 days ago - Pushed at: over 1 year ago - Stars: 482 - Forks: 64

google-research/maxvit 📦

[ECCV 2022] Official repository for "MaxViT: Multi-Axis Vision Transformer". SOTA foundation models for classification, detection, segmentation, image quality, and generative modeling...

Language: Jupyter Notebook - Size: 2.6 MB - Last synced at: 5 days ago - Pushed at: almost 2 years ago - Stars: 472 - Forks: 33

linwhitehat/ET-BERT

The repository of ET-BERT, a network traffic classification model on encrypted traffic. The work has been accepted as The Web Conference (WWW) 2022 accepted paper.

Language: Python - Size: 13.3 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 458 - Forks: 92

kyegomez/MultiModalMamba

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

Language: Python - Size: 2.2 MB - Last synced at: 7 days ago - Pushed at: about 1 month ago - Stars: 449 - Forks: 24

abdur75648/Deep-Learning-Specialization-Coursera

This repo contains the updated version of all the assignments/labs (done by me) of Deep Learning Specialization on Coursera by Andrew Ng. It includes building various deep learning models from scratch and implementing them for object detection, facial recognition, autonomous driving, neural machine translation, trigger word detection, etc.

Language: Jupyter Notebook - Size: 161 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 442 - Forks: 366

wjf5203/SeqFormer

SeqFormer: Sequential Transformer for Video Instance Segmentation (ECCV 2022 Oral)

Language: Python - Size: 16.3 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 346 - Forks: 30

ZixuanKe/PyContinual

PyContinual (An Easy and Extendible Framework for Continual Learning)

Language: Python - Size: 3 MB - Last synced at: 8 days ago - Pushed at: over 1 year ago - Stars: 314 - Forks: 65

sgrvinod/a-PyTorch-Tutorial-to-Transformers

Attention Is All You Need | a PyTorch Tutorial to Transformers

Language: Python - Size: 27.5 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 302 - Forks: 47

PengBoXiangShang/multigraph_transformer

IEEE TNNLS 2021, transformer, multi-graph transformer, graph, graph classification, sketch recognition, sketch classification, free-hand sketch, official code of the paper "Multi-Graph Transformer for Free-Hand Sketch Recognition"

Language: Python - Size: 2.36 MB - Last synced at: about 1 month ago - Pushed at: about 4 years ago - Stars: 296 - Forks: 32

UIC-Liu-Lab/ContinualLM

An Extensible Continual Learning Framework Focused on Language Models (LMs)

Language: Python - Size: 696 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 272 - Forks: 21

hkproj/transformer-from-scratch-notes

Notes about "Attention is all you need" video (https://www.youtube.com/watch?v=bCz4OMemCcA)

Size: 1.32 MB - Last synced at: 2 months ago - Pushed at: almost 2 years ago - Stars: 256 - Forks: 59

zhongkaifu/Seq2SeqSharp

Seq2SeqSharp is a tensor based fast & flexible deep neural network framework written by .NET (C#). It has many highlighted features, such as automatic differentiation, different network types (Transformer, LSTM, BiLSTM and so on), multi-GPUs supported, cross-platforms (Windows, Linux, x86, x64, ARM), multimodal model for text and images and so on.

Language: C# - Size: 432 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 203 - Forks: 42

labteral/ernie

Simple State-of-the-Art BERT-Based Sentence Classification with Keras / TensorFlow 2. Built with HuggingFace's Transformers.

Language: Python - Size: 326 KB - Last synced at: 10 days ago - Pushed at: 12 months ago - Stars: 200 - Forks: 31

VSainteuf/pytorch-psetae

PyTorch implementation of the model presented in "Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention"

Language: Python - Size: 1.98 MB - Last synced at: 8 days ago - Pushed at: over 3 years ago - Stars: 198 - Forks: 43

prakhar21/TextAugmentation-GPT2

Fine-tuned pre-trained GPT2 for custom topic specific text generation. Such system can be used for Text Augmentation.

Language: Python - Size: 655 KB - Last synced at: 7 days ago - Pushed at: almost 2 years ago - Stars: 193 - Forks: 43

onlybooks/llm

LLM을 활용한 실전 AI 애플리케이션 개발

Language: Jupyter Notebook - Size: 4.47 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 148 - Forks: 116

miccaiif/TransMEF

Official PyTorch implementation of our AAAI22 paper: TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework via Self-Supervised Multi-Task Learning.

Language: Python - Size: 10.8 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 144 - Forks: 16

jshuadvd/LongRoPE

Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper

Language: Python - Size: 562 KB - Last synced at: 3 days ago - Pushed at: 10 months ago - Stars: 136 - Forks: 14

vilari-mickopf/mmwave-gesture-recognition

Basic Gesture Recognition Using mmWave Sensor - TI AWR1642

Language: Python - Size: 1.71 MB - Last synced at: 2 days ago - Pushed at: 8 months ago - Stars: 131 - Forks: 22

jcwang123/BA-Transformer

[MICCAI 2021] Boundary-aware Transformers for Skin Lesion Segmentation

Language: Python - Size: 14.9 MB - Last synced at: 10 months ago - Pushed at: over 1 year ago - Stars: 118 - Forks: 21

quanghuy0497/Transformers4Vision

A summarization of Transformer-based architectures for CV tasks, including image classification, object detection, segmentation, and Few-shot Learning. Keep updated frequently.

Size: 13.6 MB - Last synced at: 8 days ago - Pushed at: almost 3 years ago - Stars: 109 - Forks: 18

kyegomez/Algorithm-Of-Thoughts

My implementation of "Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models"

Language: Python - Size: 294 KB - Last synced at: 8 days ago - Pushed at: over 1 year ago - Stars: 98 - Forks: 15

ra1ph2/Vision-Transformer

Implementation of Vision Transformer from scratch and performance compared to standard CNNs (ResNets) and pre-trained ViT on CIFAR10 and CIFAR100.

Language: Jupyter Notebook - Size: 9.23 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 84 - Forks: 9

jet-universe/particle_transformer

Official implementation of "Particle Transformer for Jet Tagging".

Language: Python - Size: 27.4 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 70 - Forks: 43

shamim-hussain/egt_pytorch

Edge-Augmented Graph Transformer

Language: Python - Size: 79.1 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 67 - Forks: 9

szq0214/SReT

Official PyTorch implementation of our ECCV 2022 paper "Sliced Recursive Transformer"

Language: Python - Size: 557 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 64 - Forks: 11

google-research/human-scene-transformer

Human Scene Transformer: A framework for trajectory prediction and wrappers for reframing the JRDB dataset for the prediction task.

Language: Python - Size: 2.18 MB - Last synced at: about 1 month ago - Pushed at: 9 months ago - Stars: 62 - Forks: 10

songqiang321/Awesome-AI-Papers

This repository is used to collect papers and code in the field of AI.

Size: 4.08 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 60 - Forks: 6

UARK-AICV/VLTinT

[AAAI 2023 Oral] VLTinT: Visual-Linguistic Transformer-in-Transformer for Coherent Video Paragraph Captioning

Language: Jupyter Notebook - Size: 194 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 60 - Forks: 7

AkiRusProd/numpy-transformer

A numpy implementation of the Transformer model in "Attention is All You Need"

Language: Python - Size: 316 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 43 - Forks: 7

CiscoDevNet/g2p_seq2seq_pytorch

Grapheme to phoneme model for PyTorch

Language: Python - Size: 1.48 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 41 - Forks: 11

microsoft/CASPR

CASPR is a deep learning framework applying transformer architecture to learn and predict from tabular data at scale.

Language: Python - Size: 2.47 MB - Last synced at: 1 day ago - Pushed at: over 2 years ago - Stars: 38 - Forks: 3

yliess86/BayeFormers

General API for Deep Bayesian Variational Inference by Backpropagation. The repository has been designed to work with Transformers like architectures. Compatible with the HuggingFace Transformers models.

Language: Python - Size: 21.8 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 36 - Forks: 3

maohangyu/TIT_open_source

The official implementation of "Transformer in Transformer as Backbone for Deep Reinforcement Learning"

Language: Python - Size: 1.75 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 35 - Forks: 3

dame-cell/Triformer

Transformers components but in Triton

Language: Python - Size: 2.08 MB - Last synced at: 6 days ago - Pushed at: 7 days ago - Stars: 33 - Forks: 0

scaomath/eit-transformer

[ICLR 2023] Interplay between the Attention and Electrical Impedance Tomography

Language: Python - Size: 31.3 KB - Last synced at: 2 days ago - Pushed at: over 2 years ago - Stars: 33 - Forks: 4

cosmaadrian/time-enriched-multimodal-depression-detection

Official source code for the paper: "It’s Just a Matter of Time: Detecting Depression with Time-Enriched Multimodal Transformers"

Language: Python - Size: 579 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 32 - Forks: 1

engelnico/point-transformer

This is the official repository of the original Point Transformer architecture.

Language: Python - Size: 566 KB - Last synced at: almost 2 years ago - Pushed at: about 3 years ago - Stars: 31 - Forks: 6

zouguojian/Traffic-speed-prediction

Using to predict the highway traffic speed

Language: Python - Size: 646 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 30 - Forks: 3

eth-ait/cose

Language: JavaScript - Size: 6.86 MB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 30 - Forks: 7

dongbeank/CATS

[NeurIPS 2024] Official implementation of the paper "Are Self-Attentions Effective for Time Series Forecasting?"

Language: Python - Size: 991 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 27 - Forks: 3

januverma/transformers-stuff

Codes, scripts, and notebooks on various aspects of transformer models.

Language: Jupyter Notebook - Size: 473 KB - Last synced at: 5 months ago - Pushed at: about 2 years ago - Stars: 27 - Forks: 4

Kaiseem/QueryOTR

[ECCV2022] Official PyTorch implementation of the paper "Outpainting by Queries"

Language: Python - Size: 282 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 27 - Forks: 4

MrVPlusOne/Coeditor

Coeditor: Leveraging Repo-level Diffs for Code Auto-editing

Language: Python - Size: 4.62 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 26 - Forks: 2

utopia-group/TypeT5

Seq2seq Type Inference using Static Analysis and CodeT5

Language: Jupyter Notebook - Size: 3.38 MB - Last synced at: 12 months ago - Pushed at: almost 2 years ago - Stars: 24 - Forks: 8

iamrakesh28/Video-Prediction

Implementation of Transformer Encoder Decoder Architecture for Video Predictions

Language: Python - Size: 4.17 MB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 24 - Forks: 5

abhaskumarsinha/MinimalGPT

MinimalGPT is a concise, adaptable, and streamlined code framework that encompasses the essential components necessary for the construction, training, inference, and fine-tuning of the GPT model. This framework is implemented exclusively using Keras and TensorFlow, ensuring compatibility and coherence within the broader deep learning ecosystem.

Language: Python - Size: 320 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 23 - Forks: 6

mohd-faizy/06P_Sentiment-Analysis-With-Deep-Learning-Using-BERT

Finetuning BERT in PyTorch for sentiment analysis.

Language: Jupyter Notebook - Size: 20.6 MB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 22 - Forks: 9

kyegomez/GPT3

An implementation of the base GPT-3 Model architecture from the paper by OPENAI "Language Models are Few-Shot Learners"

Language: Python - Size: 236 KB - Last synced at: 9 days ago - Pushed at: 11 months ago - Stars: 19 - Forks: 2

aimagelab/perceive-transform-and-act

PyTorch code for the paper: "Perceive, Transform, and Act: Multi-Modal Attention Networks for Vision-and-Language Navigation"

Language: C++ - Size: 148 MB - Last synced at: about 1 month ago - Pushed at: almost 4 years ago - Stars: 19 - Forks: 4

pratham16cse/AggForecaster

Code for "Coherent Probabilistic Aggregate Queries on Long-horizon Forecasts", IJCAI 2022

Language: Python - Size: 89.4 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 18 - Forks: 4

04RR/SOTA-Vision

Implementation of various state of the art architectures used in computer vision.

Language: Python - Size: 740 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 18 - Forks: 8

frankaging/Multimodal-Transformer

Attention Based Multi-modal Emotion Recognition; Stanford Emotional Narratives Dataset

Language: Python - Size: 458 MB - Last synced at: 17 days ago - Pushed at: over 5 years ago - Stars: 18 - Forks: 1

leehyeonbeen/TimeSeriesSeq2Seq

Sequence-to-sequence model implementations including RNN, CNN, Attention, and Transformers using PyTorch

Language: Python - Size: 82 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 17 - Forks: 1

tamasino52/UNETR-Pose

3D Multi-person Pose Estimation in Multi-view Environment using 3D U-Net Transformer Networks

Language: Python - Size: 7.37 MB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 17 - Forks: 1

AspirinCode/TransAntivirus

Transformer-based molecular generative model for antiviral drug design

Language: Python - Size: 439 KB - Last synced at: 3 months ago - Pushed at: 6 months ago - Stars: 16 - Forks: 9

wywyWang/ShuttleNet

Official Implementation for ShuttleNet: Position-aware Fusion of Rally Progress and Player Styles for Stroke Forecasting in Badminton (AAAI'22)

Language: Python - Size: 55.7 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 16 - Forks: 3

Parity-LRX/Parity

Deep Learning Potential model with Symmetry Invariant and Equivariant Descriptor

Language: Python - Size: 646 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 15 - Forks: 1

reymond-group/MultiStepRetrosynthesisTTL

Multi-Step Retrosynthesis Tool based on Augmented Disconnection Aware Triple Transformer Loop Predictions

Language: Python - Size: 21.9 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 15 - Forks: 4

zouguojian/Personal-Accepted-Research

Welcome to quote our published papers, and the codes have been uploaded.

Size: 165 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 15 - Forks: 3

erfanzar/OST-OpenSourceTransformers

OST Collection: An AI-powered suite of models that predict the next word matches with remarkable accuracy (Text Generative Models). OST Collection is based on a novel approach to work as a full and intelligent NLP Model.

Language: Jupyter Notebook - Size: 134 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 15 - Forks: 1

juyongjiang/BiCAT

This is the official source code of "BiCAT: Self-Knowledge Distillation with Bidirectional Chronological Augmentation of Transformer for Sequential Recommendation" based on TensorFlow.

Language: Python - Size: 34.7 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 15 - Forks: 3

omron-sinicx/crystalformer

The official code respository for "Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding" (ICLR 2024)

Language: Python - Size: 3.15 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 14 - Forks: 0

cosmaadrian/gaitformer

GaitFormer Official Codebase for the paper "Learning Gait Representations with Noisy Multi-Task Learning"

Language: Python - Size: 1.02 MB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 14 - Forks: 0

ashwanitanwar/nmt-transfer-learning-xlm-r

Improving Low-Resource Neural Machine Translation of Related Languages by Transfer Learning

Language: Python - Size: 16.5 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 14 - Forks: 5

takara-ai/SwarmFormer

A pytorch implementation of SwarmFormer for text classification.

Language: Python - Size: 53.7 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 13 - Forks: 4

mohyunho/NAS_transformer

Evolutionary Neural Architecture Search on Transformers for RUL Prediction

Language: Python - Size: 15.4 MB - Last synced at: almost 2 years ago - Pushed at: about 2 years ago - Stars: 13 - Forks: 3

nikhilroxtomar/Vision-Transformer-ViT-in-TensorFlow

Vision Transformer Implementation in TensorFlow

Language: Python - Size: 693 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 13 - Forks: 2

jamm1985/seismo-performer

The Seismo-Performer: A Novel Machine Learning Approach for General and Efficient Seismic Phase Recognition from Local Earthquakes in Real Time

Language: Jupyter Notebook - Size: 71.1 MB - Last synced at: 4 months ago - Pushed at: about 3 years ago - Stars: 13 - Forks: 5

dingo-actual/om

An LLM architecture utilizing a recurrent structure and multi-layer memory

Language: Python - Size: 1.04 MB - Last synced at: 25 days ago - Pushed at: 26 days ago - Stars: 12 - Forks: 0

stoneMo/DeepAVFusion

Official codebase for "Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling".

Language: Python - Size: 26.4 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 12 - Forks: 0

kyegomez/Tiktokx

Tiktok is an advanced multimedia recommender system that fuses the generative modality-aware collaborative self-augmentation and contrastive cross-modality dependency encoding to achieve superior performance compared to existing state-of-the-art multi-model recommenders.

Language: Python - Size: 229 KB - Last synced at: 9 days ago - Pushed at: over 1 year ago - Stars: 12 - Forks: 0