GitHub / kyegomez / ShallowFF

Zeta implemantion of "Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers"

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FShallowFF

Commit Stats

Commits: 12
Authors: 2
Mean commits per author: 6.0
Development Distribution Score: 0.083
More commit stats: https://commits.ecosyste.ms/hosts/GitHub/repositories/kyegomez/ShallowFF

Topics: artificial-intelligence, attention, attention-is-all-you-need, attention-mechanism, attention-mechanisms, feedforward, transformer, transformer-encoder, transformer-models, transformers-models

Funding links: https://github.com/sponsors/kyegomez

Files

Dependencies