GitHub topics: optimizers

Repositories

warner-benjamin/optimi

Fast, Modern, and Low Precision PyTorch Optimizers

Language: Python - Size: 857 KB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 109 - Forks: 4

renekfr/SelectiveAdam

Pytorch, triton accelerated Selective Adam Optimizer

Language: Python - Size: 73.2 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 3 - Forks: 1

inikishev/torchzero

Modular optimization library for PyTorch (work-in-progress).

Language: Python - Size: 63.4 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 6 - Forks: 0

dplassgit/d2lang

D2 is a strongly-typed, statically-typed, (mostly) inferred-type compiled language.

Language: Java - Size: 15.4 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 7 - Forks: 3

epfml/llm-optimizer-benchmark

Benchmarking Optimizers for LLM Pretraining

Size: 400 KB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

Koratahiu/MLorc

Unofficial implementation of "MLorc: Momentum Low-rank Compression for Large Language Model Adaptation"

Language: Python - Size: 43.9 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0

LoganBooker/prodigy-plus-schedule-free

Prodigy and Schedule-Free, together at last.

Language: Python - Size: 306 KB - Last synced at: 1 day ago - Pushed at: 3 months ago - Stars: 67 - Forks: 7

DavidUlloa6310/optimizers

Simple implementations of most popular optimizers like Adam, RMSProp, Adagrad, and SGD. Optimizers benchmarked against MNIST, Cifar, and IMDB dataset as referenced in the Adam paper.

Language: Python - Size: 1.79 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

AqwamCreates/DataPredict-Neural

TensorFlow-like Deep Learning Library (For Roblox And Pure Lua).

Language: Lua - Size: 9.86 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 3 - Forks: 0

lucidrains/lion-pytorch

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Language: Python - Size: 216 KB - Last synced at: 12 days ago - Pushed at: 9 months ago - Stars: 2,155 - Forks: 56

AqwamCreates/DataPredict

Lua-Based Machine, Deep And Reinforcement Learning Library (For Roblox And Pure Lua). Contains Over 40 Models!

Language: Lua - Size: 20.2 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 9 - Forks: 2

gugarosa/opytimizer

🐦 Opytimizer is a Python library consisting of meta-heuristic optimization algorithms.

Language: Python - Size: 2.05 MB - Last synced at: 1 day ago - Pushed at: 11 months ago - Stars: 622 - Forks: 42

elixir-nx/axon

Nx-powered Neural Networks

Language: Elixir - Size: 135 MB - Last synced at: 14 days ago - Pushed at: about 1 month ago - Stars: 1,637 - Forks: 112

sozelfist/annotated_deep_learning_paper_implementations Fork of labmlai/annotated_deep_learning_paper_implementations

🧑‍🏫 50! Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

Language: Python - Size: 147 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 1 - Forks: 0

decile-team/submodlib

Summarize Massive Datasets using Submodular Optimization

Language: Jupyter Notebook - Size: 28.1 MB - Last synced at: 17 days ago - Pushed at: 4 months ago - Stars: 107 - Forks: 44

labmlai/annotated_deep_learning_paper_implementations

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

Language: Python - Size: 147 MB - Last synced at: 26 days ago - Pushed at: 26 days ago - Stars: 62,440 - Forks: 6,327

Brokttv/optimizers-from-scratch

Implementations of deep learning optimizers using NumPy, including SGD, Adam, Adagrad, NAG, RMSProp, and Momentum.

Language: Jupyter Notebook - Size: 168 KB - Last synced at: 30 days ago - Pushed at: 30 days ago - Stars: 0 - Forks: 0

OverLordGoldDragon/keras-adamw

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

Language: Python - Size: 145 KB - Last synced at: 3 days ago - Pushed at: over 3 years ago - Stars: 169 - Forks: 25

CyberZHG/keras-radam 📦

RAdam implemented in Keras & TensorFlow

Language: Python - Size: 54.7 KB - Last synced at: 11 days ago - Pushed at: over 3 years ago - Stars: 326 - Forks: 45

tianshijing/Awesome-Optimizers

This repository is a meticulously assembled anthology of optimization algorithms, meticulously implemented in PyTorch, designed to cater to the diverse needs of the machine learning research community.

Language: Jupyter Notebook - Size: 6.42 MB - Last synced at: 13 days ago - Pushed at: 5 months ago - Stars: 17 - Forks: 3

rraghavkaushik/Optimizers

PyTorch Implementation of Optimizers for Deep Learning from scratch.

Language: Python - Size: 548 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 1 - Forks: 0

palle-k/DL4S

Accelerated tensor operations and dynamic neural networks based on reverse mode automatic differentiation for every device that can run Swift - from watchOS to Linux

Language: Swift - Size: 19.5 MB - Last synced at: 20 days ago - Pushed at: almost 2 years ago - Stars: 109 - Forks: 13

OptimAI-Lab/Minimalist_LLM_Pretraining

A Minimalist Optimizer Design for LLM Pretraining

Language: Python - Size: 490 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 3 - Forks: 0

boyfriend120393/transformers-tutorial

Explore the "transformers-tutorial" to master transformer architecture and GPT models through interactive HTML lessons. Start your journey on GitHub today! 🐙✨

Language: HTML - Size: 35.2 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

mertsaru/Cognitive-GLVQ

Cognitive bias inspired GLVQ optimizers

Language: Jupyter Notebook - Size: 56.6 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

jaepil/geometric-adam

A Ray Tracing-Inspired Approach to Neural Network Optimization

Language: Python - Size: 25.1 MB - Last synced at: 6 days ago - Pushed at: 3 months ago - Stars: 17 - Forks: 0

mohi-devhub/symbolic-optimizer-research

Reimplementation of the Lion optimizer using symbolic program search and PyTorch.

Language: Python - Size: 57.6 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

sail-sg/optim4rl

Optim4RL is a Jax framework of learning to optimize for reinforcement learning.

Language: Python - Size: 135 KB - Last synced at: 2 months ago - Pushed at: 9 months ago - Stars: 26 - Forks: 2

dmeoli/optiml

Optimizers for/and sklearn compatible Machine Learning models

Language: Python - Size: 483 MB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 10 - Forks: 4

OptimAI-Lab/Memory_Efficient_Pretraining

Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking

Language: Python - Size: 68.4 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 3 - Forks: 0

SamsungSAILMontreal/nino

Code for "Accelerating Training with Neuron Interaction and Nowcasting Networks" [to appear at ICLR 2025]

Language: Python - Size: 58.6 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 19 - Forks: 4

TheYali1/FixNet

All-in-one batch script to boost your PC, fix network issues, remove bloat, and squeeze out those extra FPS in your games.

Language: Batchfile - Size: 50.8 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

lucidrains/adam-atan2-pytorch

Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch

Language: Python - Size: 429 KB - Last synced at: 4 months ago - Pushed at: 9 months ago - Stars: 104 - Forks: 1

KasrAskari/DL-Optimizers

A visual and code-based deep dive into how classic optimizers update weights from scratch using NumPy.

Language: Jupyter Notebook - Size: 1000 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

CyberZHG/keras-lookahead 📦

Lookahead mechanism for optimizers in Keras.

Language: Python - Size: 31.3 KB - Last synced at: 19 days ago - Pushed at: about 4 years ago - Stars: 50 - Forks: 9

avnlp/hyperparameter-tuning

Effect of Optimizer Selection and Hyperparameter Tuning on Training Efficiency and LLM Performance

Language: Python - Size: 4.59 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0

clovaai/frostnet

FrostNet: Towards Quantization-Aware Network Architecture Search

Language: Python - Size: 505 KB - Last synced at: 5 months ago - Pushed at: over 1 year ago - Stars: 105 - Forks: 18

harshalmittal4/Hypergradient_variants

Improved Hypergradient optimizers for ML, providing better generalization and faster convergence.

Language: Jupyter Notebook - Size: 104 MB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 16 - Forks: 0

ProudElm/solaredgeoptimizers

Intergration to get optimizers information from the SolarEdge portal

Language: Python - Size: 56.6 KB - Last synced at: 6 months ago - Pushed at: about 1 year ago - Stars: 51 - Forks: 15

The-Swarm-Corporation/MultiModelOptimizer

MultiModelOptimizer: A Hierarchical Parameter Synchronization Approach for Joint Training of Multiple Transformer Models

Language: Python - Size: 469 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 6 - Forks: 0

Subhajit0210/Bank-Churn-Analysis-And-Modelling--ANN

Language: Jupyter Notebook - Size: 0 Bytes - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

Rishit-dagli/Gradient-Centralization-TensorFlow

Instantly improve your training performance of TensorFlow models with just 2 lines of code!

Language: Python - Size: 769 KB - Last synced at: 14 days ago - Pushed at: over 3 years ago - Stars: 106 - Forks: 22

shreyansh26/ML-Optimizers-JAX

Toy implementations of some popular ML optimizers using Python/JAX

Language: Python - Size: 10.7 KB - Last synced at: 5 months ago - Pushed at: about 4 years ago - Stars: 44 - Forks: 2

Arunprakash-A/ML-DL-LLM-Animations

Repo for all animations created for the ML, DL and LLM course

Language: Jupyter Notebook - Size: 13.1 MB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 1 - Forks: 0

matcor852/NeuralNet 📦

Polyvalent neural network

Language: C - Size: 36.1 KB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

reza-chehreghani/AI-Assignment-1-Optimization-SMOTE-DNN

Exploring Optimizers, Handling Imbalanced Datasets with SMOTE, and Building Classification Models Using Deep Neural Networks.

Language: Jupyter Notebook - Size: 13.6 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

ivallesp/awesome-optimizers

Neural Network optimizers implemented from scratch in numpy (Adam, Adadelta, RMSProp, SGD, etc.)

Language: Jupyter Notebook - Size: 35.9 MB - Last synced at: 10 days ago - Pushed at: over 5 years ago - Stars: 50 - Forks: 11

EPSOFT/Embedding

Embedding

Language: Jupyter Notebook - Size: 38.1 KB - Last synced at: 6 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

seungjaeryanlee/awesome-optimizers

A curated list of optimizers for machine learning.

Size: 4.88 KB - Last synced at: 10 days ago - Pushed at: over 1 year ago - Stars: 8 - Forks: 0

ndoll1998/lightgrad

Lightweight automatic differentiation library

Language: Jupyter Notebook - Size: 1.44 MB - Last synced at: 6 months ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

neyudin/AdaptiveGNMethodEquations

Code to conduct experiments for the paper Adaptive Gauss-Newton Method for Solving Systems of Nonlinear Equations.

Language: Python - Size: 10.7 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

neyudin/AcceleratedGNMethodEquations

Code to conduct experiments for the paper Regularization and acceleration of Gauss-Newton method.

Language: Python - Size: 11.7 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

devzhk/cgds-package

Package for CGD and ACGD optimizers

Language: Python - Size: 50.8 KB - Last synced at: 4 months ago - Pushed at: about 3 years ago - Stars: 20 - Forks: 4

chanzer0/NBA-DFS-Tools

A set of NBA optimizers and GPP tools to help you win daily fantasy sports

Language: Python - Size: 1.63 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 15 - Forks: 10

tensor-fusion/sophia-jax

JAX implementation of 'Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training'

Language: Python - Size: 259 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

Bayunova28/Transportation_Stock_Analysis

This repository contains about final project for bachelor degree

Language: Jupyter Notebook - Size: 391 MB - Last synced at: 5 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

GameMaximizer/GameMaximizer

This exe file give you up 50+ FPS in games!

Language: Batchfile - Size: 1010 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 1 - Forks: 0

bocb-ai/bocb-ai.github.io

Official project of BOCB (A Decade’s Battle on the Bias of Vision Backbone and Optimizer)

Language: HTML - Size: 23.5 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 0

spsingh37/Image-classification

This project explores image classification on the PROPS Classification Dataset. It includes K-Nearest Neighbors, Support Vector Machine, and Softmax classifiers, and two-layer and multi-layer neural networks. The goal is to build a machine learning pipeline, experiment with optimizers, and evaluate model performance.

Language: Jupyter Notebook - Size: 15.5 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

inikishev/image-descent

Pytorch library to test optimizers by visualizing how they descend on a your images. You can draw your own custom loss landscape and see what different optimizers do.

Language: Jupyter Notebook - Size: 40.6 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

sathishkumar67/Optimizer-Comparision

Testing out Schedule vs Schedule Free Optimizer

Language: Python - Size: 366 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

vishal815/Deep-Learing-notes

Welcome to the Deep Learning Notes repository! This collection of notes is designed to provide a deep understanding, intuition, and real-world implications of deep learning. Whether you're a beginner or preparing for exams and interviews, these comprehensive and colorful notes will be your go-to readme resource.

Size: 2.93 KB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

robjsliwa/adventures_in_dspy

Learn DSPy framework by coding text adventure game

Language: Python - Size: 7.81 KB - Last synced at: 6 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

mohrsalt/cifar10-ImageClassification-CNN

A project made for understanding how to construct CNN models from the very basics and how to choose various combinations of hyperparameters to improve accuracy of predictions

Language: Jupyter Notebook - Size: 9.67 MB - Last synced at: 5 months ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

usef-kh/fer

Code for the paper "Facial Emotion Recognition: State of the Art Performance on FER2013"

Language: Jupyter Notebook - Size: 550 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 89 - Forks: 33

dopestcardi/cardis-optimizer

Cardis Optimizer is a simple but complex optimizer that will have your pc running better in minutes!

Size: 167 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

lessw2020/QuantFour_AdamW_Cuda

Fused 4bit AdamW in Cuda

Language: Python - Size: 228 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

amkatrutsa/dl-opt

Tutorials on optimizers for deep neural networks

Size: 4.88 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

timvvvht/Neural-Networks-and-Optimizers-from-scratch

Neural Networks and optimizers from scratch in NumPy, featuring newer optimizers such as DemonAdam or QHAdam.

Language: Jupyter Notebook - Size: 498 KB - Last synced at: 23 days ago - Pushed at: over 4 years ago - Stars: 5 - Forks: 3

Joe-Khawand/loss-landscape Fork of tomgoldstein/loss-landscape

Uptodate fork for visualizing the loss landscape of neural nets

Language: Python - Size: 1.88 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

OrigamiDream/lion-tf

Lion - EvoLved Sign Momentum w/ New Optimizer API in TensorFlow 2.11+

Language: Python - Size: 4.88 KB - Last synced at: 6 days ago - Pushed at: over 2 years ago - Stars: 10 - Forks: 1

Yonghongwei/Gradient-Centralization

A New Optimization Technique for Deep Neural Networks

Language: Python - Size: 558 KB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 527 - Forks: 79

hfxunlp/transformer 📦

Neutron: A pytorch based implementation of Transformer and its variants.

Language: Python - Size: 887 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 63 - Forks: 11

EpicDima/Optimizers

This is an application for showing how optimization algorithms work

Language: Python - Size: 190 KB - Last synced at: about 2 years ago - Pushed at: about 5 years ago - Stars: 2 - Forks: 0

AaronThomasJoseph/DL-Projects

This GitHub repository contains the code used for CS-671: Introduction to Deep Learning course offered by IIT Mandi during the Even Semester of 2022. The repository includes the implementations of various deep learning algorithms and techniques covered in the course.

Language: Jupyter Notebook - Size: 31.7 MB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

vmad18/sophia

implementation of sophia (Second-Order cliPped stocHastic optimizAtion)

Language: Python - Size: 15.6 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

xin-pu/ML-Sharp

ML with C#

Language: C# - Size: 1.42 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

Jitensid/Numras

Language: Jupyter Notebook - Size: 104 KB - Last synced at: over 1 year ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 0

dunnkers/optimizers-sign-language 📦

Benchmarking Optimizers for Sign Language detection

Language: Jupyter Notebook - Size: 33.3 MB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0

Kockiee/Kockie-Optimizer-Python

Aplicação Windows desenvolvida por mim para otimização tanto do sistema como de certos jogos

Language: Python - Size: 52.4 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 12

Raffaelbdl/hyperbolic-nn-haiku

dm-haiku implementation of hyperbolic neural networks

Language: Python - Size: 9.04 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

dzonimn/Zoolia.jl

Julia implementations of various animal-inspired optimizers

Language: Julia - Size: 9.77 KB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 3 - Forks: 0

ShivamShrirao/dnn_from_scratch

A high level deep learning library for Convolutional Neural Networks,GANs and more, made from scratch(numpy/cupy implementation).

Language: Python - Size: 18.4 MB - Last synced at: over 2 years ago - Pushed at: about 4 years ago - Stars: 26 - Forks: 4

Cabbagito/Convolutional-Neural-Networks-from-Scratch

Python Library for creating and training CNNs. Implemented from scratch.

Language: Python - Size: 17.7 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

nestordemeure/flaxOptimizers

A collection of optimizers, some arcane others well known, for Flax.

Language: Python - Size: 59.6 KB - Last synced at: over 2 years ago - Pushed at: about 4 years ago - Stars: 28 - Forks: 1

SameetAsadullah/Neural-Network-Implementation

Neural Network implemented with different Activation Functions i.e, sigmoid, relu, leaky-relu, softmax and different Optimizers i.e, Gradient Descent, AdaGrad, RMSProp, Adam. You can choose different loss functions as well i.e, cross-entropy loss, hinge-loss, mean squared error (MSE)

Language: Jupyter Notebook - Size: 5.86 KB - Last synced at: 3 months ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0