GitHub topics: second-order-optimization
inikishev/torchzero
Modular optimization library for PyTorch.
Language: Python - Size: 1.91 MB - Last synced at: about 24 hours ago - Pushed at: 1 day ago - Stars: 4 - Forks: 0
gpauloski/kfac-pytorch
Distributed K-FAC preconditioner for PyTorch
Language: Python - Size: 942 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 87 - Forks: 25
lixilinx/psgd_torch
Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)
Language: Python - Size: 3.08 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 175 - Forks: 11
amirgholami/PyHessian
PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks
Language: Jupyter Notebook - Size: 2.05 MB - Last synced at: 8 days ago - Pushed at: about 1 year ago - Stars: 743 - Forks: 124
OPTAMI/OPTAMI
This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.
Language: Python - Size: 1.56 MB - Last synced at: 1 day ago - Pushed at: 11 days ago - Stars: 24 - Forks: 7
evanatyourservice/kron_torch
An implementation of PSGD Kron second-order optimizer for PyTorch
Language: Python - Size: 299 KB - Last synced at: 24 days ago - Pushed at: 3 months ago - Stars: 91 - Forks: 6
dscamiss/newt
The Newton-like learning rate scheduler
Language: Python - Size: 184 KB - Last synced at: 7 days ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0
evanatyourservice/psgd_jax
Implementation of PSGD optimizer in JAX
Language: Python - Size: 329 KB - Last synced at: 8 days ago - Pushed at: 6 months ago - Stars: 33 - Forks: 2
dscamiss/hesse
PyTorch tools for Hessian-related operations
Language: Python - Size: 148 KB - Last synced at: 5 days ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0
dahyun-kang/newoptpy
Newton’s second-order optimization methods in python
Language: Python - Size: 21.1 MB - Last synced at: 12 days ago - Pushed at: about 3 years ago - Stars: 4 - Forks: 0
ltatzel/PyTorchHessianFree
PyTorch implementation of the Hessian-free optimizer
Language: Python - Size: 161 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 34 - Forks: 6
cor3bit/awesome-soms
A curated list of resources for second-order stochastic optimization
Size: 17.6 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 1
evanatyourservice/flat-sophia
sophia optimizer further projected towards flat areas of loss landscape
Language: Python - Size: 269 KB - Last synced at: 3 months ago - Pushed at: 6 months ago - Stars: 2 - Forks: 0
cor3bit/somax
Stochastic Second-Order Methods in JAX
Language: Python - Size: 445 KB - Last synced at: about 2 months ago - Pushed at: 9 months ago - Stars: 3 - Forks: 1
EmbersArc/Epigraph
A C++ interface to formulate and solve linear, quadratic and second order cone problems.
Language: C++ - Size: 2.78 MB - Last synced at: 3 months ago - Pushed at: almost 4 years ago - Stars: 155 - Forks: 16
hsivan/fosi
FOSI library for improving first order optimizers with second order information
Language: Python - Size: 2.46 MB - Last synced at: 6 months ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 0
mizhaan23/crpn_algo
This directory contains the source code of the experiments as shown in our main paper. It is still work in progress.
Language: Python - Size: 313 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0
vinnik-dmitry07/full-batch
Super-Convergence on CIFAR10
Language: Python - Size: 4.34 MB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0
yorkerlin/StructuredNGD-DL
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Language: Python - Size: 40 KB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 1
Peachypie98/Hessian-Network-Learning
Learning Network using Hessian in PyTorch
Language: Jupyter Notebook - Size: 429 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
AntoinePassemiers/Beyond-Gradient-Descent
Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose
Language: Python - Size: 3.4 MB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 9 - Forks: 3
leotau2018/AASA_CODE
An accelerated active‑set algorithm for a quadratic semidefinite program with general constraints
Language: MATLAB - Size: 3.32 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
elated-sawyer/Convex-Relaxation
Solve DNN relaxations of nonconvex quadratic programming problems.
Language: Julia - Size: 105 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0
lixilinx/psgd_tf
Tensorflow implementation of preconditioned stochastic gradient descent
Language: Python - Size: 7.4 MB - Last synced at: 7 months ago - Pushed at: over 1 year ago - Stars: 34 - Forks: 14
amirgholami/adahessian
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Language: Python - Size: 1.95 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 225 - Forks: 47
CharlieDinh/FEDL
FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)
Language: Python - Size: 126 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 44 - Forks: 22
tomoleary/hessianlearn
Hessian-based stochastic optimization in TensorFlow and keras
Language: Python - Size: 4.21 MB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 28 - Forks: 3
IST-DASLab/EFCP
The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:
Language: Python - Size: 3.39 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 2 - Forks: 0
jmdvinodjmd/LIBS2ML
LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms
Language: C++ - Size: 221 KB - Last synced at: almost 2 years ago - Pushed at: almost 4 years ago - Stars: 9 - Forks: 7
S0852306/Numpy-Implementation-of-Neural-Nets
Numpy implementation of Neural Networks with SGDM, ADAM and BFGS solvers, suitable for surface fitting
Language: Jupyter Notebook - Size: 332 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
CharlieDinh/FEDL_pytorch
This repository implements FEDL using pytorch
Language: Python - Size: 10.7 MB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 41 - Forks: 8
kenshi84/compatible-intrinsic-triangulations
Compatible Intrinsic Triangulations (SIGGRAPH 2022)
Language: C++ - Size: 2.76 MB - Last synced at: about 2 years ago - Pushed at: about 3 years ago - Stars: 17 - Forks: 3
CharlieDinh/DONE
Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)
Language: Jupyter Notebook - Size: 58.8 MB - Last synced at: almost 2 years ago - Pushed at: about 3 years ago - Stars: 16 - Forks: 1
hiroyuki-kasai/Subsampled-RTR
Subsampled Riemannian trust-region (RTR) algorithms
Language: MATLAB - Size: 4.95 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 5 - Forks: 5
sayarghoshroy/Optimization_and_Learning
Concepts and algorithms in core learning theory
Language: Jupyter Notebook - Size: 4.6 MB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 1 - Forks: 1
qiuweili/altmin
Second-Order Convergence of Alternating Minimizations
Size: 867 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 4 - Forks: 2
yangorwell/NGPlus
NG+: A new second-order optimizer for deep learning
Language: Python - Size: 19.9 MB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 4
Edoardo-Tronci/ALA
Adaptive Linesearch Algorithm
Language: Python - Size: 42.3 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
aniruddhavpatil/second-order-model-selection
Regularization, Bayesian Model Selection and k-fold Cross-Validation Selection
Language: Python - Size: 1.69 MB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0
riccardocadei/adahessian
Discussion of advantages and disadvantages of AdaHessian, a state-of-the-art Second Order Methods over First Order Methods on a Non-Convex Optimization Problem (digits classification on MNIST database using ResNet18). - @ EPFL
Language: Jupyter Notebook - Size: 1.2 GB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0
tomoleary/MatrixFreeNewton.jl
Prototyping of matrix free Newton methods in Julia
Language: Julia - Size: 96.7 KB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 7 - Forks: 0