Topic: "second-order-optimization"
amirgholami/PyHessian
PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks
Language: Jupyter Notebook - Size: 2.05 MB - Last synced at: 17 days ago - Pushed at: about 1 year ago - Stars: 734 - Forks: 124
amirgholami/adahessian
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Language: Python - Size: 1.95 MB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 225 - Forks: 47
lixilinx/psgd_torch
Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)
Language: Python - Size: 2.72 MB - Last synced at: about 22 hours ago - Pushed at: about 23 hours ago - Stars: 175 - Forks: 12
EmbersArc/Epigraph
A C++ interface to formulate and solve linear, quadratic and second order cone problems.
Language: C++ - Size: 2.78 MB - Last synced at: about 2 months ago - Pushed at: almost 4 years ago - Stars: 155 - Forks: 16
evanatyourservice/kron_torch
An implementation of PSGD Kron second-order optimizer for PyTorch
Language: Python - Size: 299 KB - Last synced at: 1 day ago - Pushed at: about 2 months ago - Stars: 91 - Forks: 6
gpauloski/kfac-pytorch
Distributed K-FAC preconditioner for PyTorch
Language: Python - Size: 930 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 85 - Forks: 25
CharlieDinh/FEDL
FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)
Language: Python - Size: 126 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 44 - Forks: 22
CharlieDinh/FEDL_pytorch
This repository implements FEDL using pytorch
Language: Python - Size: 10.7 MB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 41 - Forks: 8
ltatzel/PyTorchHessianFree
PyTorch implementation of the Hessian-free optimizer
Language: Python - Size: 161 KB - Last synced at: about 2 months ago - Pushed at: 11 months ago - Stars: 34 - Forks: 6
lixilinx/psgd_tf
Tensorflow implementation of preconditioned stochastic gradient descent
Language: Python - Size: 7.4 MB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 34 - Forks: 14
evanatyourservice/psgd_jax
Implementation of PSGD optimizer in JAX
Language: Python - Size: 329 KB - Last synced at: 7 days ago - Pushed at: 5 months ago - Stars: 33 - Forks: 2
tomoleary/hessianlearn
Hessian-based stochastic optimization in TensorFlow and keras
Language: Python - Size: 4.21 MB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 28 - Forks: 3
OPTAMI/OPTAMI
This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.
Language: Python - Size: 1.56 MB - Last synced at: 5 days ago - Pushed at: 4 months ago - Stars: 23 - Forks: 7
kenshi84/compatible-intrinsic-triangulations
Compatible Intrinsic Triangulations (SIGGRAPH 2022)
Language: C++ - Size: 2.76 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 17 - Forks: 3
CharlieDinh/DONE
Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)
Language: Jupyter Notebook - Size: 58.8 MB - Last synced at: almost 2 years ago - Pushed at: almost 3 years ago - Stars: 16 - Forks: 1
AntoinePassemiers/Beyond-Gradient-Descent
Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose
Language: Python - Size: 3.4 MB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 9 - Forks: 3
jmdvinodjmd/LIBS2ML
LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms
Language: C++ - Size: 221 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 9 - Forks: 7
tomoleary/MatrixFreeNewton.jl
Prototyping of matrix free Newton methods in Julia
Language: Julia - Size: 96.7 KB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 7 - Forks: 0
hsivan/fosi
FOSI library for improving first order optimizers with second order information
Language: Python - Size: 2.46 MB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 0
hiroyuki-kasai/Subsampled-RTR
Subsampled Riemannian trust-region (RTR) algorithms
Language: MATLAB - Size: 4.95 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 5 - Forks: 5
inikishev/torchzero
Modular optimization library for PyTorch.
Language: Python - Size: 1.45 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 4 - Forks: 0
yorkerlin/StructuredNGD-DL
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Language: Python - Size: 40 KB - Last synced at: about 2 months ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 1
qiuweili/altmin
Second-Order Convergence of Alternating Minimizations
Size: 867 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 2
dahyun-kang/newoptpy
Newton’s second-order optimization methods in python
Language: Python - Size: 21.1 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 4 - Forks: 0
cor3bit/somax
Stochastic Second-Order Methods in JAX
Language: Python - Size: 445 KB - Last synced at: 30 days ago - Pushed at: 8 months ago - Stars: 3 - Forks: 1
cor3bit/awesome-soms
A curated list of resources for second-order stochastic optimization
Size: 17.6 KB - Last synced at: 7 days ago - Pushed at: 11 months ago - Stars: 3 - Forks: 1
yangorwell/NGPlus
NG+: A new second-order optimizer for deep learning
Language: Python - Size: 19.9 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 4
evanatyourservice/flat-sophia
sophia optimizer further projected towards flat areas of loss landscape
Language: Python - Size: 269 KB - Last synced at: about 2 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0
IST-DASLab/EFCP
The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:
Language: Python - Size: 3.39 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0
riccardocadei/adahessian
Discussion of advantages and disadvantages of AdaHessian, a state-of-the-art Second Order Methods over First Order Methods on a Non-Convex Optimization Problem (digits classification on MNIST database using ResNet18). - @ EPFL
Language: Jupyter Notebook - Size: 1.2 GB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0
sayarghoshroy/Optimization_and_Learning
Concepts and algorithms in core learning theory
Language: Jupyter Notebook - Size: 4.6 MB - Last synced at: about 2 years ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 1
aniruddhavpatil/second-order-model-selection
Regularization, Bayesian Model Selection and k-fold Cross-Validation Selection
Language: Python - Size: 1.69 MB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0
dscamiss/newt
The Newton-like learning rate scheduler
Language: Python - Size: 184 KB - Last synced at: 1 day ago - Pushed at: 19 days ago - Stars: 0 - Forks: 0
dscamiss/hesse
PyTorch tools for Hessian-related operations
Language: Python - Size: 148 KB - Last synced at: 1 day ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
mizhaan23/crpn_algo
This directory contains the source code of the experiments as shown in our main paper. It is still work in progress.
Language: Python - Size: 313 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
vinnik-dmitry07/full-batch
Super-Convergence on CIFAR10
Language: Python - Size: 4.34 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
Peachypie98/Hessian-Network-Learning
Learning Network using Hessian in PyTorch
Language: Jupyter Notebook - Size: 429 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
leotau2018/AASA_CODE
An accelerated active‑set algorithm for a quadratic semidefinite program with general constraints
Language: MATLAB - Size: 3.32 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
S0852306/Numpy-Implementation-of-Neural-Nets
Numpy implementation of Neural Networks with SGDM, ADAM and BFGS solvers, suitable for surface fitting
Language: Jupyter Notebook - Size: 332 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Edoardo-Tronci/ALA
Adaptive Linesearch Algorithm
Language: Python - Size: 42.3 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
elated-sawyer/Convex-Relaxation
Solve DNN relaxations of nonconvex quadratic programming problems.
Language: Julia - Size: 105 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
