Skip to content

Code to create orthogonal optimisers from torch.optim.Optimizer

License

Notifications You must be signed in to change notification settings

MarkTuddenham/Orthogonal-Optimisers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

f2d5bc1 · Oct 13, 2021

History

5 Commits
Oct 1, 2021
Sep 15, 2021
Sep 15, 2021
Oct 13, 2021
Sep 15, 2021

Repository files navigation

Orthogonalised Optimisers

Anonymised Repo

If you're looking at the anonymised repo then you can't clone/download it. However, you can just download the relevant files individually:

mkdir -p Orthogonal-Optimisers
cd Orthogonal-Optmiser
mkdir src/orth_optim
wget https://anonymous.4open.science/api/repo/Orthogonal-Optimisers/file/setup.py
wget https://anonymous.4open.science/api/repo/Orthogonal-Optimisers/file/src/orth_optim/__init__.py -O src/orth_optim/__init__.py

Install package

git clone https://github.com/MarkTuddenham/Orthogonal-Optimisers.git
cd Orthogonal-Optimisers
pip install .

or

pip install git+https://github.com/MarkTuddenham/Orthogonal-Optimisers.git#egg=orth_optim

Usage

And then at the top of your main python script:

from orth_optim import hook
hook()

Now the torch optimisers have an orthogonal option, e.g:

torch.optim.SGD(model.parameters(),
                lr=1e-3,
                momentum=0.9,
                orth=True)

Custom Optimisers

If you have a custom optimiser you can apply the orthogonalise decorator.

from orth_optim import orthogonalise

@orthogonalise
class LARS(torch.optim.Optimizer):
	...

About

Code to create orthogonal optimisers from torch.optim.Optimizer

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages