Support version pinning Spine packages

[suvayu]

The logic for version pinning is more complex, as we have to check all packages and update the pyproject.toml even if there are no new commits since the last release.

[ptsavol]

This would be nice for spinetoolbox-0.7.1. The METADATA file for spinetoolbox-0.7.0 looks like this

Metadata-Version: 2.1
Name: spinetoolbox
Version: 0.7.0
Summary: An application to define, manage, and execute various energy system simulation models
Author-email: Spine Project consortium <[email protected]>
License: LGPL-3.0-or-later
Project-URL: Documentation, https://spine-toolbox.readthedocs.io/
Project-URL: Repository, https://github.com/spine-tools/Spine-Toolbox
Keywords: energy system modelling,workflow,optimisation,database
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Classifier: Operating System :: OS Independent
Requires-Python: <3.12,>=3.8.1
Description-Content-Type: text/markdown
License-File: COPYING
License-File: COPYING.LESSER
Requires-Dist: pyside6 >=6.5.0
Requires-Dist: psutil <5.9.2
Requires-Dist: jupyter-client >=6.0
Requires-Dist: qtconsole >=5.1
Requires-Dist: sqlalchemy >=1.3
Requires-Dist: spinedb-api >=0.30.0
Requires-Dist: spine-engine >=0.23.0
Requires-Dist: numpy >=1.20.2
Requires-Dist: matplotlib >=3.5
Requires-Dist: scipy >=1.7.1
Requires-Dist: networkx >=2.6
Requires-Dist: pandas >=1.3.2
Requires-Dist: pygments >=2.8
Requires-Dist: jill >=0.9.2
Requires-Dist: pyzmq >=21.0
Requires-Dist: spine-items >=0.21.0
Provides-Extra: dev
Requires-Dist: coverage[toml] ; extra == 'dev'
...

I believe to make future releases robust, we need to pin spine packages to the exact version, e.g. spine-items==0.21.0 and 3rd party dependencies to less than the current minor versions, e.g. PySide6 < 6.6, pyzmq < 21.1, numpy < 1.21, etc.

Or are there any alternatives to this?

[suvayu]

The part I'm unsure about is that version pinning means a release would require the following steps:

$ edit pyproject.toml  # spine-items>=0.20.0 -> spine-items==0.21.0
$ git commit -i pyproject.toml -m "Release 0.7.0"
$ git tag 0.7.0 HEAD   # the Spine-Toolbox release
$ edit pyproject.toml  # spine-items==0.21.0 -> spine-items>=0.21.0
$ git commit -i pyproject.toml -m "Ready for development"

This actively breaks compatibility of the development version with older releases (except for the most recent release). Is that acceptable for now?

[suvayu]

and 3rd party dependencies to less than the current minor versions, e.g. PySide6 < 6.6, pyzmq < 21.1, numpy < 1.21, etc.

Here I would limit it to known difficult dependencies. So for example no upper limit on numpy/pyzmq, while putting an upper limit on PySide. My reasoning: upper limits don't age well. Every year a version of Python EOLs (3.8 next year), and that essentially would break all previous releases because these 3rd party dependencies will move on, and PyPI will only have source tarballs, and pip install will fail because the users machine cannot build it because it has a newer Python version.

[erkkar]

The part I'm unsure about is that version pinning means a release would require the following steps:

$ edit pyproject.toml  # spine-items>=0.20.0 -> spine-items==0.21.0
$ git commit -i pyproject.toml -m "Release 0.7.0"
$ git tag 0.7.0 HEAD   # the Spine-Toolbox release
$ edit pyproject.toml  # spine-items==0.21.0 -> spine-items>=0.21.0
$ git commit -i pyproject.toml -m "Ready for development"

This actively breaks compatibility of the development version with older releases (except for the most recent release). Is that acceptable for now?

Usually pyproject.toml is used for the package dependecies with minimum working version numbers (for maximum compatibility).

Required packages for an application can be listed, e.g., in a requirements.txt file. Although there is no good way of doing this if you want to install directly from PyPI, see a recent blog post on this: https://snarky.ca/differentiating-between-writing-down-dependencies-to-use-packages-and-for-packages-themselves/.

Just as you were worrying, I would not pin and unpin the package versions between releases and development, but pin those versions somewhere else than pyproject.toml.