Request for publication to PyPI

[BrianJKoopman]

Can we publish sotodlib to PyPI and make somewhat regular releases? This would help packages that depend on sotodlib, for instance socs. Currently I have to specify the install in our requirements.txt file like:

sotodlib @ git+https://github.com/simonsobs/sotodlib.git@master

With the removal of 3.8 support I need to pin to an old commit. If sotodlib had been published to PyPI, pip would have been able to find a suitable fallback version for installation, and I could more simply pin to an older version in pyproject.toml.

It seems like this was being considered at one point (#105), so I hope it isn't too big an ask.

EDIT: If this does happen, is it possible to publish one version that is 3.8 compatible that I can pin to?

[tskisner]

This sounds like a great idea. Currently in soconda we just pip install master. Obviously sotodlib is more of a rolling release, but it would still be nice to have periodic tags independent of what is actually in the tag. For example, imagine a policy where:

• We tag a new release on the first workday of the month. Everyone knows when that will happen, and if you want a PR to be merged before that, then you can push to make it happen.
• We can more easily guarantee that sotodlib tags are compatible with a specific so3g tag.

Most people actively hacking on sotodlib will still install from a git checkout, but downstream dependencies can have a known working so3g + sotodlib configuration. Thoughts?

[BrianJKoopman]

Thanks @tskisner, that sounds good to me! Just to restate my question I added in an edit so it doesn't get missed -- is it possible to publish one version that is 3.8 compatible that I can pin socs to when you start publishing?

[tskisner]

We could do that, but it would require a one-off change to the version requirements. What I would recommend / volunteer to do would be:

1. Add the github workflows to publish releases on tags. Merge that to master, but don't actually make a tag yet.
2. Once (1) is working, make a long-lived py3.8 branch or similar which changes requirements back to 3.8 and modifies any other things.
3. Tag a release on that py3.8 branch to upload packages for all python versions including 3.8
4. Tag a release on the main / master branch to upload newer versions for pythons >=3.9

I can work on the PR in (1) if there is consensus, but I'm trying to finish a variety of other package things for so3g and an upstream dependency holding up soconda. Do you have a python 3.8 solution that is working for the next ~week?

[BrianJKoopman]

We could do that, but it would require a one-off change to the version requirements. What I would recommend / volunteer to do would be:

1. Add the github workflows to publish releases on tags.  Merge that to master, but don't actually make a tag yet.

2. Once (1) is working, make a long-lived `py3.8` branch or similar which changes requirements back to 3.8 and modifies any other things.

3. Tag a release on that py3.8 branch to upload packages for all python versions including 3.8

4. Tag a release on the main / master branch to upload newer versions for pythons >=3.9

I can work on the PR in (1) if there is consensus, but I'm trying to finish a variety of other package things for so3g and an upstream dependency holding up soconda. Do you have a python 3.8 solution that is working for the next ~week?

Ah, I was thinking just a manually published version with 351e8bf (and possibly later commits, if needed) reverted as the first tag on PyPI. After that I don't really mind. I wasn't trying to ask for you to maintain a separate long-lived branch. I'm going to try to move to >=3.9 in ocs/socs too, but it's going to take some time.

In the meantime I do have a pin working in the requirements file, so can definitely wait until after the holidays.

[iparask]

This is something we missed I think in #1059.

A github workflow will be great because then we can trigger the creation of soconda envs based on these releases and have a fairly stable and reproducible environments.

[BrianJKoopman]

This is something we missed I think in #1059.

Yeah, sorry, I forgot that this was an issue until socs builds broke again.

[skhrg]

On GH workflows for this, I am really big fan of the workflows I use here where I use commitizen to automatically do semver for me and then publish a new release to pypi whenever the version is bumped. This would need a slight change in dev behavior because the commit messaged would need to be formatted in a certain way but thats manageable as part of the review process.