Option to minimise dependencies?

[takluyver]

The recent Ultralytics incident had me thinking about supply chain attacks. Twine obviously needs to have access to upload credentials, but at present it can only be installed with quite a lot of dependencies. If any of these were compromised, they could get into environments where twine is used to upload other packages, and potentially upload malicious versions of those. This is somewhat mitigated by the recommended Github action pinning all dependency versions, but that's probably not bulletproof, and it doesn't help projects not using that (e.g. on Gitlab).

I've included the log (installing from current main branch) below , but I think there are 3 main groups:

1. keyring and all of its dependencies (cryptography, jeepney, pycparser...)

• The argument seems to be that it must be as easy as possible for users to store credentials in keyring instead of in a .pypirc file. But this was stated 3 years ago (Keyring as an extra_requires? #837), and since then it's increasingly common and encouraged to upload from CI platforms, while it's less convenient to upload locally. CI platforms generally offer ways of accessing a secret which don't need keyring.

2. readme_renderer and its dependencies (docutils, pygments, nh3)

• Used by twine check. We only actually want to check that rst is valid, but doing that pulls in a syntax-highlighting library and a compiled HTML sanitiser. It also can't check Markdown without an extra dependency (cmarkgfm).

3. rich and its dependencies (markdown-it-py, mdurl)

• twine has a sprinkling of colour in the output and some nice progress bars. I like both of these things, but again the shift to publishing from CI makes slick terminal UI seem less important - you're less likely to be watching the upload in progress. And these things do not intrinsically require a markdown parser, as far as I can tell.

(I'm not describing the dependencies which seem more or less essential, like requests)

Is it worth trying to do trim this down a bit? We could either try to find/make alternatives to the direct dependencies which have fewer transitive dependencies, move some functionality to optional dependencies, or split out a core upload package which could be used directly on CI platforms, while keeping twine for uploading from a local build. 🤷

Installation log

$ /tmp/venv-test-install-twine/bin/pip install .
Processing /home/takluyver/Code/twine
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pkginfo>=1.8.1 (from twine==6.0.2.dev25+g1703ae7)
  Using cached pkginfo-1.12.0-py3-none-any.whl.metadata (12 kB)
Collecting readme-renderer>=35.0 (from twine==6.0.2.dev25+g1703ae7)
  Using cached readme_renderer-44.0-py3-none-any.whl.metadata (2.8 kB)
Collecting requests>=2.20 (from twine==6.0.2.dev25+g1703ae7)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting requests-toolbelt!=0.9.0,>=0.8.0 (from twine==6.0.2.dev25+g1703ae7)
  Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB)
Collecting urllib3>=1.26.0 (from twine==6.0.2.dev25+g1703ae7)
  Using cached urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
Collecting keyring>=15.1 (from twine==6.0.2.dev25+g1703ae7)
  Using cached keyring-25.5.0-py3-none-any.whl.metadata (20 kB)
Collecting rfc3986>=1.4.0 (from twine==6.0.2.dev25+g1703ae7)
  Using cached rfc3986-2.0.0-py2.py3-none-any.whl.metadata (6.6 kB)
Collecting rich>=12.0.0 (from twine==6.0.2.dev25+g1703ae7)
  Using cached rich-13.9.4-py3-none-any.whl.metadata (18 kB)
Collecting packaging (from twine==6.0.2.dev25+g1703ae7)
  Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
Collecting id (from twine==6.0.2.dev25+g1703ae7)
  Downloading id-1.5.0-py3-none-any.whl.metadata (5.2 kB)
Collecting jaraco.classes (from keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached jaraco.classes-3.4.0-py3-none-any.whl.metadata (2.6 kB)
Collecting jaraco.functools (from keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached jaraco.functools-4.1.0-py3-none-any.whl.metadata (2.9 kB)
Collecting jaraco.context (from keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached jaraco.context-6.0.1-py3-none-any.whl.metadata (4.1 kB)
Collecting SecretStorage>=3.2 (from keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached SecretStorage-3.3.3-py3-none-any.whl.metadata (4.0 kB)
Collecting jeepney>=0.4.2 (from keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached jeepney-0.8.0-py3-none-any.whl.metadata (1.3 kB)
Collecting nh3>=0.2.14 (from readme-renderer>=35.0->twine==6.0.2.dev25+g1703ae7)
  Using cached nh3-0.2.19-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)
Collecting docutils>=0.21.2 (from readme-renderer>=35.0->twine==6.0.2.dev25+g1703ae7)
  Using cached docutils-0.21.2-py3-none-any.whl.metadata (2.8 kB)
Collecting Pygments>=2.5.1 (from readme-renderer>=35.0->twine==6.0.2.dev25+g1703ae7)
  Using cached pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.20->twine==6.0.2.dev25+g1703ae7)
  Using cached charset_normalizer-3.4.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests>=2.20->twine==6.0.2.dev25+g1703ae7)
  Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting certifi>=2017.4.17 (from requests>=2.20->twine==6.0.2.dev25+g1703ae7)
  Using cached certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=12.0.0->twine==6.0.2.dev25+g1703ae7)
  Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=12.0.0->twine==6.0.2.dev25+g1703ae7)
  Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Collecting cryptography>=2.0 (from SecretStorage>=3.2->keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Downloading cryptography-44.0.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.7 kB)
Collecting more-itertools (from jaraco.classes->keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached more_itertools-10.5.0-py3-none-any.whl.metadata (36 kB)
Collecting cffi>=1.12 (from cryptography>=2.0->SecretStorage>=3.2->keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached cffi-1.17.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=2.0->SecretStorage>=3.2->keyring>=15.1->twine==6.0.2.dev25+g1703ae7)
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Using cached keyring-25.5.0-py3-none-any.whl (39 kB)
Using cached pkginfo-1.12.0-py3-none-any.whl (32 kB)
Using cached readme_renderer-44.0-py3-none-any.whl (13 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
Using cached rfc3986-2.0.0-py2.py3-none-any.whl (31 kB)
Using cached rich-13.9.4-py3-none-any.whl (242 kB)
Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
Downloading id-1.5.0-py3-none-any.whl (13 kB)
Using cached packaging-24.2-py3-none-any.whl (65 kB)
Using cached certifi-2024.8.30-py3-none-any.whl (167 kB)
Using cached charset_normalizer-3.4.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB)
Using cached docutils-0.21.2-py3-none-any.whl (587 kB)
Using cached idna-3.10-py3-none-any.whl (70 kB)
Using cached jeepney-0.8.0-py3-none-any.whl (48 kB)
Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Using cached nh3-0.2.19-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (748 kB)
Using cached pygments-2.18.0-py3-none-any.whl (1.2 MB)
Using cached SecretStorage-3.3.3-py3-none-any.whl (15 kB)
Using cached jaraco.classes-3.4.0-py3-none-any.whl (6.8 kB)
Using cached jaraco.context-6.0.1-py3-none-any.whl (6.8 kB)
Using cached jaraco.functools-4.1.0-py3-none-any.whl (10 kB)
Downloading cryptography-44.0.0-cp39-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 2.0 MB/s eta 0:00:00
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached more_itertools-10.5.0-py3-none-any.whl (60 kB)
Using cached cffi-1.17.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: twine
  Building wheel for twine (pyproject.toml) ... done
  Created wheel for twine: filename=twine-6.0.2.dev25+g1703ae7-py3-none-any.whl size=39609 sha256=45d66b27fe621cff995e07ba832b5f6b70085d7f642ac2519f217911c48e4329
  Stored in directory: /tmp/pip-ephem-wheel-cache-2_4jfjc9/wheels/83/bf/8b/233621bf4658dcaaead6824e2be9d1029d5c10419740228570
Successfully built twine
Installing collected packages: nh3, urllib3, rfc3986, Pygments, pycparser, pkginfo, packaging, more-itertools, mdurl, jeepney, jaraco.context, idna, docutils, charset-normalizer, certifi, requests, readme-renderer, markdown-it-py, jaraco.functools, jaraco.classes, cffi, rich, requests-toolbelt, id, cryptography, SecretStorage, keyring, twine
Successfully installed Pygments-2.18.0 SecretStorage-3.3.3 certifi-2024.8.30 cffi-1.17.1 charset-normalizer-3.4.0 cryptography-44.0.0 docutils-0.21.2 id-1.5.0 idna-3.10 jaraco.classes-3.4.0 jaraco.context-6.0.1 jaraco.functools-4.1.0 jeepney-0.8.0 keyring-25.5.0 markdown-it-py-3.0.0 mdurl-0.1.2 more-itertools-10.5.0 nh3-0.2.19 packaging-24.2 pkginfo-1.12.0 pycparser-2.22 readme-renderer-44.0 requests-2.32.3 requests-toolbelt-1.0.0 rfc3986-2.0.0 rich-13.9.4 twine-6.0.2.dev25+g1703ae7 urllib3-2.2.3

[takluyver]

Anyone? I might try playing with a fork of twine to trim dependencies.

[sigmavirus24]

I don't feel strongly about this being necessary personally

[woodruffw]

Same, personally -- IMO the overall dependency closure here isn't too big or unwieldy. I think maybe making keyring optional would probably be the biggest win in terms of dep reduction, although I suspect that there are a lot of people using it outside of the PyPI + CI/CD happy path who wouldn't love a change there (and a lot of those deps are central ones by prominent Python community members anyways).