Skip to content

Commit

Permalink
Handle broken metadata by discarding package, or give a useful error
Browse files Browse the repository at this point in the history
Fixes jazzband#2118.

Now, with this in requirements.txt:

    django-debug-toolbar
    django-geojson
    wagtail>=4.2,<4.3

I get:

    $ env/bin/pip-compile --resolver=backtracking --generate-hashes --allow-unsafe --output-file requirements.txt requirements.in
        error: subprocess-exited-with-error

        × python setup.py egg_info did not run successfully.
        │ exit code: 1
        ╰─> [18 lines of output]
            /home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/dist.py:268: UserWarning: Unknown distribution option: 'test_suite'
              warnings.warn(msg)
            Traceback (most recent call last):
              File "<string>", line 2, in <module>
              File "<pip-setuptools-caller>", line 34, in <module>
              File "/tmp/pip-resolve-o2r8fhgd/geojson_90f4efbe8a2247f9a1883c03e4988381/setup.py", line 6, in <module>
                setup(name          = "geojson",
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
                return distutils.core.setup(**attrs)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 146, in setup
                _setup_distribution = dist = klass(attrs)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 289, in __init__
                self.metadata.version = self._normalize_version(self.metadata.version)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 325, in _normalize_version
                normalized = str(Version(version))
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
                match = self._regex.search(version)
            TypeError: cannot use a string pattern on a bytes-like object
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
    Discarding django-geojson (from -r requirements.in (line 11)) to proceed the resolution.
    It looks like it has a release with broken metadata, causing this resolver to fail.
    You may try to pin that package's version (probably to a recent one), so the resolver doesn't consider the broken release.

The exception is still shown. Also we log a warning to tell the user the package was discarded.

The way to find the offending pacakge is crude (regex in error message string).

When the regex search for package name fails, we re-raise the error so no requirements file is generated.

In that case we log an error, which is hopefully helpful.
  • Loading branch information
khink committed Jul 31, 2024
1 parent 5330964 commit 9281ee8
Showing 1 changed file with 31 additions and 1 deletion.
32 changes: 31 additions & 1 deletion piptools/resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@

import collections
import copy
import re
from abc import ABCMeta, abstractmethod
from functools import partial
from itertools import chain, count, groupby
from typing import Any, Container, DefaultDict, Iterable, Iterator

import click
from pip._internal.exceptions import DistributionNotFound
from pip._internal.exceptions import DistributionNotFound, MetadataGenerationFailed
from pip._internal.operations.build.build_tracker import (
get_build_tracker,
update_env_context_manager,
Expand Down Expand Up @@ -670,6 +671,35 @@ def _do_resolve(

return False

except MetadataGenerationFailed as err:
# Find (part of) broken package name in the error message
match = re.search('name\s*=\s*"(\w+)"', str(err.__cause__.context))
if match:
partial_name = match.group(1)
possible_constraints = [c for c in self.constraints if partial_name in c.name]
for constraint in possible_constraints:
# Remove existing constraint that *might* cause the error
log.warning(
f"Discarding {constraint} to proceed the resolution.\n"
"It looks like it has a release with broken metadata, causing this resolver to fail.\n"
"You may try to pin that package's version (probably to a recent one), "
"so the resolver doesn't consider the broken release."
)
self.constraints.remove(constraint)

# We remove only one constraint at a time.
# If there are multiple candidates, we'll find them in the next round.
return False

# We also get here if there was a match, but possible_constraints is an empty list.
log.error(
"A package release has broken metadata, but we can't tell which package it is.\n "
"Try reading the error message above to identify the package.\n"
"Next, you may try to pin that package's version (probably to a recent one), "
"so the resolver doesn't consider the broken release."
)
raise

return True

def _get_install_requirements(
Expand Down

0 comments on commit 9281ee8

Please sign in to comment.