ChefBuildError from FileNotFoundError: [WinError 3] during installation of a dependency while "pip wheel --no-cache-dir --use-pep517 <dependency> works

[Gwenn-LR]

Description

Hi ! First of all, thanks for the amazing work of all your team !

I've stumbled upon an error which I think migth interest you.

While solving dependency requirements during the installation, poetry had to install pytils==0.4.1. However it resulted in this ChefBuildError:

PS C:\<...>poetry install --sync
Installing dependencies from lock file

Package operations: 26 installs, 2 updates, 0 removals

  - Installing pytils (0.4.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  Traceback (most recent call last):
    File "C:\<...>\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 389, in <module>
      main()
    File "C:\<...>\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 373, in main 
      json_out["return_val"] = hook(**hook_input["kwargs"])
    File "C:\<...>\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 280, in build_wheel
      return _build_backend().build_wheel(
    File "C:\<...>\Temp\tmp2a_4fbgy\.venv\lib\site-packages\poetry\core\masonry\api.py", line 58, in build_wheel
      return WheelBuilder.make_in(
    File "C:\<...>\Temp\tmp2a_4fbgy\.venv\lib\site-packages\poetry\core\masonry\builders\wheel.py", line 88, in make_in
      wb.build(target_dir=directory)
    File "C:\<...>\Temp\tmp2a_4fbgy\.venv\lib\site-packages\poetry\core\masonry\builders\wheel.py", line 105, in build
      target_dir.mkdir()
    File "C:\<...>\Programs\Python\Python310\lib\pathlib.py", line 1175, in mkdir
      self._accessor.mkdir(self, mode)
  FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\<...>\\pypoetry\\Cache\\artifacts\\96\\84\\bd\\8764e60a986edd98023d64f08c6c1d6d16f2156eb2a2b4740087c8a022'  
  

  at ~\<...>\pypoetry\venv\Lib\site-packages\poetry\installation\chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pytils (0.4.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pytils (==0.4.1)"'.

I've tried to install directly from the repository with:

poetry add "git+https://github.com/last-partizan/pytils.git"

I've also added the flag --no-cache to the previous command, but I was getting the same error over and over.

Since I had no issue on Ubuntu 22.04 and it happened working on a system under Win 11, it might comes from an interaction with this platform.

Workarounds

Finally I've decided to give a try to the Note at the end of the message to get a better idea of the issue and tried this command:

python -m pip wheel --no-cache-dir --use-pep517 "pytils (==0.4.1)"

It worked as a charm, it generated the .whl associated to the package which I could include to the poetry.toml with:

pytils = {path = "dist/pytils-0.4.1-py3-none-any.whl"}

Finally I've been able to resume my first command and the installation has finished without any error.

Poetry Installation Method

install.python-poetry.org

Operating System

Win 11

Poetry Version

1.8.4

Poetry Configuration

cache-dir = "C:\\<...>\\AppData\\Local\\pypoetry\\Cache"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.pytorch_cuda121.url = "https://download.pytorch.org/whl/cu121"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\<...>\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

I will, next time I have an issue with Poetry which require to open an issue.

[abn]

@Gwenn-LR thank you for your report, but I have been unable to reproduce this on my machine.

It would help if you can,

1. Setup a new project.
2. Try adding via poetry add -vvv 'pytils==0.4.1'.
3. Post the logs here or to a gist and link it here.

[dimbleby]

it'll be windows long paths, it's always windows long paths

[abn]

@Gwenn-LR if as @dimbleby mentioned, the issue is long paths, please see https://docs.python.org/3.13/using/windows.html#removing-the-max-path-limitation.

[Gwenn-LR]

At first I thought @dimbleby call was a good one but LongPathsEnabled is already set... :/

Here is my log from poetry add -vvv 'pytils==0.4.1' in a new poetry env spawned from a virtual environment set with venv: https://gist.github.com/Gwenn-LR/6e7ad1429cdd072a42e4492edfbab134

While checking its content, I think I've found the issue, but I can't understand why it happens.

poetry tries to build from

'C:\\Users\\leroch\\AppData\\Local\\pypoetry\\Cache\\artifacts\\96\\84\\bd\\8764e60a986edd98023d64f08c6c1d6d16f2156eb2a2b4740087c8a022'

However, the folder where I can find the pytils-0.4.1.tar is:

C:\Users\leroch\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\artifacts\96\84\bd\8764e60a986edd98023d64f08c6c1d6d16f2156eb2a2b4740087c8a022

I think the issue could come from my python setup : I've installed it from the Microsoft store and I can't remember all the steps I've followed after to setup a working poetry. If you have no idea how to change my current behaviour, I might reinstall poetry from zero when my current project is stabilized and I'll keep you in touch.

In any case, thanks to you @abn and @dimbleby for your quick answers !

[abn]

Ah, your issue might be related to python-poetry/install.python-poetry.org#135 perhaps?

[Gwenn-LR]

Yeap clearly, good catch ! But I don't understand how the other 100+ packages of my project haven't not came accross the same issue, only this one...

I'll try another day to reinstall from scratch Python and I'll keep in touch with you when its done. Meanwhile I back crazytan enhancement suggestion to at least warn the user if a Microsoft Store Python is detected if it is possible.

[Gwenn-LR]

I don't know if it can help you get a better understanding of the issue, but it just happened on another branch of my project with matplotlib.

[abn]

@Gwenn-LR honestly, right now the best option for you might be to uninstall the MS Store Python and install a system Python version using the installer at https://www.python.org/downloads/. Based on what I can read in the above issue, it could keep happening if you don't.

[abn]

@Gwenn-LR alternatively try setting the cache directory manually.

poetry config cache-dir /path/to/home/directory/pypoetry

[Gwenn-LR]

@Gwenn-LR alternatively try setting the cache directory manually.

poetry config cache-dir /path/to/home/directory/pypoetry

I'm currently remote working on a station that is not the one I use normally, so this solution is the best for me at this time.

@Gwenn-LR honestly, right now the best option for you might be to uninstall the MS Store Python and install a system Python version using the installer at https://www.python.org/downloads/. Based on what I can read in the above issue, it could keep happening if you don't.

However I'll try a clean install next time !

In any case, thanks again for the time you've granted my issue ! Have a good weekend :)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.