Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pdb prompt is nonresponsive in Python 3.13.0 from python-build-standalone #12888

Open
bluthej opened this issue Oct 13, 2024 · 10 comments · May be fixed by #12961
Open

pdb prompt is nonresponsive in Python 3.13.0 from python-build-standalone #12888

bluthej opened this issue Oct 13, 2024 · 10 comments · May be fixed by #12961
Labels
plugin: capture related to the capture builtin plugin plugin: debugging related to the debugging builtin plugin type: bug problem that needs to be addressed

Comments

@bluthej
Copy link
Contributor

bluthej commented Oct 13, 2024

With Python 3.13.0 installed with uv (i.e. from python-build-standalone), the pdb prompt is completely nonresponsive when triggered either by the --pdb flag or by a breakpoint.

This can be reproduced by the following test in test_foo.py:

def test_bar():
    raise ValueError

and running:

pytest test_foo.py --pdb

I tested the same code with the official source release from python.org for 3.13.0 on the same computer and the same code works just fine, so it seems it's somehow related to the build by python-build-standalone. Note that putting a breakpoint in a regular Python script and running it in the same interpreter does work, so this seems to be an issue with the combination with pytest.

OS: Arch Linux.

Pytest version

❯ pytest --version
pytest 8.3.3

`pip list` output

❯ pip list
Package   Version
--------- -------
iniconfig 2.0.0
packaging 24.1
pip       24.2
pluggy    1.5.0
pytest    8.3.3

@bluthej bluthej changed the title pdb prompt is nonresponsive pdb prompt is nonresponsive in Python 3.13.0 from python-build-standalone Oct 13, 2024
@The-Compiler
Copy link
Member

I've never used uv before, but this was a good opportunity to give it a try:

  • uv python install
  • uv venv
  • uv pip install -e .

All that took about a second, which is indeed pretty damn crazy 🤯

And indeed I can reproduce with that. A breakpoint() in a normal Python file (outside of pytest) seems to work fine. And disabling pytest's output capturing with -s seems to fix things too!

@The-Compiler The-Compiler added type: bug problem that needs to be addressed plugin: capture related to the capture builtin plugin plugin: debugging related to the debugging builtin plugin labels Oct 13, 2024
@webknjaz
Copy link
Member

@The-Compiler
Copy link
Member

Hmm, I don't think so. I don't see such an error message, and for what it's worth, setting TERMINFO_DIRS=/etc/terminfo:/lib/terminfo:/usr/share/terminfo doesn't change anything either.

@The-Compiler
Copy link
Member

The-Compiler commented Oct 13, 2024

Oh, this one seems like a likely culprit, however:

Use of libedit on Linux

Python 3.10+ Linux distributions link against libedit (as opposed to readline) by default, as libedit is supported on 3.10+ outside of macOS.

Most Python builds on Linux will link against readline because readline is the dominant library on Linux.

Some functionality may behave subtly differently as a result of our choice to link libedit by default. (We choose libedit by default to avoid GPL licensing requirements of readline.)

Perhaps something similar to #1281 could be going on with libedit (or their usage of it) too?

@The-Compiler
Copy link
Member

The-Compiler commented Oct 13, 2024

Yep, it's that indeed. Built a vanilla CPython 3.13.0 with ./configure --with-readline=editline and it behaves in the exact same way. It would also explain why @jaraco saw this on macOS over in #12497 (comment) because from what I understand, CPython uses it on macOS by default.

Tried CPython v3.12.7 built the same way, that works. Will now bisect between the two...

@The-Compiler
Copy link
Member

Well, this was a bit annoying as two build fixes for readline were in between:

cpython$ git log --oneline v3.12.7..v3.13.0 -- Modules/readline.c
417dd3aca7b gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_SetGIL (GH-118645)
c2627d6eea9 gh-116322: Add Py_mod_gil module slot (#116882)
!! 8515fd79fef gh-117845: Detect libedit hook function signature in configure (#117870)
e7e1116a781 gh-105323: Remove `WITH_APPLE_EDITLINE` to use the same declaration for all editline (gh-112513)
c2982380f82 gh-112510: Add `readline.backend` for the backend readline uses (GH-112511)
2df26d83486 gh-112105: Make completer delims work on libedit (gh-112106)
154f099e611 gh-112292 : Catch import error conditions with readline hooks (gh-112313)
!! f4cb0d27cc0 gh-109191: Fix build with newer editline (gh-110239)
501939c9c14 gh-105323: Update readline module to detect apple editline variant (gh-108665)
e7de0c5901b gh-108765: Python.h no longer includes <sys/time.h> (#108775)
b32d4cad15f gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
d228825e088 gh-106320: Remove _PyOS_ReadlineTState API (#107034)
c9ce983ae1a gh-106320: Remove private pylifecycle.h functions (#106400)
a9305b5e80e gh-105240: add missing function prototypes (#105241)
9ab587b7146 gh-89886: Rely on HAVE_SYS_TIME_H (#105058)

but with a bit of git cherry-pick to get them to whatever commit I was testing, I was able to figure out the culprit:

commit 01e7405da400e8997f8964d06cc414045e144681 (HEAD)
Author: Tian Gao <[email protected]>
Date:   Mon Mar 25 08:18:09 2024 -0700

    gh-112948: Make pdb completion similar to repl completion (#112950)

 Lib/pdb.py                                                              | 43 ++++++++++++++++++++++++++++---------------
 Lib/test/test_pdb.py                                                    | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst |  1 +
 3 files changed, 80 insertions(+), 15 deletions(-)
 create mode 100644 Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst

that is:

which notably seems to use rlcompleter now in pdb.

Not sure where to go from here. If we want to open a CPython bug about this, we probably need a more minimal reproducer? But I have no clue whether the culprit is CPython or pytest here.

@bluthej
Copy link
Contributor Author

bluthej commented Oct 14, 2024

@The-Compiler wow that's some impressive investigative skills right there! I certainly wouldn't have been able to figure this all out so fast...

All that took about a second, which is indeed pretty damn crazy 🤯

Yeah it's pretty sweet ^^

@skytwosea
Copy link

... disabling pytest's output capturing with -s seems to fix things too!

@The-Compiler my hero. My sanity owes you a debt of gratitude!

Chiming in to confirm that I've been experiencing this bug as well, specifically in a uv-generated venv (but haven't checked using any other package managers):

  • kernel 6.8
  • python 3.13
  • uv 0.4.25

@patryk4815
Copy link

@The-Compiler similar bug happen in gdb: (because of python/cpython#112948 )

$ gdb
(gdb) st<tab>
start     starti    step      stepi     stepping  stop      strace
(gdb) pi
>>> [o for o in sys.modules.keys() if 'readline' in o]
[]
>>> import pdb
>>> [o for o in sys.modules.keys() if 'readline' in o]
['readline']
>>> quit

(gdb) st<tab><tab>

After import pdb in py3.13, autocomplete is broken

@The-Compiler
Copy link
Member

Someone (not me) will need to report this to gdb, and ideally cook up a reproducer and report it to CPython and/or libedit. I won't be spending any more time on this, as it doesn't affect me and we have a workaround for pytest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: capture related to the capture builtin plugin plugin: debugging related to the debugging builtin plugin type: bug problem that needs to be addressed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants