autodoc incorrectly displays class methods for builtin types

[skirpichev]

Describe the bug

My package has two types. One implemented as pure-Python class and other with CPython C-API. Both have class methods with correct signatures:

>>> import inspect
>>> from gmp import mpq, mpz
>>> inspect.isbuiltin(mpq.from_decimal)
False
>>> inspect.isbuiltin(mpz.from_bytes)
True
>>> inspect.signature(mpq.from_decimal)
<Signature (dec)>
>>> inspect.signature(mpz.from_bytes)
<Signature (bytes, byteorder='big', *, signed=False)>
>>> help(mpq.from_decimal)
Help on method from_decimal in module gmp_fractions:

from_decimal(dec) class method of gmp.mpq
    Converts a finite Decimal instance to a rational number, exactly.

>>> help(mpz.from_bytes)
Help on built-in function from_bytes:

from_bytes(bytes, byteorder='big', *, signed=False) class method of gmp.mpz
    Return the integer represented by the given array of bytes.

    bytes
      Holds the array of bytes to convert.  The argument must either
[...]

As you can see, both correctly displayed by the builtin help(). However, sphinx shows second method like that:

The method miss "classmethod" mark and also the first argument (bytes) - was removed.

How to Reproduce

Sphinx configuration: https://github.com/diofant/python-gmp/tree/e281e2a75a435c1ca52efdf1777ea420ef02ae22/docs

Environment Information

Platform:              linux; (Linux-6.1.0-28-amd64-x86_64-with-glibc2.36)
Python version:        3.13.1 (tags/v3.13.1:0671451779, Dec  4 2024, 07:55:26) [GCC 12.2.0])
Python implementation: CPython
Sphinx version:        8.1.3
Docutils version:      0.21.2
Jinja2 version:        3.1.4
Pygments version:      2.18.0

Sphinx extensions

No response

Additional context

No response

[electric-coder]

This isn't a minimal reproducible example because the report doesn't have the bare minimum code and thus isn't self-contained instead linking to an external repo. But it's a known issue that Sphinx can't extract everything from a CPython C-API because the C implementation doesn't allow for complete introspection. Your can search the repo and Stack Overflow but the usual workaround is to manually declare the needed doc elements in the .rst.

[skirpichev]

This isn't a minimal reproducible example

Well, I can create somewhat more minimal one. But it will require at least one builtin type and at least one method on it. A lot of code, anyway.

it's a known issue that Sphinx can't extract everything from a CPython C-API

As it shown above - inspect module can. And builtin help() function too (which uses inspect module capabilities).

the C implementation doesn't allow for complete introspection

There is no documented way to provide introspection capabilities for C extensions. However, it's possible and in my case the gmp module does this properly.

The sphinx should utilize that and work correctly just as help() does. There is nothing wrong: help() uses public interfaces from the inspect module.

the usual workaround is to manually declare the needed doc elements in the .rst.

I.e. not use autodoc at all.

In principle, another such "workaround" is using PEP 7-style docstrings (e.g. the gmpy2 package does that). But this will break introspection support.

[picnixz]

I think the signature object should contain "cls" as first parameter, otherwise we don't detect it as a class method. I don't know if help() doesn't just render the signature as is.

I suspect we just consider bytes to be "self" and thus we don't know that ir was actually a class method. I'll have a look at it in a few days or so, or just ping me if I forgot.

[skirpichev]

I think the signature object should contain "cls" as first parameter

No, it shouldn't. This is only a naming convention for pure-Python code.

I don't know if help() doesn't just render the signature as is.

No, it's correctly determine the method as a class method for a builtin. Then the first argument - properly ignored (not the second one, i.e. bytes).

I suspect we just consider bytes to be "self" and thus we don't know that ir was actually a class method.

Well, the pydoc does something like that with all available introspection:

>>> inspect.signature(mpz.from_bytes)  # how signature looks
<Signature (bytes, byteorder='big', *, signed=False)>
>>> inspect.isbuiltin(mpz.from_bytes)
True
>>> mpz.from_bytes.__self__
<class 'gmp.mpz'>
>>> inspect.isclass(mpz.from_bytes.__self__)  # aha, probably it's a class method
True
>>> self = mpz.from_bytes.__self__
>>> name = mpz.from_bytes.__name__
>>> self.__qualname__ + '.' + name == mpz.from_bytes.__qualname__  # second check
True

I'll have a look at it in a few days or so, or just ping me if I forgot.

Well, I'll have to prepare some workaround for the next release, so it's not urgent. I'll provide a smaller reproducer for you and, perhaps, will dig into this if I've found a time.

[skirpichev]

Ok, I did a smaller reproducer. Just unpack and do:

pip install -e .[docs] -q
sphinx-build --color -W --keep-going -b html docs build/sphinx/html

Here is how it's docs are rendered.

In fact, both types have class methods:

>>> inspect.signature(omg.Spam.runit)
<Signature (spam, eggs='big', *, bacon=True)>
>>> inspect.signature(omg.MoreSpam.runit)
<Signature (spam, /, eggs='big', bacon=True)>
>>> help(omg.Spam.runit)
Help on built-in function runit:

runit(spam, eggs='big', *, bacon=True) class method of omg.Spam
    Spam!  Spam!

>>> help(omg.MoreSpam.runit)
Help on method runit in module omg_spam:

runit(spam, /, eggs='big', bacon=True) class method of omg.MoreSpam
    I'm a class method too!

python-omg.tar.gz

[picnixz]

I'll be without network access for the next two days so I'll try fixing this in the meantime.

[picnixz]

FTR, I have fixed the issue locally (and found a separate issue at the same time that is not related). I'll push something tomorrow (the issue was that a class method implemented in C is not an instance of classmethod but a class method descriptor... but for static methods that's not the case...)