Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't special-case class instances in binary expression inference #15161

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
+555 −60
Open

Don't special-case class instances in binary expression inference #15161

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
c8c99f6
Regroup literal cases together
dcreager Dec 27, 2024
58ada96
Add Todo and Never cases
dcreager Dec 27, 2024
029932d
General Instance fallback
dcreager Dec 27, 2024
3b1622f
Add a slew of tests
dcreager Dec 27, 2024
33409ba
Linter
dcreager Dec 27, 2024
9fb3013
Add mis-typed test
dcreager Dec 27, 2024
f4a65c2
More recursive class definitions cause panics
dcreager Dec 27, 2024
a5f0dad
Add test case for class union
dcreager Dec 27, 2024
5239fa9
Add test case for class union < 3.10
dcreager Dec 27, 2024
a52486f
Special-case ClassLiteral | ClassLiteral
dcreager Dec 27, 2024
dffd23d
Merge branch 'main' into dcreager/infer-binary
dcreager Dec 30, 2024
fe73357
Revert "Special-case ClassLiteral | ClassLiteral"
dcreager Dec 30, 2024
a70fc31
Move subclass tests to be closer to similar class tests
dcreager Dec 30, 2024
0acdbe4
Add reflected dunder test cases
dcreager Dec 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions crates/red_knot_python_semantic/resources/mdtest/binary/booleans.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,50 @@ reveal_type(a | b) # revealed: Literal[True]
reveal_type(b | a) # revealed: Literal[True]
reveal_type(b | b) # revealed: Literal[False]
```

## Arithmetic with a variable

```py
a = True
b = False

def lhs_is_int(x: int):
reveal_type(x + a) # revealed: int
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x % a) # revealed: int

def rhs_is_int(x: int):
reveal_type(a + x) # revealed: int
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a % x) # revealed: int

def lhs_is_bool(x: bool):
reveal_type(x + a) # revealed: int
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x % a) # revealed: int

def rhs_is_bool(x: bool):
reveal_type(a + x) # revealed: int
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a % x) # revealed: int

def both_are_bool(x: bool, y: bool):
reveal_type(x + y) # revealed: int
reveal_type(x - y) # revealed: int
reveal_type(x * y) # revealed: int
reveal_type(x // y) # revealed: int
reveal_type(x / y) # revealed: float
reveal_type(x % y) # revealed: int
```
27 changes: 27 additions & 0 deletions crates/red_knot_python_semantic/resources/mdtest/binary/classes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Binary operations on classes

## Union of two classes

Unioning two classes via the `|` operator is only available in Python 3.10 and later.

```toml
[environment]
python-version = "3.10"
```

```py
class A: ...
class B: ...

reveal_type(A | B) # revealed: Literal[A, B]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think Literal[A, B] is the right type to assign to the value expression A | B.

In a type expression, A | B spells the type Type::Union[Type::Instance(A), Type::Instance(B)] -- that is, the type consisting of all instances of A (or any subclass) and all instances of B (or any subclass).

The value expression A if flag else B would correctly be assigned the type Literal[A, B], since its runtime value must either be the class object A or the class object B.

But the runtime value of the value expression A | B is not either of those, so its type can't be Literal[A, B]. Its runtime value is an instance of types.UnionType, which can't be treated as if it were a class object at all. So the type we should assign to it is Type::Instance(types.UnionType).

Suggested change
reveal_type(A | B) # revealed: Literal[A, B]
reveal_type(A | B) # revealed: UnionType

If we did any special-casing here, the correct special casing would be to create a custom type representation for types.UnionType so we can record the fact that its __args__ attribute (in this case) has the type tuple[Literal[A], Literal[B]]. But I don't think this is worth doing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted the special case, this is now returning UnionType again as before

```

## Union of two classes (prior to 3.10)

```py
class A: ...
class B: ...

# error: "Operator `|` is unsupported between objects of type `Literal[A]` and `Literal[B]`"
reveal_type(A | B) # revealed: Unknown
```
217 changes: 217 additions & 0 deletions crates/red_knot_python_semantic/resources/mdtest/binary/custom.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
# Custom binary operations

## Class instances

```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"

def __sub__(self, other) -> Literal["-"]:
return "-"

def __mul__(self, other) -> Literal["*"]:
return "*"

def __matmul__(self, other) -> Literal["@"]:
return "@"

def __truediv__(self, other) -> Literal["/"]:
return "/"

def __mod__(self, other) -> Literal["%"]:
return "%"

def __pow__(self, other) -> Literal["**"]:
return "**"

def __lshift__(self, other) -> Literal["<<"]:
return "<<"

def __rshift__(self, other) -> Literal[">>"]:
return ">>"

def __or__(self, other) -> Literal["|"]:
return "|"

def __xor__(self, other) -> Literal["^"]:
return "^"

def __and__(self, other) -> Literal["&"]:
return "&"

def __floordiv__(self, other) -> Literal["//"]:
return "//"

class Sub(Yes): ...
class No: ...

# Yes implements all of the dunder methods.
reveal_type(Yes() + Yes()) # revealed: Literal["+"]
reveal_type(Yes() - Yes()) # revealed: Literal["-"]
reveal_type(Yes() * Yes()) # revealed: Literal["*"]
reveal_type(Yes() @ Yes()) # revealed: Literal["@"]
reveal_type(Yes() / Yes()) # revealed: Literal["/"]
reveal_type(Yes() % Yes()) # revealed: Literal["%"]
reveal_type(Yes() ** Yes()) # revealed: Literal["**"]
reveal_type(Yes() << Yes()) # revealed: Literal["<<"]
reveal_type(Yes() >> Yes()) # revealed: Literal[">>"]
reveal_type(Yes() | Yes()) # revealed: Literal["|"]
reveal_type(Yes() ^ Yes()) # revealed: Literal["^"]
reveal_type(Yes() & Yes()) # revealed: Literal["&"]
reveal_type(Yes() // Yes()) # revealed: Literal["//"]

# Sub inherits Yes's implementation of the dunder methods.
reveal_type(Sub() + Sub()) # revealed: Literal["+"]
reveal_type(Sub() - Sub()) # revealed: Literal["-"]
reveal_type(Sub() * Sub()) # revealed: Literal["*"]
reveal_type(Sub() @ Sub()) # revealed: Literal["@"]
reveal_type(Sub() / Sub()) # revealed: Literal["/"]
reveal_type(Sub() % Sub()) # revealed: Literal["%"]
reveal_type(Sub() ** Sub()) # revealed: Literal["**"]
reveal_type(Sub() << Sub()) # revealed: Literal["<<"]
reveal_type(Sub() >> Sub()) # revealed: Literal[">>"]
reveal_type(Sub() | Sub()) # revealed: Literal["|"]
reveal_type(Sub() ^ Sub()) # revealed: Literal["^"]
reveal_type(Sub() & Sub()) # revealed: Literal["&"]
reveal_type(Sub() // Sub()) # revealed: Literal["//"]

# No does not implement any of the dunder methods.
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `No` and `No`"
reveal_type(No() + No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `No` and `No`"
reveal_type(No() - No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `No` and `No`"
reveal_type(No() * No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `No` and `No`"
reveal_type(No() @ No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `No` and `No`"
reveal_type(No() / No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `No` and `No`"
reveal_type(No() % No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `No` and `No`"
reveal_type(No() ** No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `No` and `No`"
reveal_type(No() << No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `No` and `No`"
reveal_type(No() >> No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `No` and `No`"
reveal_type(No() | No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `No` and `No`"
reveal_type(No() ^ No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `No` and `No`"
reveal_type(No() & No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `No` and `No`"
reveal_type(No() // No()) # revealed: Unknown

# Yes does not implement any of the reflected dunder methods.
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() + Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() - Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() * Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() @ Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() / Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() % Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() ** Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() << Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() >> Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() | Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() ^ Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() & Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() // Yes()) # revealed: Unknown
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like we are missing the Yes() <op> No() case here; is there a reason you didn't include that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure if that would be needed, since No is not a subclass of Yes. So it doesn't matter whether it implements __radd__, we're always going to use Yes.__add__, and so that would give the same result as the Yes() <op> Yes() case.

So I think what I want is another stanza, where Sub and No both implemented the reflected methods, to show that we correctly use the Sub reflections since it's a subclass, but don't pick up the No reflections since it's not.


## Classes

Dunder methods defined in a class are available to instances of that class, but not to the class
itself. (For these operators to work on the class itself, they would have to be defined on the
class's type, i.e. `type`.)

```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"

class Sub(Yes): ...
class No: ...

# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[Yes]` and `Literal[Yes]`"
reveal_type(Yes + Yes) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[Sub]` and `Literal[Sub]`"
reveal_type(Sub + Sub) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[No]` and `Literal[No]`"
reveal_type(No + No) # revealed: Unknown
```

## Function literals

```py
def f():
pass

# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f + f) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f - f) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f * f) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f @ f) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f / f) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f % f) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f**f) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f << f) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f >> f) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f | f) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f ^ f) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f & f) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f // f) # revealed: Unknown
```

## Subclass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels very closely related to the ## Classes section above -- maybe group them together instead of having the intervening function literals section?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"

class Sub(Yes): ...
class No: ...

def yes() -> type[Yes]:
return Yes

def sub() -> type[Sub]:
return Sub

def no() -> type[No]:
return No

# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[Yes]` and `type[Yes]`"
reveal_type(yes() + yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[Sub]` and `type[Sub]`"
reveal_type(sub() + sub()) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[No]` and `type[No]`"
reveal_type(no() + no()) # revealed: Unknown
```
33 changes: 33 additions & 0 deletions crates/red_knot_python_semantic/resources/mdtest/binary/integers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,34 @@ reveal_type(3 * -1) # revealed: Literal[-3]
reveal_type(-3 // 3) # revealed: Literal[-1]
reveal_type(-3 / 3) # revealed: float
reveal_type(5 % 3) # revealed: Literal[2]

# TODO: We don't currently verify that the actual parameter to int.__add__ matches the declared
# formal parameter type.
reveal_type(2 + "f") # revealed: int
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test produced a @Todo type before, since our literal coercions weren't complete


def lhs(x: int):
reveal_type(x + 1) # revealed: int
reveal_type(x - 4) # revealed: int
reveal_type(x * -1) # revealed: int
reveal_type(x // 3) # revealed: int
reveal_type(x / 3) # revealed: float
reveal_type(x % 3) # revealed: int

def rhs(x: int):
reveal_type(2 + x) # revealed: int
reveal_type(3 - x) # revealed: int
reveal_type(3 * x) # revealed: int
reveal_type(-3 // x) # revealed: int
reveal_type(-3 / x) # revealed: float
reveal_type(5 % x) # revealed: int

def both(x: int):
reveal_type(x + x) # revealed: int
reveal_type(x - x) # revealed: int
reveal_type(x * x) # revealed: int
reveal_type(x // x) # revealed: int
reveal_type(x / x) # revealed: float
reveal_type(x % x) # revealed: int
```

## Power
Expand All @@ -21,6 +49,11 @@ largest_u32 = 4_294_967_295
reveal_type(2**2) # revealed: Literal[4]
reveal_type(1 ** (largest_u32 + 1)) # revealed: int
reveal_type(2**largest_u32) # revealed: int

def variable(x: int):
reveal_type(x**2) # revealed: @Todo(return type)
reveal_type(2**x) # revealed: @Todo(return type)
reveal_type(x**x) # revealed: @Todo(return type)
```

## Division by Zero
Expand Down
2 changes: 1 addition & 1 deletion crates/red_knot_python_semantic/resources/mdtest/slots.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ class A:
__slots__ = ()
__slots__ += ("a", "b")

reveal_type(A.__slots__) # revealed: @Todo(Support for more binary expressions)
reveal_type(A.__slots__) # revealed: @Todo(return type)

class B:
__slots__ = ("c", "d")
Expand Down
4 changes: 4 additions & 0 deletions crates/red_knot_python_semantic/resources/mdtest/unary/custom.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ reveal_type(~No()) # revealed: Unknown

## Classes

Dunder methods defined in a class are available to instances of that class, but not to the class
itself. (For these operators to work on the class itself, they would have to be defined on the
class's type, i.e. `type`.)

```py
class Yes:
def __pos__(self) -> bool:
Expand Down
Loading
Loading