PEP 764: Inlined typed dictionaries

[Viicos]

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

• PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
• Suggested sections included (unless not applicable)
  • Motivation
  • Rationale
  • Specification
  • Backwards Compatibility
  • Security Implications
  • How to Teach This
  • Reference Implementation
  • Rejected Ideas
  • Open Issues
• Python-Version set to valid (pre-beta) future Python version, if relevant
• Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
• Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

---

📚 Documentation preview 📚: https://pep-previews--4082.org.readthedocs.build/pep-0764/

[AA-Turner]

Thank you for opening this PR. PEPs require a core developer or PEP editor who is willing to sponsor it, which your PR currently lacks. I would advise you to start a thread on the https://discuss.python.org/ forum to see if there is support for this proposal and if someone is willing to sponsor it.

A

[erictraut]

Thanks for writing this. From my perspective, it's looking pretty good.

I added some comments and questions for your consideration.

[erictraut]

I think that Required, NotRequired and ReadOnly should be permitted here. Absent any mention of this, the typing spec could be interpreted as disallowing this. It's better to make it clear.

I presume that all inlined TypedDicts are implicitly "total" (i.e. all items are assumed to be Required unless explicitly NotRequired). This makes sense because it's consistent with precedent, but it would be good to spell it out in the spec.

I think it's also worth considering making all inlined TypedDicts implicitly "closed" (as defined in draft PEP 728). This would deviate from precedent, but I think one could make a good argument for why inlined TypedDicts should be closed.

[Viicos]

I think that Required, NotRequired and ReadOnly should be permitted here. Absent any mention of this, the typing spec could be interpreted as disallowing this. It's better to make it clear.

I presume that all inlined TypedDicts are implicitly "total" (i.e. all items are assumed to be Required unless explicitly NotRequired). This makes sense because it's consistent with precedent, but it would be good to spell it out in the spec.

Updated.

I think it's also worth considering making all inlined TypedDicts implicitly "closed" (as defined in draft PEP 728). This would deviate from precedent, but I think one could make a good argument for why inlined TypedDicts should be closed.

Sounds reasonable. I have to say I'm still a bit confused with PEP 728 and closed=True. If we disallow subclassing inlined typed dictionaries, it means that every inlined typed dictionary is implicitly @final. What's the purpose of having closed=True then? I guess I should ask my question in the PEP 728 discussion thread instead.

[erictraut]

Am I correct in assuming that there is no way for an inlined TypedDict to "extend" another (named) TypedDict? One could imagine supporting this using dictionary expansion syntax, but that adds complexity that may not be justifiable. It would also impose additional runtime requirements on TypedDict classes (they'd need to be iterable).

[JelleZijlstra]

A good way to support inheritance could be to allow more than one argument to the subscript: TypedDict[Base, {"a": int}].

[erictraut]

Did you consider reusing typing.Dict here? I can see some potential advantages. It's less verbose, doesn't have the baggage of builtins.dict, and the fact that this is a "typed" dict is already implicit given that it is imported from typing and involves type annotations.

[tomasr8]

Showing my TypeScript bias here 😅 but I'm wondering how sacrilegious it would be to avoid the subscription all together and simply use the dictionary itself?

def get_movie() -> {'name': str, 'year': int}:
    return {
        'name': 'Blade Runner',
        'year': 1982,
    }

[JelleZijlstra]

This causes problems with the | operator at least. I've been thinking of writing up something like "Why can't we ...?" discussing why certain "obvious" ways to make typing more ergonomic may not work well in practice.

[tomasr8]

I think that such writeup would be very useful!

[Viicos]

I added an extra section in the rejected ideas regarding the plain dict syntax.

On using typing.Dict, I kind of like the idea for the reasons you mentioned @erictraut. Dict[{...}] implicitly spells out as something like:

Dict[{...}]
|      |------> ...with some typed fields
|--> a dict... 

However, I still think this is debatable for a couple reasons:

• typing.Dict is currently marked as deprecated in the Python documentation (although not scheduled for removal). It might be confusing to undeprecate it.
• The same subscripting syntax — with the only difference being the number of type arguments — means two different things. Can be confusing as well.

I'll add an entry to the open issues so that we can further discuss this option.

[erictraut]

No, this shouldn't be allowed by a static type checker. In this example, InlinedTD is a type alias.

If you change this to InlinedTD = TypedDict('InlinedTD', {'a': int}), then it should be allowed (as it is today) because InlinedTD is now a class, which can be used as a base class.

[Viicos]

Yes, I think we should also discuss whether an inlined typed dictonary is a proper class or an instance of some new internal typing._InlinedTypedDict class. As specified currently in this PEP, an inlined typed dictionary is a also a class, with an empty string for __name__ (which isn't ideal to be fair). Initially, I wanted inlined typed dictionaries to be an instance of some _InlinedTypedDict class, but this complicates the runtime behavior. For example, should we allow accessing the introspection attributes?

InlinedTD = TypedDict[{'a': int}]
InlinedTD.__required_keys__
InlinedTD.__total__
# etc

[erictraut]

@Viicos, as a member of the typing council, I'm willing to sponsor this PEP.

[JelleZijlstra]

Reopened as Eric offered to sponsor (thanks Eric!).

[Viicos]

I just realized I opened this PR on the original repository, but meant to open it on my fork as part of this discussion.

But I'm happy to keep going with this PEP. Regardless of the chosen implementation for partial, pick, etc, I can see how inlined typed dictionaries can be useful (Partial[SomeTD] needs to return a new typed dictionary, and this raises the question of which __name__ to choose — inlined typed dictionaries fulfill this requirement).

Thanks for the feedback. Should I open a separate discussion on the forum? Should it be in the PEPs/Typing category?

[hugovk]

I just realized I opened this PR on the original repository, but meant to open it on my fork as part of this discussion.

Whichever you prefer :) Probably depends on how much more work you think it needs before you're ready to submit this to be merged as a draft.

Should I open a separate discussion on the forum? Should it be in the PEPs/Typing category?

Wait until this draft is merged, then open a new discussion in the Typing category so people can discuss the merged and proposed draft (see https://peps.python.org/pep-0001/#discussing-a-pep).

Also please review PEP 1 and PEP 12 for the PEP process.

I've updated the OP to include the new PEP checklist, please work through it and check off things as appropriate. Thanks!

[erictraut]

I don't think we should support nested inlined TypedDict definitions as shown above. We should require that the type annotation for each item be a valid type expression, and a dictionary expression not enclosed by a TypedDict is an invalid type expression. The above example should therefore be changed to:

def get_movie() -> TypedDict[{"name": str, "year": int, "production": TypedDict[{"name": str, "location": str}]}]: ...

The grammar definition proposed above by Jelle already implies this, but the spec should make it clear that using nested dictionary expressions (like in the current code sample) is not allowed and should result in a type checker error.

I understand this might seem needlessly verbose, but if we allow dictionary expressions in type expressions outside of a TyepdDict type argument, it will create a cascade of exceptions and future composability issues. For example, we'd need to support raw dictionary expressions within Required, NotRequired, and ReadOnly.

[Viicos]

Indeed using a nested inlined typed dictionary is more consistent! Changed.

(Using typing.Dict instead of TypedDict might help reducing verbosity here).

[erictraut]

Pyright 1.1.387, which will be published in the next day, includes experimental support for the latest draft of this PEP.

[Viicos]

Amazing, updated to reference the future release.

[sydney-runkle]

@Viicos, awesome job here! So exciting to see this awesome contribution!!

[sydney-runkle]

I wonder, is there a world in which eventually you could do something like:

Movie = Annotated[TypedDict[{'name': str, 'year': int}], Total(False)]

Or something like that for class args?

[Viicos]

This would require:

• introducing a typing.Total class/function
• setting a precedent regarding the meaning of Annotated metadata, which is currently ignored by type checkers.

[sydney-runkle]

Hmm, I like the strict parametrization of TypedDict better, I feel like this would set an odd precedent and confuse some users.

[ZeroIntensity]

Apologies for the small (now undone) edit of one of the checkmarks, I was reading the PR and pushed the wrong button 😅

[KubaO]

From the perspective of this user of type annotations, I have only one reservation: isn't this encouraging an antipattern? In my mind, if you need to type-annotate a tree-like dictionary structure, then isn't it time to come up with a tree of classes or named tuples or any other data structure to make it explicit? I would think that a dict[A | B | C, D | E | F], ie. flat with a few possible types on each side is about as far as I'd encourage a dict use with type annotations.

With a complex tree-like dict, wouldn't it make more sense to add assert isinstance at the point of use to specify the structure in a scalable way? An annotation can be factored out with type statements, but I'm not convinced that the flexibility afforded by dictionaries warrants such rigidity. IMHO that's not what dicts-of-dicts are for...

[DetachHead]

From the perspective of this user of type annotations, I have only one reservation: isn't this encouraging an antipattern? In my mind, if you need to type-annotate a tree-like dictionary structure, then isn't it time to come up with a tree of classes or named tuples or any other data structure to make it explicit?

regardless of what your opinion is on whether or not TypedDicts should be defined in-line or not, you can define them with a type alias anyway and this syntax is more flexible than the class syntax because it allows for keys that aren't valid python names, which can be useful when dealing with data structures you don't control, eg:

type Foo = TypedDict[{"foo bar": int}]

(yes i know you can also do type Foo = TypedDict("Foo", {"foo bar": int}) but this is gross because you have to duplicate its name)

another point i'd like to make is that the current class "syntax" for defining TypedDicts is quite confusing imo. it makes the type look like a regular class when that's not what it represents at all. i think allowing a proper syntax to define them in-line makes the most sense.