[0.4.x] Support Pyodide 0.27

[bollwyvl]

References

• addresses micropip 0.7 will break piplite (Pyodide 0.27) #147
• alternative to Update to Pyodide 0.27, and fix compatibility with micropip 0.8 #153 (with manual cherry-picks)
  • targets 0.4.x instead of main (just

Changes

• bump pyodide in all the various places
• handle API differences
• from Update to Pyodide 0.27, and fix compatibility with micropip 0.8 #153
  • invert order of piplite.piplite._query_package's index_urls and fetch_kwargs
  • apply spelling/branding updates
  • add new accepted wheel pyodide_*_wasm32.whl
    • kept WASM_WHL constant to avoid needless breakage
  • update reference piplite CLI spec with support annotations
  • ???

[github-actions]

👈 Try it on ReadTheDocs

[agriyakhetarpal]

Thanks!

add new accepted wheel pyodide_*_wasm32.whl

kept WASM_WHL constant to avoid needless breakage

xref to the issue about this in micropip here: pyodide/micropip#158. Should I put together a PR to resolve this in micropip first before this new ABI tag lands here (and subsequently, have it in a new micropip 0.8.1 release via a Pyodide 0.27.1 release)?

Either way, I think this PR looks good for Pyodide 0.27.0 and micropip 0.8.0 in specific, and the above can be taken into account with another [0.4.y] update after this one.

[agriyakhetarpal]

Actually, never mind, that was fixed with pyodide/micropip#159 which made it to the 0.7.2 release already, it is just that the issue was not closed (I'll mark it as closed now). All good, sorry for the confusion!

[bollwyvl]

never mind, that was fixed

Yeah, that's why I don't bother following micropip too tightly: the only version i ever look at is what gets shipped to CDN.

It's... non-trivial to replace it from a jupyterlite perspective, as at least today, we must bootstrap a couple patched wheels, and pyodide-lock isn't fit for this purpose, as it won't generate a solve without a full runtime which has lead me to rather... absurd lengths to get to something user-serviceable for the 80% case of "my noarch wheel and its deps."

I don't think we can do anything about the @types/emscripten vs pyodide/types.d.ts at this point, it was nice while it lasted, and every as any makes the baby Isaac Newton (maybe Leibniz) cry.

I am trying some more notebooks from jupyterlite's docs on RTD, and may do some more abusive checks with those wheels, but otherwise, this is likely ready for more review and testing.

[agriyakhetarpal]

Thanks for the additional context! We are planning to unvendor recipes with Pyodide 0.28 (it is our near term plan). Would it be worthwhile to unvendor micropip and treat it as a special package, given how extensively it is used here, so that handling Pyodide releases is easier?

Yes, pyodide-lock also has a few issues, though I have not been involved with it much. But, I'm not sure if https://pypi.org/project/jupyterlite-pyodide-lock/ would also fully work (it's awesome if it does), because the last time I tried using a custom wheel with JupyterLite, I ran into something similar to jupyterlite/jupyterlite#1502 (it has been a while, though).

[bollwyvl]

Yeah, jlpl works, but only through a rather ridiculous contraption of browser automation, and a custom web server/proxy.

Until such a time as we can do an accurate micropip.freeze solve in a "regular" CPython, it's the best I've got.

unvendor recipes

Yeah, no skin in the game there. We'll continue to minimize user-space breakage, and support upstream features as best we can here.

handling Pyodide releases

I wouldn't worry too much about us: that's why use all those types, have tests, and generally try to get our downstream releases out quickly, but only after an actual upstream release. There are too many moving upstream parts for us to do much else.

[jtpio]

Thanks!

[jtpio]

@meeseeksdev please backport to main

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout main
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 c608e3b084bd15a78dc3160542d83de6583a489f

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #154: [0.4.x] Support Pyodide 0.27'

4. Push to a named branch:

git push YOURFORK main:auto-backport-of-pr-154-on-main

5. Create a PR against branch main, I would have named this PR:

"Backport PR #154 on branch main ([0.4.x] Support Pyodide 0.27)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[jtpio]

Looks like we still need a manual backport to main.

In the meantime, starting a new 0.4.7 release with this.

[bollwyvl]

Cool, thanks for the release. Also merged the conda-forge PR, will see about getting the beta out a tick.