Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that macOS binaries are cross platform, even if wheels aren't #1475

Merged
merged 12 commits into from
Oct 7, 2023

Conversation

freakboy3742
Copy link
Member

@freakboy3742 freakboy3742 commented Oct 3, 2023

Fixes #1217.

If a third-party package provides a universal2 wheel, any BeeWare macOS app using that binary will also be universal.

However, if a third party package is single platform (e.g., numpy and Pillow both publish single-platform wheels), then the resulting binary will only run on the platform where it was built.

This modifies the macOS build process to:

  1. Install all packages for the build architecture into a temporary app_packages folder
  2. Inspect all the installed wheels, looking for any wheel that is marked as non-pure and non-universal
  3. Perform a second install of just the non-pure, non-universal packages, into a second temporary app_packages folder. This a similar site-packages hack as the iOS backend to trick pip into installing binaries for a different platform.
  4. Merge the contents of the two app_packages folders into the final "production" app-packages folder.

Since the support package now contains multiple pieces that aren't needed at runtime (an XCFramework, plus the platform site-packages), the support package for the macOS app backend is also installed into a temporary location, and the standard library is copied into the "production" location.

Also modifies the handling of verbosity. The macOS backend has 2 steps where it prints a lot of output - purging content (where all the __pycache__ folders are removed, amongst other files), and signing (where each file being signed is listed). This output is useful for debugging, but can easily obscure warnings like potential signing problems. It also obscured warnings about discrepancies between platform files. Therefore, this PR modifies output so that:

  • At default verbosity, a progress bar is shown as files are deleted/signed/merged, but the individual files are not displayed. Any warnings will be visible.
  • At verbosity 1 (-v), the individual files being deleted/signed/merged will also be printed.
  • At verbosity 2 (-vv), the deep console debug (showing the actual commands being executed, shell environment, and the raw console response) will also be printed

PR Checklist:

  • All new features have been tested
  • All new features have been documented
  • I have read the CONTRIBUTING.md file
  • I will abide by the code of conduct

@freakboy3742 freakboy3742 marked this pull request as ready for review October 5, 2023 06:33
@freakboy3742 freakboy3742 requested a review from mhsmith October 5, 2023 07:00
@@ -0,0 +1 @@
macOS apps are now guaranteed to be universal binaries when dependencies only provide single-architecture binary wheels.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarify that the intention is for ALL apps to be universal binaries.

Suggested change
macOS apps are now guaranteed to be universal binaries when dependencies only provide single-architecture binary wheels.
macOS apps are now guaranteed to be universal binaries, even when dependencies only provide single-architecture binary wheels.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the path forward when packages stop releasing x86_64 and universal2 wheels, and focus only on arm64? I guess we'll need to re-introduce an option to build single-platform apps again.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that - it's potentially even an issue right now, if you have a particularly progressive app developer who decides that they either (a) don't care about x86 users, or (b) are impacted by an upstream dependency that doesn't provide an x86 package. Both are entirely possible outcomes - given that there are projects already deprecated Python 3.8 support, I wouldn't think we're too far off this being a practical concern.

My inclination is to land this patch without support for single platform builds, and tackle that as a subsequent PR - I'm guessing a mac-specific app option universal2_build which is True by default (for now), but can be configured to False, which reverts the install process to "old style" single platform installs; at some point in the future, we change the default, then deprecate the option.

However, if you think this is a feature we can't ship without including, I'll work up the modifications.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we can postpone this for now, but it would be worth creating an issue to track it.

src/briefcase/platforms/macOS/utils.py Outdated Show resolved Hide resolved
@freakboy3742 freakboy3742 requested a review from mhsmith October 7, 2023 03:53
@mhsmith mhsmith merged commit 8f79c2d into beeware:main Oct 7, 2023
@freakboy3742 freakboy3742 deleted the macos-universal-binary branch October 9, 2023 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

macOS apps aren't portable across x86_64/M1 unless all binary wheels are universal2
2 participants