[python] Add an iree.build package with API/tooling for program building. #18630

stellaraccident · 2024-09-28T21:06:59Z

This is the first step of providing a unified user-oriented build tool for IREE export and compilation. This initial version supports the CLI environment, network/fetch actions, and ONNX import+upgrade. A follow-on step will build out the compiler invocation rules, specifically focusing on getting option, device, and parameter handling and manifest/metadata propagated properly.

The intent is that downstream users of the IREE/Turbine tools can have one stop modules to efficiently export and compile arbitrarily complicated pipelines without reinventing the wheel or needing to do the advanced compiler juggling that comes with some of the more complicated (and performant) workflows.

As a by-product this will standardize a directory layout and manifest/metadata to accompany it for pipeline construction. The intent is that this will simplify serving tooling, which currently has to get a number of out of band parameters and files put together in the right way and matching how they were compiled.

.gitignore

compiler/bindings/python/iree/build/__main__.py

compiler/bindings/python/iree/build/executor.py

compiler/bindings/python/iree/build/onnx_actions.py

compiler/bindings/python/iree/build/executor.py

compiler/bindings/python/iree/build/__init__.py

stellaraccident · 2024-09-30T15:52:11Z

Thanks for early look. PR is still draft. Tests and compile actions are coming.

I really want it to be iree.build. better for user tools and can fix the ignore list

…ing. Signed-off-by: Stella Laurenzo <[email protected]>

Signed-off-by: Stella Laurenzo <[email protected]>

monorimet

LGTM. And thank you. It's really important that we have a unified methodology here. With the compiler action this will cover a lot of ground that otherwise usually turns into some clunky, bespoke process.

compiler/bindings/python/iree/build/onnx_actions.py

ScottTodd · 2024-10-31T16:38:48Z

compiler/bindings/python/iree/build/onnx_actions.py

+    def invoke(self):
+        import iree.compiler.tools.import_onnx.__main__ as m
+
+        args = m.parse_arguments(
+            [
+                str(self.input_file.get_fs_path()),
+                "-o",
+                str(self.output_file.get_fs_path()),
+            ]
+        )
+        m.main(args)


Does this support passing extra args/kwargs to the underlying script/tool? iree-import-onnx has a few other supported arguments:

iree/compiler/bindings/python/iree/compiler/tools/import_onnx/__main__.py

Lines 118 to 134 in df2b8a4

parser.add_argument(

"--no-verify",

action="store_true",

help="Disable verification prior to printing",

)

parser.add_argument(

"--data-prop",

default=True,

action=argparse.BooleanOptionalAction,

help="Toggle data propogation for onnx shape inference",

)

parser.add_argument(

"--data-dir",

help="Path to the base directory of the data."

" Defaults to the directory of the input file.",

type=Path,

)

It can be made to do so. Let's get the basics in and iterate. TBH - I only wrote this boilerplate ONNX support for testing the basic tool flow because ONNX is the only path that doesn't require carefully juggling the knives of an ML framework. So it was a pragmatic thing to get something going vs a fully formed thought on "this is the full solution for ONNX".

ScottTodd · 2024-10-31T16:50:51Z

compiler/bindings/python/CMakeLists.txt

Trying to test this locally on Windows since MLIR Python bindings tend to break in weird ways there. Bumped into an unrelated error first though: #18910 (comment)

That will also require followons I expect.

I just don't want our 6 hour nightly Windows build to fail if we merge this before testing on Windows. (#18813 will get faster builds running more frequently)

Well I'm still reconstructing my Windows build environment... but I think I got enough signal from a partial build to trust this.

ScottTodd · 2024-11-01T13:02:16Z

Test failed on macos bot: https://github.com/iree-org/iree/actions/runs/11627272067/job/32380320905#step:9:1650

AttributeError: module 'enum' has no attribute 'StrEnum'

Python 3.10

ScottTodd · 2024-11-01T14:38:13Z

Test also failed on Windows bot, same error: https://github.com/iree-org/iree/actions/runs/11627295212/job/32380385457#step:8:1703

The release build succeeded though, so that's less urgent to fix

ScottTodd · 2024-11-05T16:56:52Z

compiler/bindings/python/iree/build/executor.py

+_locals = threading.local()
+
+
+class FileNamespace(enum.StrEnum):


enum.StrEnum was added in Python 3.11, so this code is failing to run on Python 3.10, which we still support

https://docs.python.org/3/library/enum.html

Added in version 3.11: StrEnum, EnumCheck, ReprEnum, FlagBoundary, property, member, nonmember, global_enum, show_flag_values

Failing CI builds:

Windows: https://github.com/iree-org/iree/actions/runs/11681252881/job/32525923299#step:8:1702

macOS: https://github.com/iree-org/iree/actions/runs/11681217580/job/32525815632#step:9:1648

Actually, we only publish wheels for Python 3.11+ on Windows and macOS, so we could bump those workflows to use newer Python versions. I'll send a PR.

Pending CI builds (since these are nightly / on-demand and take hours): * Windows: https://github.com/iree-org/iree/actions/runs/11689195978 * macOS: https://github.com/iree-org/iree/actions/runs/11689197108 The new `iree.build` package depends on Python 3.11: #18630 (comment). We currently publish Python packages for 3.9+ on Linux but only 3.11+ on macOS and Windows: * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/build_tools/python_deploy/build_linux_packages.sh#L68 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L259-L266 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L247-L254 We can relax the 3.11 dependency in the package, but we might as well update these workflows to test with the version that we distribute packages for.

As discussed on #18630 (comment), now that `iree-import-onnx` supports upgrading the model version itself (added in #18880), we don't need to also maintain this logic in the `iree.build` package.

Pending CI builds (since these are nightly / on-demand and take hours): * Windows: https://github.com/iree-org/iree/actions/runs/11689195978 * macOS: https://github.com/iree-org/iree/actions/runs/11689197108 The new `iree.build` package depends on Python 3.11: iree-org#18630 (comment). We currently publish Python packages for 3.9+ on Linux but only 3.11+ on macOS and Windows: * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/build_tools/python_deploy/build_linux_packages.sh#L68 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L259-L266 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L247-L254 We can relax the 3.11 dependency in the package, but we might as well update these workflows to test with the version that we distribute packages for.

…rg#19210) As discussed on iree-org#18630 (comment), now that `iree-import-onnx` supports upgrading the model version itself (added in iree-org#18880), we don't need to also maintain this logic in the `iree.build` package.

…ing. (iree-org#18630) This is the first step of providing a unified user-oriented build tool for IREE export and compilation. This initial version supports the CLI environment, network/fetch actions, and ONNX import+upgrade. A follow-on step will build out the compiler invocation rules, specifically focusing on getting option, device, and parameter handling and manifest/metadata propagated properly. The intent is that downstream users of the IREE/Turbine tools can have one stop modules to efficiently export and compile arbitrarily complicated pipelines without reinventing the wheel or needing to do the advanced compiler juggling that comes with some of the more complicated (and performant) workflows. As a by-product this will standardize a directory layout and manifest/metadata to accompany it for pipeline construction. The intent is that this will simplify serving tooling, which currently has to get a number of out of band parameters and files put together in the right way and matching how they were compiled. --------- Signed-off-by: Stella Laurenzo <[email protected]> Signed-off-by: Giacomo Serafini <[email protected]>

Pending CI builds (since these are nightly / on-demand and take hours): * Windows: https://github.com/iree-org/iree/actions/runs/11689195978 * macOS: https://github.com/iree-org/iree/actions/runs/11689197108 The new `iree.build` package depends on Python 3.11: iree-org#18630 (comment). We currently publish Python packages for 3.9+ on Linux but only 3.11+ on macOS and Windows: * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/build_tools/python_deploy/build_linux_packages.sh#L68 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L259-L266 * https://github.com/iree-org/iree/blob/9650bfe441d5f8ce4a16395ee6b3ad43968caa2e/.github/workflows/build_package.yml#L247-L254 We can relax the 3.11 dependency in the package, but we might as well update these workflows to test with the version that we distribute packages for. Signed-off-by: Giacomo Serafini <[email protected]>

…rg#19210) As discussed on iree-org#18630 (comment), now that `iree-import-onnx` supports upgrading the model version itself (added in iree-org#18880), we don't need to also maintain this logic in the `iree.build` package. Signed-off-by: Giacomo Serafini <[email protected]>

ScottTodd reviewed Sep 30, 2024

View reviewed changes

[python] Add an iree.build package with API/tooling for program build…

8502cdd

…ing. Signed-off-by: Stella Laurenzo <[email protected]>

stellaraccident force-pushed the users/stellaraccident/iree_build_tool branch from 6ff9f64 to 8502cdd Compare October 31, 2024 01:45

stellaraccident marked this pull request as ready for review October 31, 2024 01:46

Add ignored file (?)

325dbbc

Signed-off-by: Stella Laurenzo <[email protected]>

stellaraccident force-pushed the users/stellaraccident/iree_build_tool branch from 73d9120 to 325dbbc Compare October 31, 2024 01:54

Fix f-string issue for old python.

9e84aee

Signed-off-by: Stella Laurenzo <[email protected]>

ScottTodd self-requested a review October 31, 2024 15:07

monorimet approved these changes Oct 31, 2024

View reviewed changes

ScottTodd added bindings/python Python wrapping IREE's C API compiler/tools IREE's compiler tooling (iree-compile, iree-opt, etc) labels Oct 31, 2024

ScottTodd reviewed Oct 31, 2024

View reviewed changes

ScottTodd approved these changes Oct 31, 2024

View reviewed changes

stellaraccident merged commit 0077358 into main Oct 31, 2024
39 checks passed

stellaraccident deleted the users/stellaraccident/iree_build_tool branch October 31, 2024 21:06

stellaraccident mentioned this pull request Oct 31, 2024

Find More General and Easier to use Alternative For Compiling Models for Shortfin LLM Server nod-ai/shark-ai#402

Open

ScottTodd mentioned this pull request Nov 4, 2024

Document iree.build package #19019

Open

4 tasks

ScottTodd reviewed Nov 5, 2024

View reviewed changes

ScottTodd mentioned this pull request Nov 5, 2024

Fix Windows+macOS CI by updating to Python 3.11. #19029

Merged

ScottTodd mentioned this pull request Nov 19, 2024

Use iree-import-onnx --opset-version N in ImportOnnxAction. #19210

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[python] Add an iree.build package with API/tooling for program building. #18630

[python] Add an iree.build package with API/tooling for program building. #18630

stellaraccident commented Sep 28, 2024 •

edited

Loading

stellaraccident commented Sep 30, 2024

monorimet left a comment •

edited

Loading

ScottTodd Oct 31, 2024

stellaraccident Oct 31, 2024

ScottTodd Oct 31, 2024

ScottTodd Oct 31, 2024

stellaraccident Oct 31, 2024

ScottTodd Oct 31, 2024

ScottTodd Oct 31, 2024

ScottTodd commented Nov 1, 2024

ScottTodd commented Nov 1, 2024

ScottTodd Nov 5, 2024

ScottTodd Nov 5, 2024 •

edited

Loading

	parser.add_argument(
	"--no-verify",
	action="store_true",
	help="Disable verification prior to printing",
	)
	parser.add_argument(
	"--data-prop",
	default=True,
	action=argparse.BooleanOptionalAction,
	help="Toggle data propogation for onnx shape inference",
	)
	parser.add_argument(
	"--data-dir",
	help="Path to the base directory of the data."
	" Defaults to the directory of the input file.",
	type=Path,
	)

		_locals = threading.local()


		class FileNamespace(enum.StrEnum):

[python] Add an iree.build package with API/tooling for program building. #18630

[python] Add an iree.build package with API/tooling for program building. #18630

Conversation

stellaraccident commented Sep 28, 2024 • edited Loading

stellaraccident commented Sep 30, 2024

monorimet left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ScottTodd commented Nov 1, 2024

ScottTodd commented Nov 1, 2024

Choose a reason for hiding this comment

ScottTodd Nov 5, 2024 • edited Loading

Choose a reason for hiding this comment

stellaraccident commented Sep 28, 2024 •

edited

Loading

monorimet left a comment •

edited

Loading

ScottTodd Nov 5, 2024 •

edited

Loading