Trigger presubmit ci workflows from ci.yml via workflow_call. (#1…

…9445) ## History These workflow jobs were originally part of `ci.yml` but they were moved out of part of #17957. ## Reasons to keep them separate * Run history for each workflow is independent, e.g. https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang.yml and https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml. * Performance metrics for each workflow are independent: ![image](https://github.com/user-attachments/assets/5832d491-541d-46e4-9626-422f989e8879) * Status badges are independent ## Reasons to merge them back * Less code duplication, particularly for `setup` and `summary` steps (see #19444) * Less noise in the checks view in pull requests (note all the `setup` jobs, and the PR above will add similarly dupliated `summary` jobs): ![image](https://github.com/user-attachments/assets/37628719-4d23-46f5-89ff-9da7f391695b) * Centralized logs for checks, not split between multiple pages like they are currently: Current | With this PR -- | -- ![image](https://github.com/user-attachments/assets/5adc117e-aa76-47a7-973f-6f1e7adcbc9c) | ![image](https://github.com/user-attachments/assets/2cb55430-63f5-4b1f-b64c-550b919fe51f) * This gives us a single `ci_summary` check in `ci.yml` to set as a required check instead of multiple split across workflows * Given enough runner capacity / budget, we could run all these jobs on presubmit/postsubmit and not just on nightly schedules
iree-org · Dec 19, 2024 · 5c4bc67 · 5c4bc67
1 parent 01f0900
commit 5c4bc67
Show file tree

Hide file tree

Showing 6 changed files with 65 additions and 114 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -46,6 +46,9 @@ jobs:
   setup:
     uses: ./.github/workflows/setup.yml
 
+  ##############################################################################
+  # Runtime builds
+
   runtime:
     needs: setup
     name: "runtime :: ${{ matrix.name }}"
@@ -196,16 +199,44 @@ jobs:
       - name: CMake - build
         run: cmake --build ${BUILD_DIR} -- -k 0
 
+  ##############################################################################
+  # Full project builds
+
+  linux_x64_bazel:
+    needs: setup
+    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_bazel')
+    uses: ./.github/workflows/ci_linux_x64_bazel.yml
+    secrets: inherit
+
+  linux_x64_clang:
+    needs: setup
+    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_clang')
+    uses: ./.github/workflows/ci_linux_x64_clang.yml
+    secrets: inherit
+
+  linux_x64_clang_asan:
+    needs: setup
+    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_clang_asan')
+    uses: ./.github/workflows/ci_linux_x64_clang_asan.yml
+    secrets: inherit
+
   ##############################################################################
 
   # Aggregate job status and alerting on failures.
   ci_summary:
     if: always()
     needs:
       - setup
+
+      # Runtime builds.
       - runtime
       - runtime_small
       - runtime_tracing
+
+      # Full project builds.
+      - linux_x64_bazel
+      - linux_x64_clang
+      - linux_x64_clang_asan
     uses: ./.github/workflows/workflow_summary.yml
     secrets: inherit
     with:

diff --git a/.github/workflows/ci_linux_x64_bazel.yml b/.github/workflows/ci_linux_x64_bazel.yml
@@ -7,27 +7,11 @@
 name: CI - Linux x64 bazel
 
 on:
+  workflow_call:
   workflow_dispatch:
-  pull_request:
-  push:
-    branches:
-      - main
-
-concurrency:
-  # A PR number if a pull request and otherwise the commit hash. This cancels
-  # queued and in-progress runs for the same PR (presubmit) or commit
-  # (postsubmit). The workflow name is prepended to avoid conflicts between
-  # different workflows.
-  group: ${{ github.workflow }}-${{ github.event.number || github.sha }}
-  cancel-in-progress: true
 
 jobs:
-  setup:
-    uses: ./.github/workflows/setup.yml
-
   linux_x64_bazel:
-    needs: setup
-    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_bazel')
     runs-on: azure-linux-scale
     container:
       image: ghcr.io/iree-org/cpubuilder_ubuntu_jammy@sha256:78a558b999b230f7e1da376639e14b44f095f30f1777d6a272ba48c0bbdd4ccb
@@ -54,10 +38,4 @@ jobs:
           /usr/local/bin/fetch_cuda_deps.sh ${IREE_CUDA_DEPS_DIR}
           ./build_tools/bazel/build_test_all.sh
 
-      - name: Post to Discord on Failure
-        uses: sarisia/actions-status-discord@ce8cc68e4e626000136b3c702d049a154243e490 # v1.14.7
-        if: failure() && github.ref_name == 'main' && github.repository_owner == 'iree-org'
-        with:
-          webhook: ${{ secrets.DISCORD_WEBHOOK }}
-          description: "The ${{ github.workflow }} workflow failed"
-          url: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}"
+      # Alerting on failure is the responsibility of the calling job.
diff --git a/.github/workflows/ci_linux_x64_clang.yml b/.github/workflows/ci_linux_x64_clang.yml
@@ -7,27 +7,11 @@
 name: CI - Linux x64 clang
 
 on:
+  workflow_call:
   workflow_dispatch:
-  pull_request:
-  push:
-    branches:
-      - main
-
-concurrency:
-  # A PR number if a pull request and otherwise the commit hash. This cancels
-  # queued and in-progress runs for the same PR (presubmit) or commit
-  # (postsubmit). The workflow name is prepended to avoid conflicts between
-  # different workflows.
-  group: ${{ github.workflow }}-${{ github.event.number || github.sha }}
-  cancel-in-progress: true
 
 jobs:
-  setup:
-    uses: ./.github/workflows/setup.yml
-
   linux_x64_clang:
-    needs: setup
-    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_clang')
     runs-on: azure-linux-scale
     container:
       image: ghcr.io/iree-org/cpubuilder_ubuntu_jammy@sha256:78a558b999b230f7e1da376639e14b44f095f30f1777d6a272ba48c0bbdd4ccb
@@ -71,10 +55,4 @@ jobs:
       - name: Test iree-dialects
         run: ./build_tools/cmake/test_iree_dialects.sh "${BUILD_DIR}"
 
-      - name: Post to Discord on Failure
-        uses: sarisia/actions-status-discord@ce8cc68e4e626000136b3c702d049a154243e490 # v1.14.7
-        if: failure() && github.ref_name == 'main' && github.repository_owner == 'iree-org'
-        with:
-          webhook: ${{ secrets.DISCORD_WEBHOOK }}
-          description: "The ${{ github.workflow }} workflow failed"
-          url: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}"
+      # Alerting on failure is the responsibility of the calling job.
diff --git a/.github/workflows/ci_linux_x64_clang_asan.yml b/.github/workflows/ci_linux_x64_clang_asan.yml
@@ -7,27 +7,11 @@
 name: CI - Linux x64 clang ASan
 
 on:
+  workflow_call:
   workflow_dispatch:
-  pull_request:
-  push:
-    branches:
-      - main
-
-concurrency:
-  # A PR number if a pull request and otherwise the commit hash. This cancels
-  # queued and in-progress runs for the same PR (presubmit) or commit
-  # (postsubmit). The workflow name is prepended to avoid conflicts between
-  # different workflows.
-  group: ${{ github.workflow }}-${{ github.event.number || github.sha }}
-  cancel-in-progress: true
 
 jobs:
-  setup:
-    uses: ./.github/workflows/setup.yml
-
   linux_x64_clang_asan:
-    needs: setup
-    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'linux_x64_clang_asan')
     runs-on: azure-linux-scale
     container: ghcr.io/iree-org/cpubuilder_ubuntu_jammy@sha256:78a558b999b230f7e1da376639e14b44f095f30f1777d6a272ba48c0bbdd4ccb
     defaults:
@@ -54,10 +38,4 @@ jobs:
           ./build_tools/cmake/build_and_test_asan.sh
           sccache --show-stats
 
-      - name: Post to Discord on Failure
-        uses: sarisia/actions-status-discord@ce8cc68e4e626000136b3c702d049a154243e490 # v1.14.7
-        if: failure() && github.ref_name == 'main' && github.repository_owner == 'iree-org'
-        with:
-          webhook: ${{ secrets.DISCORD_WEBHOOK }}
-          description: "The ${{ github.workflow }} workflow failed"
-          url: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}"
+      # Alerting on failure is the responsibility of the calling job.
diff --git a/README.md b/README.md
@@ -39,9 +39,11 @@ Python iree-base-runtime | [![PyPI version](https://badge.fury.io/py/iree-base-r
 [![CI](https://github.com/iree-org/iree/actions/workflows/ci.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci.yml?query=branch%3Amain+event%3Apush)
 [![PkgCI](https://github.com/iree-org/iree/actions/workflows/pkgci.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/pkgci.yml?query=branch%3Amain+event%3Apush)
 
-| Host platform | Build status |
+#### Nightly build status
+
+| Operating system | Build status |
 | -- | --: |
-Linux | [![CI - Linux x64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang.yml?query=branch%3Amain+event%3Apush)<br>[![CI - Linux arm64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml?query=branch%3Amain+event%3Aschedule)
+Linux | [![CI - Linux arm64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml?query=branch%3Amain+event%3Aschedule)
 macOS | [![CI - macOS x64 clang](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml?query=branch%3Amain+event%3Aschedule)
 Windows | [![CI - Windows x64 MSVC](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml?query=branch%3Amain+event%3Aschedule)
 

diff --git a/docs/website/docs/developers/general/github-actions.md b/docs/website/docs/developers/general/github-actions.md
@@ -69,7 +69,29 @@ graph
 
 ## :material-list-status: Workflow descriptions and status
 
-### Package tests
+### "CI" - Core builds and tests
+
+These workflows build the project from source then run unit tests.
+
+* To keep these workflows focused, they should not need any special hardware
+  (e.g. GPUs).
+* Some workflows in this category use sanitizers, debug builds, alternate
+  compilers, and other features that maintainers want automated coverage for.
+
+Workflow file | Build status | Event triggers
+-- | --: | --
+[`ci.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci.yml) | [![cI](https://github.com/iree-org/iree/actions/workflows/ci.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
+ | |
+[`ci_linux_arm64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_arm64_clang.yml) | [![CI - Linux arm64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+[`ci_macos_x64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_macos_x64_clang.yml) | [![CI - macOS x64 clang](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+[`ci_windows_x64_msvc.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_windows_x64_msvc.yml) | [![CI - Windows x64 MSVC](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+ | |
+[`ci_linux_x64_clang_byollvm.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_byollvm.yml) | [![CI - Linux x64 clang_byollvm](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_byollvm.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_byollvm.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+[`ci_linux_x64_clang_debug.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_debug.yml) | [![CI - Linux x64 clang debug](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_debug.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_debug.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+[`ci_linux_x64_clang_tsan.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_tsan.yml) | [![CI - Linux x64 clang TSan](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_tsan.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_tsan.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+[`ci_linux_x64_gcc.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_gcc.yml) | [![CI - Linux x64 gcc](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_gcc.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_gcc.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
+
+### "PkgCI" - Package builds and tests
 
 These workflows build packages from source then run test suites using them.
 
@@ -102,50 +124,12 @@ Workflow file | Build status | Event triggers
 Package tests | |
 [`pkgci.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/pkgci.yml) | [![PkgCI](https://github.com/iree-org/iree/actions/workflows/pkgci.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/pkgci.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
 
-### Platform builds
-
-These workflows build the full project from source using standard options then
-run basic tests.
-
-* To keep these workflows focused, they should not need any special hardware
-  (e.g. GPUs).
-
-Workflow file | Build status | Event triggers
--- | --: | --
-[`ci_linux_x64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang.yml) | [![CI - Linux x64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
-[`ci_linux_arm64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_arm64_clang.yml) | [![CI - Linux arm64 clang](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_arm64_clang.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_macos_x64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_macos_x64_clang.yml) | [![CI - macOS x64 clang](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_macos_x64_clang.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_windows_x64_msvc.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_windows_x64_msvc.yml) | [![CI - Windows x64 MSVC](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_windows_x64_msvc.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-
-<!-- TODO(scotttodd): macos_arm64_clang -->
-
-### Other build configurations
-
-These workflows build the full project from source using optional settings
-then run basic tests.
-
-* Workflows in this category can use sanitizers, debug builds, alternate
-  compilers, and other features that maintainers want automated coverage for.
-
-Workflow file | Build status | Event triggers
--- | --: | --
-[`ci_linux_x64_clang_asan.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_asan.yml) | [![CI - Linux x64 clang ASan](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_asan.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_asan.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
-[`ci_linux_x64_clang_tsan.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_tsan.yml) | [![CI - Linux x64 clang TSan](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_tsan.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_tsan.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_linux_x64_clang_debug.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_debug.yml) | [![CI - Linux x64 clang debug](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_debug.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_debug.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_linux_x64_gcc.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_gcc.yml) | [![CI - Linux x64 gcc](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_gcc.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_gcc.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_linux_x64_clang_byollvm.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_byollvm.yml) | [![CI - Linux x64 clang_byollvm](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_byollvm.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_clang_byollvm.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
-[`ci_linux_x64_bazel.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_bazel.yml) | [![CI - Linux x64 bazel](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_bazel.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci_linux_x64_bazel.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
-
-<!-- TODO(scotttodd): the remains of ci.yml (runtime builds, etc.) -->
-<!-- TODO(scotttodd): cross-compilation jobs -->
-
 ### Other workflows
 
 Workflow file | Build status | Event triggers
 -- | --: | --
-[`ci.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci.yml) | [![cI](https://github.com/iree-org/iree/actions/workflows/ci.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/ci.yml?query=branch%3Amain+event%3Apush) | `pull_request`, `push`
 [`build_package.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/build_package.yml) | [![Build Release Packages](https://github.com/iree-org/iree/actions/workflows/build_package.yml/badge.svg)](https://github.com/iree-org/iree/actions/workflows/build_package.yml) | `schedule`
-[`publish_website.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/publish_website.yml) | [![publish_website](https://github.com/iree-org/iree/actions/workflows/publish_website.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/publish_website.yml?query=branch%3Amain+event%3Apush) | `push`
+[`publish_website.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/publish_website.yml) | [![publish_website](https://github.com/iree-org/iree/actions/workflows/publish_website.yml/badge.svg?query=branch%3Amain+event%3Apush)](https://github.com/iree-org/iree/actions/workflows/publish_website.yml?query=branch%3Amain+event%3Apush) | `push`, `release`, `schedule`
 [`samples.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/samples.yml) | [![Samples](https://github.com/iree-org/iree/actions/workflows/samples.yml/badge.svg?query=branch%3Amain+event%3Aschedule)](https://github.com/iree-org/iree/actions/workflows/samples.yml?query=branch%3Amain+event%3Aschedule) | `schedule`
 
 ## :octicons-pencil-16: Writing and editing workflows