Skip to content

Commit

Permalink
ci: share onnx model between jobs
Browse files Browse the repository at this point in the history
python and java application merged into same workflow
model converted to onnx is also shared between those jobs
  • Loading branch information
Ahwar committed Aug 25, 2024
1 parent f0b98c8 commit 3ee90d3
Show file tree
Hide file tree
Showing 2 changed files with 61 additions and 47 deletions.
73 changes: 61 additions & 12 deletions .github/workflows/java-app.yml
Original file line number Diff line number Diff line change
@@ -1,34 +1,83 @@
# This workflow will build a Java project with Gradle and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-java-with-gradle

name: Java CI with Gradle
# This workflow converts a Hugging Face model to ONNX format, builds a Java application with Gradle,
# and generates and submits a dependency graph for the project.
name: Java application

on:
push:
branches: [ "main" ]
branches: ["main"]
pull_request:
branches: [ "main" ]
branches: ["main"]

permissions:
contents: read

jobs:
build:
convert-model-to-onnx:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -q transformers[onnx] transformers[sentencepiece] torch
- name: Hugging Face Model to ONNX
run: |
# Convert Hugging Face Model to ONNX
python -m transformers.onnx --opset 16 --atol 0.005 --feature=token-classification --model=xlm-roberta-large-finetuned-conll03-english onnx_model/
# Upload the ONNX model as an artifact
- name: Upload ONNX model
uses: actions/[email protected]
with:
# Artifact name
name: onnx_model # default: artifact
# Files to upload
path: onnx_model/
# Behavior if no files found: warn, error, ignore
if-no-files-found: error
# Expiration in days (1-90, 0 for default)
retention-days: 0
# Compression level (0-9, default: 6)
compression-level: 7
# Overwrite existing artifact (default: false)
overwrite: true

# Build Java application with Gradle
build-java:
runs-on: ubuntu-latest
permissions:
contents: read
needs: convert-model-to-onnx

steps:
- uses: actions/checkout@v4

# Set up JDK 17
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
java-version: "17"
distribution: "temurin"

# Configure Gradle for optimal use in GitHub Actions, including caching of downloaded dependencies.
# See: https://github.com/gradle/actions/blob/main/setup-gradle/README.md
- name: Setup Gradle
uses: gradle/actions/setup-gradle@af1da67850ed9a4cedd57bfd976089dd991e2582 # v4.0.0

- name: Download ONNX model from Artifact
uses: actions/[email protected]
with:
name: onnx_model
path: raw-files # path to download the artifact to
run-id: ${{ github.run_id }}

- name: See artifact contents
run: tree raw-files

# This job is responsible for generating and submitting a dependency graph for the project.
# Generates and submits a dependency graph, enabling Dependabot Alerts for all project dependencies.
# See: https://github.com/gradle/actions/blob/main/dependency-submission/README.md
Expand All @@ -43,8 +92,8 @@ jobs:
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
java-version: "17"
distribution: "temurin"

- name: Generate and submit dependency graph
uses: gradle/actions/dependency-submission@af1da67850ed9a4cedd57bfd976089dd991e2582 # v4.0.0
35 changes: 0 additions & 35 deletions .github/workflows/python-app.yml

This file was deleted.

0 comments on commit 3ee90d3

Please sign in to comment.