Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Container Images] Changes to integrate us with heighliner #519

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from 21 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/container-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Create and Push Container Image (heighliner)

on:
push:
tags:
- '**'
branches:
- '**'

jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image
uses: strangelove-ventures/[email protected]
with:
chain: pocket-network
dockerfile: cosmos
platform: linux/arm64,linux/amd64
pre-build: |
make -d heighliner_pre_build
build-target: |
make -d install
binaries: |
- /go/bin/poktrolld
193 changes: 94 additions & 99 deletions .github/workflows/main-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,81 +13,76 @@ jobs:
build-push-container:
runs-on: ubuntu-latest
steps:
- name: install ignite
# TODO_TECHDEBT: upgrade to the latest Ignite (the latest at the moment of creating a note is 0.28). Need to downgrade to fix CI pipelines. Might be done in scope of #240.
run: |
# curl https://get.ignite.com/cli! | bash
wget https://github.com/ignite/cli/releases/download/v28.2.0/ignite_28.2.0_linux_amd64.tar.gz
tar -xzf ignite_28.2.0_linux_amd64.tar.gz
sudo mv ignite /usr/local/bin/ignite
ignite version

- uses: actions/checkout@v3
with:
fetch-depth: "0" # Per https://github.com/ignite/cli/issues/1674#issuecomment-1144619147

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: "1.21.6"

- name: Install CI dependencies
run: make install_ci_deps

- name: Generate protobufs
run: make proto_regen

- name: Build
run: ignite chain build -v --debug --skip-proto

- name: Set up Docker Buildx
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/setup-buildx-action@v3

- name: Docker Metadata action
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
id: meta
uses: docker/metadata-action@v5
env:
DOCKER_METADATA_PR_HEAD_SHA: "true"
with:
images: |
ghcr.io/pokt-network/poktrolld
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha
type=sha,format=long

- name: Login to GitHub Container Registry
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Copy binaries to inside of the Docker context
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
run: |
mkdir -p ./bin # Make sure the bin directory exists
cp $(which ignite) ./bin # Copy ignite binary to the repo's bin directory
cp $(go env GOPATH)/bin/poktrolld ./bin # Copy the binary to the repo's bin directory
ls -la ./bin

- name: Build and push Docker image
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/build-push-action@v5
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# NB: Uncomment below if arm64 build is needed; arm64 builds are off by default because build times are significant.
platforms: linux/amd64 #,linux/arm64
file: Dockerfile.dev
cache-from: type=gha
cache-to: type=gha,mode=max
context: .
- uses: actions/checkout@v3
with:
fetch-depth: "0" # Per https://github.com/ignite/cli/issues/1674#issuecomment-1144619147

- name: install ignite
run: |
make ignite_install

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: "1.22.2"

- name: Install CI dependencies
run: make install_ci_deps

- name: Generate protobufs
run: make proto_regen

- name: Build
run: ignite chain build -v --debug --skip-proto

- name: Set up Docker Buildx
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/setup-buildx-action@v3

- name: Docker Metadata action
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
id: meta
uses: docker/metadata-action@v5
env:
DOCKER_METADATA_PR_HEAD_SHA: "true"
with:
images: |
ghcr.io/pokt-network/poktrolld
tags: |
type=ref,event=branch,suffix=-dev
type=ref,event=pr
type=sha
type=sha,format=long

- name: Login to GitHub Container Registry
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Copy binaries to inside of the Docker context
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
run: |
mkdir -p ./bin # Make sure the bin directory exists
cp $(which ignite) ./bin # Copy ignite binary to the repo's bin directory
cp $(go env GOPATH)/bin/poktrolld ./bin # Copy the binary to the repo's bin directory
ls -la ./bin

- name: Build and push Docker image
if: (github.ref == 'refs/heads/main') || (contains(github.event.pull_request.labels.*.name, 'push-image'))
uses: docker/build-push-action@v5
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# NB: Uncomment below if arm64 build is needed; arm64 builds are off by default because build times are significant.
platforms: linux/amd64 #,linux/arm64
file: Dockerfile.dev
cache-from: type=gha
cache-to: type=gha,mode=max
context: .

run-e2e-tests:
timeout-minutes: 20
Expand All @@ -98,27 +93,27 @@ jobs:
GKE_CLUSTER: protocol-us-central1
GKE_ZONE: us-central1
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
.github

- id: "auth"
name: "Authenticate to Google Cloud"
uses: "google-github-actions/auth@v1"
with:
credentials_json: "${{ secrets.GKE_PROTOCOL_US_CENTRAL }}"

- uses: google-github-actions/get-gke-credentials@v1
with:
cluster_name: ${{ env.GKE_CLUSTER }}
location: ${{ env.GKE_ZONE }}
project_id: ${{ secrets.GKE_PROTOCOL_PROJECT }}

- name: Run E2E test job
env:
IMAGE_TAG: sha-${{ github.event.pull_request.head.sha || github.sha }}
NAMESPACE: devnet-issue-${{ github.event.number }}
JOB_NAME: e2e-test-${{ github.event.pull_request.head.sha || github.sha }}
POCKET_NODE: tcp://devnet-issue-${{ github.event.number }}-validator-poktrolld:36657
run: bash .github/workflows-helpers/run-e2e-test.sh
- uses: actions/checkout@v4
with:
sparse-checkout: |
.github

- id: "auth"
name: "Authenticate to Google Cloud"
uses: "google-github-actions/auth@v1"
with:
credentials_json: "${{ secrets.GKE_PROTOCOL_US_CENTRAL }}"

- uses: google-github-actions/get-gke-credentials@v1
with:
cluster_name: ${{ env.GKE_CLUSTER }}
location: ${{ env.GKE_ZONE }}
project_id: ${{ secrets.GKE_PROTOCOL_PROJECT }}

- name: Run E2E test job
env:
IMAGE_TAG: sha-${{ github.event.pull_request.head.sha || github.sha }}
NAMESPACE: devnet-issue-${{ github.event.number }}
JOB_NAME: e2e-test-${{ github.event.pull_request.head.sha || github.sha }}
POCKET_NODE: tcp://devnet-issue-${{ github.event.number }}-validator-poktrolld:36657
run: bash .github/workflows-helpers/run-e2e-test.sh
59 changes: 27 additions & 32 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,35 +17,30 @@ jobs:
go-test:
runs-on: ubuntu-latest
steps:
- name: install ignite
# TODO_TECHDEBT: upgrade to the latest Ignite (the latest at the moment of creating a note is 0.28). Need to downgrade to fix CI pipelines. Might be done in scope of #240.
run: |
# curl https://get.ignite.com/cli! | bash
wget https://github.com/ignite/cli/releases/download/v28.2.0/ignite_28.2.0_linux_amd64.tar.gz
tar -xzf ignite_28.2.0_linux_amd64.tar.gz
sudo mv ignite /usr/local/bin/ignite
ignite version

- uses: actions/checkout@v3
with:
fetch-depth: "0" # Per https://github.com/ignite/cli/issues/1674#issuecomment-1144619147

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: "1.21.6"

- name: Install CI dependencies
run: make install_ci_deps

- name: Generate protobufs
run: make proto_regen

- name: Generate mocks
run: make go_mockgen

- name: Run golangci-lint
run: make go_lint

- name: Test
run: make go_test
- uses: actions/checkout@v3
with:
fetch-depth: "0" # Per https://github.com/ignite/cli/issues/1674#issuecomment-1144619147

- name: install ignite
run: |
make ignite_install

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: "1.22.2"

- name: Install CI dependencies
run: make install_ci_deps

- name: Generate protobufs
run: make proto_regen

- name: Generate mocks
run: make go_mockgen

- name: Run golangci-lint
run: make go_lint

- name: Test
run: make go_test
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ api/poktroll/**/*.go-E

# Relase artifacts produced by `ignite chain build --release`
release
release_binaries

# Only keep one go module in our codebase
go.work.sum
Expand All @@ -92,4 +93,4 @@ summary.html
shannon_appgate_config.yaml
shannon_app_config.yaml

gentx-*.json
gentx-*.json
4 changes: 2 additions & 2 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Run `asdf plugin add golang` and `asdf install` to install the dependencies,
# and `asdf current` to switch to the versions of dependencies listed below
golang 1.21.6
go 1.21.6
golang 1.22.2
go 1.22.2
2 changes: 1 addition & 1 deletion Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This Dockerfile is used to build container image for development purposes.
# It intentionally contains no security features, ships with code and troubleshooting tools.

FROM golang:1.21.6 as base
FROM golang:1.22.2 as base

RUN apt update && \
apt-get install -y \
Expand Down
Loading