Skip to content

Latest commit

 

History

History
67 lines (41 loc) · 2.81 KB

README.md

File metadata and controls

67 lines (41 loc) · 2.81 KB
Raw

Automated Prometheus E2E Testing and Benchmarking

Prombench Design

This setup leverages GitHub Actions and Google Kubernetes Engine (GKE), but is designed to be extendable to other Kubernetes providers.

Configuration Files

The ./manifest directory contains configuration files. We can outline :

  • ./manifest/cluster_gke.yaml: Creates the Main Node in GKE.
  • ./manifest/cluster_eks.yaml: Creates the Main Node in EKS.
  • ./manifest/cluster-infra/: Contains persistent components of the Main Node.
  • ./manifest/prombench/: Resources created and destroyed for each Prombench test. See its README.md for details.

Prombench Setup

Prombench can be run on different providers. Follow these instructions based on your provider:

Setting Up GitHub Actions

  1. Place a workflow file in the .github directory of your repository. Refer to the Prometheus GitHub repository for an example.

  2. Create a GitHub Action secret TEST_INFRA_PROVIDER_AUTH with the base64 encoded content of the AUTH_FILE:

    cat $AUTH_FILE | base64 -w 0

  3. Configure webhook to cluster's comment-monitor as described here.

Prombench Usage

Triggering Tests via GitHub Comment

Starting Tests:

  • /prombench main or /prombench master - Compare PR with the main/master branch.
  • /prombench v2.4.0 - Compare PR with a specific release version (e.g., from quay.io/prometheus/prometheus:releaseVersion).
  • /prombench v2.4.0 --bench.version=@aca1803ccf5d795eee4b0848707eab26d05965cc - Compare with 2.4.0 release, but use a specific aca1803ccf5d795eee4b0848707eab26d05965cc commit on this repository for ./manifests/prombench resources.
  • /prombench v2.4.0 --bench.version=mybranch - Compare with 2.4.0 release, but use a specific mybranch on this repository for ./manifests/prombench resources.
  • /prombench v2.4.0 --bench.directory=manifests/prombench-agent-mode - Compare with 2.4.0 release, but use a specific resource directory on master branch for this repository. Currently there is only ./manifests/prombench available (default), we might add more modes in the future.

Restarting Tests:

  • /prombench restart <release_version>
  • /prombench restart <release_version> --bench.version=... --bench.directory...

Stopping Tests:

  • /prombench cancel

Printing available commands:

  • /prombench help

Building the Docker Image

Build the Docker image with:

docker build -t prominfra/prombench:master .