LAB / PROJECT: KALE (Kubeflow Automated PipeLines Engine) and KATIB (AutoML: Finding Best Hyperparameter Values)
This lab/project shows:
- how to use KALE and KATIB in a project.
- You should have Kubeflow Environment (Easiest Way: Using MiniKF)
-
Create a new notebook server pod and connect:
-
Run Terminal to download examples:
-
Clone Kale Examples:
git clone https://github.com/kubeflow-kale/kale
-
Open the ipynb file (kale/examples/openvaccine-kaggle-competition/open-vaccine.ipynb)
-
Run the cell "pip install -r requirements.txt" to install requirements
-
Then, after installing required packages, restart the kernel.
-
Open the KALE section from left and enable KALE
-
After opening KALE feature, it is seen that each cells are tagged with steps (e.g. imports, pipeline-parameters, etc.)
-
At the end of the notebook, add new cell with "print(validation_loss)" and change the tag (cell-type) "Pipeline Metrics"
-
Enable KATIB:
-
After opening KATIB setting parameters:
-
Run "Compile and Run KATIB Job", this will run KALE and KATIB:
-
After running, click "View" button:
-
We can see the hyperparameter and trials:
-
After waiting some time to finish all trials to find best hyperparameters:
-
In the "Run" section, it can be seen that pipeline is completed and details can be reachable clicking on the each block step:
