diff --git a/docs/images/models-workshop/create-OCP-route.png b/docs/images/models-workshop/create-OCP-route.png new file mode 100644 index 00000000..6e2c6a53 Binary files /dev/null and b/docs/images/models-workshop/create-OCP-route.png differ diff --git a/docs/images/models-workshop/register-model.png b/docs/images/models-workshop/register-model.png new file mode 100644 index 00000000..333dfe0d Binary files /dev/null and b/docs/images/models-workshop/register-model.png differ diff --git a/docs/images/models-workshop/upload-model-yaml.png b/docs/images/models-workshop/upload-model-yaml.png new file mode 100644 index 00000000..a66c32f0 Binary files /dev/null and b/docs/images/models-workshop/upload-model-yaml.png differ diff --git a/docs/images/models-workshop/view-all-models.png b/docs/images/models-workshop/view-all-models.png new file mode 100644 index 00000000..0fc1d2b9 Binary files /dev/null and b/docs/images/models-workshop/view-all-models.png differ diff --git a/docs/mlx-models-workshop.md b/docs/mlx-models-workshop.md new file mode 100644 index 00000000..a43c0157 --- /dev/null +++ b/docs/mlx-models-workshop.md @@ -0,0 +1,276 @@ +# Workshop - How to Add a New Model to MLX + +## Table of Contents + + + +- [Prerequisites](#prerequisites) +- [Add a New Model to MLX](#add-a-new-model-to-mlx) + - [Containerize the Model](#containerize-the-model) + - [Create the Model Metadata (YAML)](#create-the-model-metadata-yaml) + - [Create a Model README (Optional)](#create-a-model-readme-optional) + - [Register Model in MLX Catalog](#register-model-in-mlx-catalog) + - [Publish and Feature the Model](#publish-and-feature-the-model) + - [Delete a Model](#delete-a-model) + - [Create `catalog_upload.json` to Upload Multiple Models at Once](#create-catalog_uploadjson-to-upload-multiple-models-at-once) +- [Serve the Model](#serve-the-model) + - [Launch Model Deployment Pipeline](#launch-model-deployment-pipeline) + - [Create Route or Port Forward For Inferencing](#create-route-or-port-forward-for-inferencing) + - [Test the Model](#test-the-model) + - [Conclusion](#conclusion) + + + + +# Prerequisites + +For this workshop you will need access to a Kubernetes cluster with MLX deployed, +or a local deployment of MLX on KIND (Kubernetes in Docker). The Quickstart with +Docker Compose allows uploading and deleting models, but it does not support +serving models using Kubeflow Pipelines. + +- [Deploy MLX on a Kubernetes cluster](mlx-setup.md) +- [Deploy MLX locally on KIND](install-mlx-on-kind.md) + +There are only minor differences in how to navigate the Kubernetes resources +depending on the selected deployment option, with the most notable difference +being the host and port of the MLX (UI) server. + +You do need to be logged in as an `admin` user in order to register and deploy +models. + + +# Add a New Model to MLX + +## Containerize the Model + +Most of the sample models in the MLX catalog are from IBM Model Asset Exchange (MAX) which +was sunset early 2022. The model files are hosted on IBM Cloud Object Storage. +The code to deploy the model as a web service in a Docker container can be found in their +respective [Github repositories](https://github.com/orgs/IBM/repositories?q=%22MAX-%22&type=all&language=&sort=name) +i.e. [CodeNet-Language-Classification](https://github.com/CODAIT/MAX-CodeNet-Language-Classification) +A detailed guide on how to create a Flask app to wrap the model along with a Swagger +API user interface can be found in this +[step-by-step guide](https://github.com/IBM/MAX-skeleton#step-by-step-guide-to-wrapping-a-model) + +Once the model is containerized, it has to be pushed to a public container registry +like [Quay.io](https://quay.io/) or [DockerHub](https://hub.docker.com/) so it can be +referenced using the `container_image_url` field in the model YAML in the next step. + +## Create the Model Metadata (YAML) + +The model metadata has required fields to display the model in the MLX UI as well as optional fields which determine what +can be done with the model, i.e. + +```YAML +name: "CodeNet Language Classifier" +model_identifier: "codenet-language-classification" +description: "A convolutional deep neural network to classify snippets of code" +framework: + name: "TensorFlow" + version: "2.5" + +license: "Apache 2.0" +domain: "Code Classification" +website: "https://github.com/CODAIT/MAX-CodeNet-Language-Classification" + +serve: + servable: true + tested_platforms: + - kubernetes + - kfserving + serving_container_image: + container_image_url: "codait/codenet-language-classifier" + +readme_url: "https://raw.githubusercontent.com/machine-learning-exchange/katalog/main/model-samples/codenet-language-classification/codenet-language-classification.md" +``` + +A detailed description of how to create model metadata (YAML) for either training or serving +can be found here [models/README.md](/models/README.md#create-model-metadata) + +## Create a Model README (Optional) + +Each model can have a Github-flavored markdown (`.md`) file which will be rendered +in the MLX UI. If no `readme_url` is specified in the model's YAML file, a general +overview is created from the contents of the YAML file. Examples can be found in +the [MLX Katalog repo](https://github.com/machine-learning-exchange/katalog/tree/main/model-samples) +i.e. the [Codenet README](https://github.com/machine-learning-exchange/katalog/blob/main/model-samples/codenet-language-classification/codenet-language-classification.md) + +## Register Model in MLX Catalog + +The model metadata YAML file can be uploaded from a local file or by providing a +raw url to a README.md file in a Github repository. + +![register model](images/models-workshop/register-model.png) + +1. Click on the "Models" link in left-hand navigation panel +2. Click on "Register a Model" +3. Click on the "Browse" button to select a YAML file or provide a URL to download the YAML file + * `.tar.gz` and `.tgz` files containing the compressed + `.yaml` specification can also be uploaded +4. Enter a name for the model; Otherwise the name from the YAML file will be used + +![upload model YAML](images/models-workshop/upload-model-yaml.png) + +More details can be found [here](/models/README.md#register-a-model) + + +## Publish and Feature the Model + +When a new model is uploaded, it is automatically "featured" on the main Models +page. In order to remove (or add models back) to the featured page, click on the +**View all Models** button and select/unselect the respective check boxes. + +![VIEW ALL MODELS](images/models-workshop/view-all-models.png) + +## Delete a Model + +The **View all Models** page can also be used to remove models from the MLX catalog. + +## Create `catalog_upload.json` to Upload Multiple Models at Once + +MLX supports uploading multiple catalog assets at once ("bulk"-upload) as described +in this [guide](/docs/import-assets.md). + +![Catalog Import Screenshot](/docs/images/CatalogImport.png) + +Take a look at the `catalog_upload.json` file under [MLX Bootstrapper](/bootstrapper/catalog_upload.json) + +```JSON +{ + "models": [ + { + "name": "CodeNet Language Classification", + "url": "https://raw.githubusercontent.com/machine-learning-exchange/katalog/main/model-samples/codenet-language-classification/codenet-language-classification.yaml" + }, + { + "name": "Human Pose Estimator", + "url": "https://raw.githubusercontent.com/machine-learning-exchange/katalog/main/model-samples/max-human-pose-estimator/max-human-pose-estimator.yaml" + }, + { + "name": "Image Caption Generator", + "url": "https://raw.githubusercontent.com/machine-learning-exchange/katalog/main/model-samples/max-image-caption-generator/max-image-caption-generator.yaml" + } + ] +} +``` + + +# Serve the Model + +Models can easily be trained or served using Kubeflow Pipelines depending on the +`train` or `serve` metadata specified in the YAML file. + +More details can be found [here](/models/README.md#serve-the-model) and in this +[MLX Workshop](mlx-workshop.md#serve-the-model) using the Codenet Language +Classification model as an example. + + +## Launch Model Deployment Pipeline + +Models that have the `serve` specification can be deployed directly from the MLX UI: + +1. Under the models tab, select a model +2. Switch to the "LAUNCH" tab +3. Optionally, give the pipeline run a name +4. Click submit to run the pipeline which will deploy the model + +We have created a containerized version of the **CodeNet Language Classification** +model that can be deployed on Kubernetes to allow inferencing via `curl`, or, +directly from a web browser using a simple Swagger UI to the inferencing service. + +![Models Featured Page](./images/workshop/Models-featured-page.png) + +Click on the **Models** menu item and select the **CodeNet Language Classification** +model. + +* The **DESCRIPTION** tab shows the details of the model with links to the original + dataset +* The **LAUNCH** tab allows users to (train or) serve the model on Kubernetes +* The **YAML DEFINITION** tab shows the metadata required by MLX +* The **SAMPLE SERVING CODE** tab displays a generated sample pipeline to serve + the model + +![Models Launch Tab](./images/workshop/Models-launch.png) + +Click on the **LAUNCH** tab to start the pipeline run to serve the model +* The **Launch Type** is `Serving` +* The **Platform** is `Kubernetes` +* Enter a **Run Name** or leave the default +* Click **SUBMIT** to start the pipeline run + +![Models Run Output](./images/workshop/Models-run-output.png) + +Now the Kubeflow Pipeline run view should appear, and get updated as the pipeline +execution progresses. You can click on the task in the pipeline graph and follow +the **Logs**. Once the second step of the pipeline is completed, you can find the +name of the deployed model "`codenet-language-classification`". + + +## Create Route or Port Forward For Inferencing + +Once a model has been deployed for inferencing, it has to be made available via +a local port forwarding or a OpenShift Route. + +![Create Model Route](images/models-workshop/create-OCP-route.png) + +For MLX on KIND we need to make the port available on localhost by running the +following command: + +```Bash +kubectl wait --for=condition=ready pod -l app=codenet-language-classification && \ + kubectl port-forward svc/codenet-language-classification 5000:5000 +``` + +Now we should be able to access the UI page for the deployed model by pasting +`localhost:5000` into our browser's address bar: + +``` +http://localhost:5000/ +``` + + +## Test the Model + +Expand the first section called **model** and click on **POST /model/predict** +and the click on the **Try it out** button on the very right of the **Parameters** +line. + +![Models Swagger UI](./images/workshop/Models-Swagger-UI.png) + +In order to test the classification model, we need to provide a source code file +in any of the 10 supported languages like C/C++, Java, JavaScript or Python. +You could either use a source file from one of your local projects or download +the [CodeNet Sample Dataset for Language Classification](https://dax-cdn.cdn.appdomain.cloud/dax-project-codenet/1.0.0/Project_CodeNet_LangClass.tar.gz) +and extract it to a local folder. + +On the model prediction input dialog, click **Browse** and select a source +file + +![Open Source File](./images/workshop/Models-predict-open-source-file.png) + +Now click **Execute** to get the model's prediction. + +The Response should look similar to this: + +**Response body** + + { + "status": "ok", + "predictions": [ + { + "language": "Haskell", + "probability": 0.9999926090240479 + } + ] + } + + +## Conclusion + +In this workshop we demonstrated how to use MLX to register a model to the MLX +catalog and how MLX can be used to deploy pretrained containerized models on +Kubernetes. + +For comments, suggestions or bug reports, please do to +https://github.com/machine-learning-exchange/mlx/issues diff --git a/models/README.md b/models/README.md index 3cb320a8..2dd048f2 100644 --- a/models/README.md +++ b/models/README.md @@ -1,6 +1,18 @@ # Models -MLX supports the use of pre-trained models. +MLX supports training models and serving of pre-trained models. + +## Table of Contents + + + +- [Create Model Metadata](#create-model-metadata) + - [Metadata template for a trainable model](#metadata-template-for-a-trainable-model) + - [Metadata template for a servable model](#metadata-template-for-a-servable-model) +- [Register a Model](#register-a-model) +- [Serve the Model](#serve-the-model) + + ## Create Model Metadata @@ -35,6 +47,7 @@ framework: license: "Apache 2.0" domain: "Domain Area" website: # Can be GitHub link +readme_url: # Github-flavored markdown, Github raw URL, will be displayed in MLX UI train: trainable: true @@ -87,6 +100,7 @@ framework: license: "Apache 2.0" domain: "Domain Area" website: # Can be GitHub link +readme_url: # Github-flavored markdown, Github raw URL, will be displayed in MLX UI serve: servable: true @@ -97,16 +111,20 @@ serve: container_image_url: ``` -## Register Model +## Register a Model + 1. Click on the "Models" link in left-hand navigation panel -2. Click on "Upload a Model" -3. Select a file to upload (Must be `.tar.gz` or `.tgz` format) - * This will be the compressed `.yaml` specification -4. Enter a name for the model; Otherwise a default will be given +2. Click on "Register a Model" +3. Select a YAML file to be uploaded or provide a URL to download the YAML file + * `.tar.gz` and `.tgz` files containing the compressed + `.yaml` specification can also be uploaded +4. Enter a name for the model; Otherwise the name from the YAML file will be used + +## Serve the Model -## Use Models in Pipelines Models can easily be executed based on the metadata specified in the YAML file for a particular function 1. Under the models tab, select a model -2. Switch to the "CREATE RUN" section -3. Give the name a run and click submit to serve the model +2. Switch to the "LAUNCH" tab +3. Optionally, provide a run name +4. Click submit to run the pipeline which will deploy the model diff --git a/models/template.yaml b/models/template.yaml index 1b9ce25d..0331be6e 100644 --- a/models/template.yaml +++ b/models/template.yaml @@ -33,6 +33,7 @@ framework: # license: (Optional) License for this model. # domain: (Optional) Domain metadata for this model. # website: (Optional) Links that explain this model in more details +# readme_url: (Optional) A Github-flavored markdown file to be rendered in the MLX UI license: "Apache 2.0" domain: "Facial Recognition" @@ -40,6 +41,7 @@ website: "https://developer.ibm.com/exchanges/models/all/max-facial-age-estimato labels: - url: - pipeline_uuids: ["abcd1234"] +readme_url: "https://raw.githubusercontent.com/IBM/MAX-Facial-Age-Estimator/master/README.md" # train: (optional) # trainable: (optional) Indicate the model is trainable. Default: False diff --git a/tools/bash/mdtoc.sh b/tools/bash/mdtoc.sh new file mode 100755 index 00000000..670c0104 --- /dev/null +++ b/tools/bash/mdtoc.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# Copyright 2022 The MLX Contributors +# +# SPDX-License-Identifier: Apache-2.0 + + +# This script will generate a table of contents (ToC) for Markdown (MD) files. +# +# 1. Remove the paragraphs (headings) above the "Table of Contents" +# 2. Remove code blocks fenced by tripple back-ticks, to not treat #-comments as markdown headings +# 3. Find the paragraph headings with grep (1st through 4th level heading starting with "#" and "####") +# 4. Extract the heading's text with sed and transform into '|'-separated records of the form '###|Full Text|Full Text' +# 5. Generate the ToC lines with awk by replacing '#' with ' ', converting spaces to dashes '-', +# removing special chars (like back-ticks, dot, parenthesis, colon, comma) from TOC anchor links, +# and lower-case all capital letters +# 6. Remove leading 2 spaces in case ToC does not include 1st level headings, otherwise the TOC becomes a code block +# +# Inspired by https://medium.com/@acrodriguez/one-liner-to-generate-a-markdown-toc-f5292112fd14 + +SEP="|" + +[ -z "${1}" ] && echo -e "Usage:\n\n $BASH_SOURCE \n" && exit 1 + +sed -n '/Table of Contents/,$p' "${1}" | tail -n +2 | \ +sed '/^```/,/^```/d' | \ +grep -E "^#{1,4}" | \ +sed -E "s/(#+) (.+)/\1${SEP}\2${SEP}\2/g" | \ +awk -F "${SEP}" '{ gsub(/#/," ",$1); gsub(/[ ]/,"-",$3); gsub(/[`.():,&]/,"",$3); print $1 "- [" $2 "](#" tolower($3) ")" }' | \ +sed -e 's/^ //g'