From 29defdc1949335acfaf79a02a173c6de425ccb74 Mon Sep 17 00:00:00 2001 From: Arpan Tripathi Date: Sat, 27 Aug 2022 01:33:05 +0530 Subject: [PATCH] v0.8 --- HF-only/README.md | 7 ++++--- HF-only/prepare_mar_from_hf.sh | 9 +++++---- README.md | 4 +++- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/HF-only/README.md b/HF-only/README.md index fd9645f..a8d95be 100644 --- a/HF-only/README.md +++ b/HF-only/README.md @@ -141,10 +141,11 @@ The high level approach to use any of the 🤗 pipelines for a desired [*task*]( * Decide a name (variable `$modelName` below) for the task (variable `$task` below) supported by a 🤗 model that supports the task (its repo link as variable `$repoUrl` below), then run the below commands to download the models and create the `.mar` file: ``` -$modelName="vitxxsmall" -$task="image-classification" -$repoUrl="https://huggingface.co/apple/mobilevit-xx-small/tree/main" +modelName="vitxxsmall" +task="image-classification" +repoUrl="https://huggingface.co/apple/mobilevit-xx-small" +chmod +x prepare_mar_from_hf.sh ./prepare_mar_from_hf.sh -t $task -n $modelName -u $repoUrl ``` diff --git a/HF-only/prepare_mar_from_hf.sh b/HF-only/prepare_mar_from_hf.sh index c12f4af..fd42f68 100644 --- a/HF-only/prepare_mar_from_hf.sh +++ b/HF-only/prepare_mar_from_hf.sh @@ -5,12 +5,13 @@ for arg in "$@" do case $arg in -t|--task) - sed -i "s/task=\".*/task=\"$2\"/g" HF-only/scripts/torchserve_vitxxsmall_handler.py - shift + sed -i "s/task=\".*/task=\"$2\"/g" scripts/torchserve_vitxxsmall_handler.py + shift 2 ;; -n|--model-name) MODEL_NAME="$2" - shift + echo "MODEL_NAME is $MODEL_NAME" + shift 2 ;; -u|--hf-hub-link) HF_REPO_URL="$2" @@ -23,7 +24,7 @@ do touch dummy_file.pth torch-model-archiver --model-name $MODEL_NAME --serialized-file dummy_file.pth --version 1.0 --handler scripts/torchserve_vitxxsmall_handler.py --export-path model-store -r requirements.txt rm -f dummy_file.pth - shift + shift 2 ;; esac done \ No newline at end of file diff --git a/README.md b/README.md index 50a39b4..c26483d 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ This repository contains an example to deploy models with third-party dependenci For the context of this repository, we would deploy the models on an AWS [`t2.micro`](https://aws.amazon.com/ec2/instance-types/) instance which can be used for free (for 750 hours) on a new AWS account. We work with a 🤗 MobileViT Transformer [model](https://huggingface.co/apple/mobilevit-xx-small) for the task of image classification by using its [`pipeline`](https://huggingface.co/docs/transformers/main_classes/pipelines) feature, the handler code in `scripts` can also be used as a simplistic template to deploy an 🤗 `pipeline`. +This work can also be [extended to deploy *any* 🤗 `pipeline` for *any* supported task with Torchserve](https://github.com/tripathiarpan20/HF-torchserve-pipeline/tree/main/HF-only#instructions-to-use-any--model-from-the-hub-for-any-task-supported-by-the--pipeline). + This work *may* also be extended to deploy the Torchserve Docker containers with HF models at scale with [AWS Cloudformation](https://github.com/pytorch/serve/tree/master/examples/cloudformation) & [AWS EKS](https://github.com/pytorch/serve/tree/master/kubernetes/EKS) as explained in the official Torchserve repo & [AWS Sagemaker](https://github.com/tescal2/TorchServeOnAWS/tree/master/3_torchserve_byoc_with_amazon_sagemaker), incorporating utilities like AWS ELB & Cloudwatch. We would also benchmark the REST API calls in time units and compare the model performances for the following approaches: @@ -12,7 +14,7 @@ We would also benchmark the REST API calls in time units and compare the model p ## Todo -- [ ] Verify HF pipeline functionality with AWS EC2 `t2.micro` +- [X] Verify HF pipeline functionality with AWS EC2 `t2.micro` - [ ] Add [AB Testing](https://github.com/pytorch/serve/tree/master/benchmarks) experiments with Torchserve Benchmarking utilities. - [ ] Add dynamic batching explanation - [ ] Add Torchscripted model code