Skip to content

Commit

Permalink
docs: Add notes for loading HF private models (#5166)
Browse files Browse the repository at this point in the history
Add notes for loading hf private model

Signed-off-by: Sherlock113 <[email protected]>
  • Loading branch information
Sherlock113 authored Jan 3, 2025
1 parent 9cf780d commit 3fe9653
Showing 1 changed file with 7 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,13 @@ BentoML provides an efficient mechanism for loading AI models to accelerate mode
By default, ``HuggingFaceModel`` returns the downloaded model path as a string, which means you can directly pass the path into libraries like ``transformers`` for model loading.

If you deploy the HF model to BentoCloud, you can view and verify it within your Bento on the details page. It is indicated with the HF icon. Clicking it redirects you to the model page on HF.
If your model is hosted in a private repository, specify your endpoint URL through the ``endpoint`` parameter, which defaults to ``https://huggingface.co/``.

.. code-block:: python
model_path = HuggingFaceModel("your_model_id", endpoint="https://my.huggingface.co/")
After deploying the HF model to BentoCloud, you can view and verify it on the Bento details page. It is indicated with the HF icon. Clicking it redirects you to the model page on HF.

.. image:: ../../_static/img/build-with-bentoml/model-loading-and-management/hf-model-on-bentocloud.png
:alt: Hugging Face model marked with an icon on BentoCloud console
Expand Down

0 comments on commit 3fe9653

Please sign in to comment.