[LLM] merge code from fastdeploy

[kevincheng2]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

Others

PR changes

Others

Description

merge code from fastdeploy

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.37%. Comparing base (fb3e4c0) to head (f17e071).
Report is 4 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #9791      +/-   ##
===========================================
+ Coverage    51.64%   52.37%   +0.73%     
===========================================
  Files          741      730      -11     
  Lines       119141   115270    -3871     
===========================================
- Hits         61525    60376    -1149     
+ Misses       57616    54894    -2722     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM