deploy: b74dd23

docs/development/custom-models/index.xml

@@ -10,7 +10,7 @@ warning 注意，ChatGLM2-6B 权重对学术研究完全开放，在获得官方
 这里的 http://0.0.0.0:6006 就是连接地址。
 docker 部署 link 镜像名: luanshaotong/reranker:v0.1 端口号: 6006 大小：约8GB 设置安全凭证（即oneapi中的渠道密钥）
 ACCESS_TOKEN=mytoken 运行命令示例
-docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.</description></item><item><title>接入 M3E 向量模型</title><link>/docs/development/custom-models/m3e/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/docs/development/custom-models/m3e/</guid><description>前言 linkFastGPT 默认使用了 openai 的 embedding 向量模型，如果你想私有部署的话，可以使用 M3E 向量模型进行替换。M3E 向量模型属于小模型，资源使用不高，CPU 也可以运行。下面教程是基于 “睡大觉” 同学提供的一个的镜像。
+无需GPU环境，使用CPU运行 docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.</description></item><item><title>接入 M3E 向量模型</title><link>/docs/development/custom-models/m3e/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/docs/development/custom-models/m3e/</guid><description>前言 linkFastGPT 默认使用了 openai 的 embedding 向量模型，如果你想私有部署的话，可以使用 M3E 向量模型进行替换。M3E 向量模型属于小模型，资源使用不高，CPU 也可以运行。下面教程是基于 “睡大觉” 同学提供的一个的镜像。
 部署镜像 link镜像名: stawky/m3e-large-api:latest
 国内镜像： registry.cn-hangzhou.aliyuncs.com/fastgpt_docker/m3e-large-api:latest 端口号: 6008 环境变量：
 # 设置安全凭证（即oneapi中的渠道密钥） 默认值：sk-aaabbbcccdddeeefffggghhhiiijjjkkk 也可以通过环境变量引入：sk-key。有关docker环境变量引入的方法请自寻教程，此处不再赘述。 接入 One API link添加一个渠道，参数如下：

docs/development/custom-models/reranker/index.html

@@ -27,8 +27,28 @@
 <span class=toggle-dark><svg xmlns="http://www.w3.org/2000/svg" height="30" width="30" viewBox="0 0 48 48" fill="currentcolor"><path d="M24 42q-7.5.0-12.75-5.25T6 24t5.25-12.75T24 6q.4.0.85.025.45.025 1.15.075-1.8 1.6-2.8 3.95t-1 4.95q0 4.5 3.15 7.65Q28.5 25.8 33 25.8q2.6.0 4.95-.925T41.9 22.3q.05.6.075.975Q42 23.65 42 24q0 7.5-5.25 12.75T24 42zm0-3q5.45.0 9.5-3.375t5.05-7.925q-1.25.55-2.675.825Q34.45 28.8 33 28.8q-5.75.0-9.775-4.025T19.2 15q0-1.2.25-2.575t.9-3.125q-4.9 1.35-8.125 5.475Q9 18.9 9 24q0 6.25 4.375 10.625T24 39zm-.2-14.85z"/></svg></span><span class=toggle-light><svg xmlns="http://www.w3.org/2000/svg" height="30" width="30" viewBox="0 0 48 48" fill="currentcolor"><path d="M24 31q2.9.0 4.95-2.05T31 24t-2.05-4.95T24 17t-4.95 2.05T17 24t2.05 4.95T24 31zm0 3q-4.15.0-7.075-2.925T14 24t2.925-7.075T24 14t7.075 2.925T34 24t-2.925 7.075T24 34zM3.5 25.5q-.65.0-1.075-.425Q2 24.65 2 24t.425-1.075Q2.85 22.5 3.5 22.5h5q.65.0 1.075.425Q10 23.35 10 24t-.425 1.075T8.5 25.5zm36 0q-.65.0-1.075-.425Q38 24.65 38 24t.425-1.075T39.5 22.5h5q.65.0 1.075.425Q46 23.35 46 24t-.425 1.075-1.075.425zM24 10q-.65.0-1.075-.425Q22.5 9.15 22.5 8.5v-5q0-.65.425-1.075Q23.35 2 24 2t1.075.425T25.5 3.5v5q0 .65-.425 1.075Q24.65 10 24 10zm0 36q-.65.0-1.075-.425T22.5 44.5v-5q0-.65.425-1.075Q23.35 38 24 38t1.075.425.425 1.075v5q0 .65-.425 1.075Q24.65 46 24 46zM12 14.1l-2.85-2.8q-.45-.45-.425-1.075.025-.625.425-1.075.45-.45 1.075-.45t1.075.45L14.1 12q.4.45.4 1.05.0.6-.4 1-.4.45-1.025.45T12 14.1zm24.7 24.75L33.9 36q-.4-.45-.4-1.075t.45-1.025q.4-.45 1-.45t1.05.45l2.85 2.8q.45.45.425 1.075-.025.625-.425 1.075-.45.45-1.075.45t-1.075-.45zM33.9 14.1q-.45-.45-.45-1.05.0-.6.45-1.05l2.8-2.85q.45-.45 1.075-.425.625.025 1.075.425.45.45.45 1.075t-.45 1.075L36 14.1q-.4.4-1.025.4t-1.075-.4zM9.15 38.85q-.45-.45-.45-1.075t.45-1.075L12 33.9q.45-.45 1.05-.45.6.0 1.05.45.45.45.45 1.05.0.6-.45 1.05l-2.8 2.85q-.45.45-1.075.425-.625-.025-1.075-.425zM24 24z"/></svg></span></button></div></div></div><div class=container-fluid><div class=layout-spacing><div class="d-md-flex justify-content-between align-items-center"><nav aria-label=breadcrumb class="d-inline-block pb-2 mt-1 mt-sm-0"><ul id=breadcrumbs class="breadcrumb bg-transparent mb-0" itemscope itemtype=https://schema.org/BreadcrumbList><li class="breadcrumb-item text-capitalize active" aria-current=page itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/><i class="material-icons size-20 align-text-bottom" itemprop=name>Home</i></a><meta itemprop=position content="1"></li><li class="breadcrumb-item text-capitalize" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/development/><span itemprop=name>开发与部署指南</span></a><meta itemprop=position content="2"></li><li class="breadcrumb-item text-capitalize" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/development/custom-models/><span itemprop=name>本地模型使用</span></a><meta itemprop=position content="3"></li><li class="breadcrumb-item text-capitalize active" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><span itemprop=name>接入 ReRank 重排模型</span><meta itemprop=position content="4"></li></ul></nav></div><div class="row flex-xl-nowrap"><div class="docs-toc col-xl-3 d-xl-block"><toc><div class="fw-bold text-uppercase mb-2">接入 ReRank 重排模型</div><nav id=toc><ul><li><ul><li><a href=#推荐配置>推荐配置</a></li><li><a href=#部署>部署</a><ul><li><a href=#环境要求>环境要求</a></li><li><a href=#源码部署>源码部署</a></li><li><a href=#docker-部署>docker 部署</a></li></ul></li><li><a href=#接入-fastgpt>接入 FastGPT</a></li></ul></li></ul></nav></toc></div><div class="docs-toc-mobile d-print-none d-xl-none"><button id=toc-dropdown-btn class="btn-secondary dropdown-toggle" type=button data-bs-toggle=dropdown data-bs-offset=0,0 aria-expanded=false>
 Table of Contents</button><nav id=toc-mobile><ul class=dropdown-menu><li><ul><li><a href=#推荐配置>推荐配置</a></li><li><a href=#部署>部署</a><ul><li><a href=#环境要求>环境要求</a></li><li><a href=#源码部署>源码部署</a></li><li><a href=#docker-部署>docker 部署</a></li></ul></li><li><a href=#接入-fastgpt>接入 FastGPT</a></li></ul></li></ul></nav></div><div class="docs-content col-12 col-xl-9 mt-0"><div class="mb-0 d-flex"><i class="material-icons title-icon me-2">sort</i><h1 class="content-title mb-0">接入 ReRank 重排模型</h1></div><p class="lead mb-3">接入 ReRank 重排模型</p><div id=content class=main-content data-bs-spy=scroll data-bs-root-margin="0px 0px -65%" data-bs-target=#toc-mobile><div data-prismjs-copy data-prismjs-copy-success data-prismjs-copy-error><h2 id=推荐配置>推荐配置 <a href=#%e6%8e%a8%e8%8d%90%e9%85%8d%e7%bd%ae class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><p>推荐配置如下：</p><table class="table table-hover table-striped-columns"><thead><tr><th>类型</th><th>内存</th><th>显存</th><th>硬盘空间</th><th>启动命令</th></tr></thead><tbody><tr><td>base</td><td>>=4GB</td><td>>=3GB</td><td>>=8GB</td><td>python app.py</td></tr></tbody></table><h2 id=部署>部署 <a href=#%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><h3 id=环境要求>环境要求 <a href=#%e7%8e%af%e5%a2%83%e8%a6%81%e6%b1%82 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>Python 3.10.11</li><li>CUDA 11.7</li><li>科学上网环境</li></ul><h3 id=源码部署>源码部署 <a href=#%e6%ba%90%e7%a0%81%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ol><li>根据上面的环境配置配置好环境，具体教程自行 GPT；</li><li>下载 <a href=https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base rel=external target=_blank>python 文件<svg width="16" height="16" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="currentcolor" d="M14 5c-.552.0-1-.448-1-1s.448-1 1-1h6c.552.0 1 .448 1 1v6c0 .552-.448 1-1 1s-1-.448-1-1V6.414l-7.293 7.293c-.391.39-1.024.39-1.414.0-.391-.391-.391-1.024.0-1.414L17.586 5H14zM5 7c-.552.0-1 .448-1 1v11c0 .552.448 1 1 1h11c.552.0 1-.448 1-1v-4.563c0-.552.448-1 1-1s1 .448 1 1V19c0 1.657-1.343 3-3 3H5c-1.657.0-3-1.343-3-3V8c0-1.657 1.343-3 3-3h4.563c.552.0 1 .448 1 1s-.448 1-1 1H5z"/></svg></a></li><li>在命令行输入命令 <code>pip install -r requirements.txt</code>；</li><li>按照<a href=https://huggingface.co/BAAI/bge-reranker-base rel=external target=_blank>https://huggingface.co/BAAI/bge-reranker-base<svg width="16" height="16" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="currentcolor" d="M14 5c-.552.0-1-.448-1-1s.448-1 1-1h6c.552.0 1 .448 1 1v6c0 .552-.448 1-1 1s-1-.448-1-1V6.414l-7.293 7.293c-.391.39-1.024.39-1.414.0-.391-.391-.391-1.024.0-1.414L17.586 5H14zM5 7c-.552.0-1 .448-1 1v11c0 .552.448 1 1 1h11c.552.0 1-.448 1-1v-4.563c0-.552.448-1 1-1s1 .448 1 1V19c0 1.657-1.343 3-3 3H5c-1.657.0-3-1.343-3-3V8c0-1.657 1.343-3 3-3h4.563c.552.0 1 .448 1 1s-.448 1-1 1H5z"/></svg></a>下载模型仓库到app.py同级目录</li><li>添加环境变量 <code>export ACCESS_TOKEN=XXXXXX</code> 配置 token，这里的 token 只是加一层验证，防止接口被人盗用，默认值为 <code>ACCESS_TOKEN</code> ；</li><li>执行命令 <code>python app.py</code>。</li></ol><p>然后等待模型下载，直到模型加载完毕为止。如果出现报错先问 GPT。</p><p>启动成功后应该会显示如下地址：</p><p><img src=https://cdn.jsdelivr.us/gh/yangchuansheng/fastgpt-imgs@main/imgs/chatglm2.png alt loading=lazy class=medium-zoom-image></p><blockquote><p>这里的 <code>http://0.0.0.0:6006</code> 就是连接地址。</p></blockquote><h3 id=docker-部署>docker 部署 <a href=#docker-%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>镜像名: <code>luanshaotong/reranker:v0.1</code></li><li>端口号: 6006</li><li>大小：约8GB</li></ul><p><strong>设置安全凭证（即oneapi中的渠道密钥）</strong></p><div class=prism-codeblock><pre id=e45f452 class=language->
   <code>ACCESS_TOKEN=mytoken</code>
-  </pre></div><p><strong>运行命令示例</strong></p><div class=prism-codeblock><pre id=621e3ae class=language-sh>
+  </pre></div><p><strong>运行命令示例</strong></p><ul><li>无需GPU环境，使用CPU运行</li></ul><div class=prism-codeblock><pre id=621e3ae class=language-sh>
   <code>docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.1</code>
+  </pre></div><ul><li>需要CUDA 11.7环境</li></ul><div class=prism-codeblock><pre id=e30aa75 class=language-sh>
+  <code>docker run -d --gpus all --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.1</code>
+  </pre></div><p><strong>docker-compose.yml示例</strong></p><div class=prism-codeblock><pre id=d55dc35 class=language->
+  <code>version: &#34;3&#34;
+services:
+  reranker:
+    image: luanshaotong/reranker:v0.1
+    container_name: reranker
+    # GPU运行环境，如果宿主机未安装，将deploy配置隐藏即可
+    deploy:
+      resources:
+        reservations:
+          devices:
+          - driver: nvidia
+            count: all
+            capabilities: [gpu]
+    ports:
+      - 6006:6006
+    environment:
+      - ACCESS_TOKEN=mytoken</code>
   </pre></div><h2 id=接入-fastgpt>接入 FastGPT <a href=#%e6%8e%a5%e5%85%a5-fastgpt class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><p>参考 <a data-bs-delay='{"hide":300,"show":550}' data-bs-html=true data-bs-title="<a href='/docs/development/configuration/'><p>开发与部署指南</p><strong>配置文件介绍</strong><br>FastGPT 配置参数介绍</a>" data-bs-toggle=tooltip href=/docs/development/configuration/#rerank-接入>ReRank模型接入</a>，host 变量为部署的域名。</p></div><div class="gitinfo d-flex flex-wrap justify-content-between align-items-center opacity-85 pt-3"><div id=edit-this-page class=mt-1><a href=https://github.com/labring/FastGPT/blob/main/docSite/content/docs/development/custom-models/reranker.md alt="接入 ReRank 重排模型" rel="noopener noreferrer" target=_blank><span class="me-1 align-text-bottom"><svg width="20" height="20" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg" fill="currentcolor"><path d="M16 .396c-8.839.0-16 7.167-16 16 0 7.073 4.584 13.068 10.937 15.183.803.151 1.093-.344 1.093-.772.0-.38-.009-1.385-.015-2.719-4.453.964-5.391-2.151-5.391-2.151-.729-1.844-1.781-2.339-1.781-2.339-1.448-.989.115-.968.115-.968 1.604.109 2.448 1.645 2.448 1.645 1.427 2.448 3.744 1.74 4.661 1.328.14-1.031.557-1.74 1.011-2.135-3.552-.401-7.287-1.776-7.287-7.907.0-1.751.62-3.177 1.645-4.297-.177-.401-.719-2.031.141-4.235.0.0 1.339-.427 4.4 1.641 1.281-.355 2.641-.532 4-.541 1.36.009 2.719.187 4 .541 3.043-2.068 4.381-1.641 4.381-1.641.859 2.204.317 3.833.161 4.235 1.015 1.12 1.635 2.547 1.635 4.297.0 6.145-3.74 7.5-7.296 7.891.556.479 1.077 1.464 1.077 2.959.0 2.14-.02 3.864-.02 4.385.0.416.28.916 1.104.755 6.4-2.093 10.979-8.093 10.979-15.156.0-8.833-7.161-16-16-16z"/></svg></span>编辑此页面</a></div></div></div><div><hr class=doc-hr><div id=doc-nav class=d-print-none><div class="row flex-xl-nowrap"><div class="col-sm-6 pt-2 doc-next"><a href=/docs/development/custom-models/chatglm2/><div class="card h-100 my-1"><div class="card-body py-2"><p class="card-title fs-5 fw-semibold lh-base mb-0"><i class="material-icons align-middle">navigate_before</i> 接入 ChatGLM2-6B</p><p class="card-text ms-2">将 FastGPT 接入私有化模型 ChatGLM2-6B</p></div></div></a></div><div class="col-sm-6 pt-2 doc-prev"><a class=ms-auto href=/docs/development/custom-models/m3e/><div class="card h-100 my-1 text-end"><div class="card-body py-2"><p class="card-title fs-5 fw-semibold lh-base mb-0">接入 M3E 向量模型 <i class="material-icons align-middle">navigate_next</i></p><p class="card-text me-2">将 FastGPT 接入私有化模型 M3E</p></div></div></a></div></div></div></div></div></div></div></div><footer class="shadow py-3 d-print-none"><div class="row align-items-center" style=height:90px><div class=col><div class="text-sm-start text-center mx-md-2"><p class=mb-0>© 2024 the FastGPT Authors.</p><p class=github-badge><span class=badge-subject>云操作系统</span><span class="badge-value bg-blue"><a style=color:#fff href=https://sealos.io/ target=_blank>Sealos</a></span>
 <span class=badge-subject>云开发</span><span class="badge-value bg-brightgreen"><a style=color:#fff href=https://laf.run target=_blank>Laf</a></span>
 <span class=badge-subject>云原生存储</span><span class="badge-value bg-orange"><a style=color:#fff href=https://github.com/labring/sealfs target=_blank>Sealfs</a></span></p><div id=fixed-box><div class=feedback-btn-wrapper><a href=https://fael3z0zfze.feishu.cn/share/base/form/shrcnRxj3utrzjywsom96Px4sud style=text-decoration:none target=_blank><button id=feedback-btn title="Give feedback"><svg class="inline w-5 h-5" viewBox="0 0 20 20" fill="currentcolor"><path fill-rule="evenodd" d="M18 13V5a2 2 0 00-2-2H4A2 2 0 002 5v8a2 2 0 002 2h3l3 3 3-3h3a2 2 0 002-2zM5 7a1 1 0 011-1h8a1 1 0 110 2H6A1 1 0 015 7zm1 3a1 1 0 100 2h3a1 1 0 100-2H6z" clip-rule="evenodd"/></svg>