-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
rebuild specially for pepper project
Signed-off-by: cbh778899 <[email protected]>
- Loading branch information
Showing
35 changed files
with
1,088 additions
and
3,703 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,16 +1,2 @@ | ||
| APP_PORT=8000 | ||
| APP_EXPOSE_PORT=8000 | ||
| ENG_ACCESS_PORT=8080 | ||
| MODEL_SAVE_PATH=volumes/models | ||
| DATABASE_BIND_PATH=./lancedb | ||
| INFERENCE_ENG=llamacpp | ||
| INFERENCE_ENG_PORT=8080 | ||
| INFERENCE_ENG_VERSION=server--b1-27d4b7c | ||
| NUM_CPU_CORES=8.00 | ||
| NUM_THREADS_COUNT=8.00 | ||
| EMBEDDING_ENG=embedding_eng | ||
| EMBEDDING_ENG_PORT=8081 | ||
| NUM_CPU_CORES_EMBEDDING=4.00 | ||
| NUM_THREAD_COUNTS_EMBEDDING=4.00 | ||
| LANGUAGE_MODEL_NAME=Phi3-mini-4k-instruct-Q4.gguf | ||
| EMBEDDING_MODEL_NAME=all-MiniLM-L6-v2-Q4_K_M-v2.gguf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,104 +1,11 @@ | ||
| # VOYAGER | ||
| The project is OpenAI-like API service of SkywardAI ecosystem. | ||
|
|
||
| [](https://github.com/SkywardAI/voyager/actions/workflows/linter-builder-checker.yml) | ||
| [](https://github.com/SkywardAI/voyager/actions/workflows/release-drafter.yml) | ||
| [](https://github.com/SkywardAI/voyager/actions/workflows/release-image.yml) | ||
| Voyager Specially build for RMIT Race Hub Pepper Robot project | ||
|
|
||
| ## BUILD & RUN | ||
|
|
||
| ### Setup and API usage demo video | ||
|
|
||
| https://github.com/user-attachments/assets/fa7059c2-309e-486a-a28a-45867613c84b | ||
|
|
||
| ### CLI 💥 | ||
| Introducing our new CLI tool! | ||
| > Make sure you can normally run `make`, `docker compose`, `gcc`, `sh` in your host machine. | ||
| Simply run `make setup` in the root folder to compile & run the CLI tool. | ||
|
|
||
| Don't want to set? Directly go to `Save & Build` menu in it and use `Build and start the server` option to load the app in default settings. | ||
|
|
||
| No gcc compiler? You can choose to compile the file `/setup/setup.c` yourself. | ||
|
|
||
| Explore it yourself to find more settings! | ||
|
|
||
|
|
||
| ### Local Machine | ||
| * Please make sure you installed `Node.js` on your local machine. | ||
| * This project developed on Node Version `v20.15.0`. | ||
|
|
||
| ```shell | ||
| # Manage package by pnpm | ||
| # Install pnpm globally, or change it to your local machine location | ||
| npm install -g pnpm | ||
|
|
||
| # Install dependencies | ||
| pnpm install | ||
| # OR | ||
| npm install | ||
|
|
||
| # RUN | ||
| npm run | ||
| ``` | ||
|
|
||
| ### Container | ||
| * Please make sure you have `docker` and `make` installed in your server. | ||
| * Docker version for testing is `27.0.3, build 7d4bcd8`. | ||
| ```shell | ||
| # to simply start with all needed containers started, please run | ||
| Simply run | ||
| ```sh | ||
| make up | ||
| # if you just want to build this project to docker container, please run | ||
| make build | ||
| # if you want to start only this project in docker, please run | ||
| make start | ||
| # PLEASE NOTE: make start will automatically run make build first | ||
|
|
||
| # to run a container bind to your local machine volume, run | ||
| # OR | ||
| make dev | ||
| # this will do the same thing as `make up` but allows you to make changes and sync with container | ||
| ``` | ||
| **NOTE:** `make dev` Requires Node.js environment installed, or at least have `node_modules` specified in `package.json` installed on your server. Please see [Local Machine](#local-machine) section. | ||
|
|
||
| ## Lint | ||
| To start lint your code, simply run | ||
| ```shell | ||
| npm run lint | ||
| ``` | ||
|
|
||
| ## APIs | ||
|
|
||
| ### Docs | ||
| Go to the url of your project, default [http://localhost:8000](http://localhost:8000) if you didn't disabled the `Docs` route, then you can see docs and try it on. | ||
| See [demo video](#setup-and-api-usage-demo-video). | ||
|
|
||
| ### Monitor | ||
| This project got monitor build with swagger-stats, when you got this project running, just go to `<Your Server>:<Your Port>/stats`. | ||
| For example, [http://localhost:8000/stats](http://localhost:8000/stats) | ||
|
|
||
| ### Chatbox | ||
| > When you set up the project and didn't disabled the `chatbox` API, you can get a quick-setup chatbot with some basic styles on your own website, which calls the `/v1/chat/completions` API for inference. | ||
| To set it up, simply add | ||
| ```html | ||
| <script src='http://localhost:8000/chatbox' defer></script> | ||
| ``` | ||
| into the bottom of your html body element. So easy! | ||
|
|
||
| If you want to hide the real link, in your javascript code you can do | ||
| ```js | ||
| const chatbox_script = await (await fetch("http://localhost:8000/chatbox")).blob(); | ||
| const chatbox_url = URL.createObjectURL(chatbox_script); | ||
| const script_elem = document.createElement('script'); | ||
| script_elem.src = chatbox_url; | ||
| document.body.append(script_elem); | ||
| ``` | ||
| And remember to use `URL.revokeObjectURL(chatbox_url)` if you don't need it anymore. | ||
|
|
||
| Extra parameters ([request query](https://en.wikipedia.org/wiki/Query_string)) you can add to it are: | ||
| * `base_url`: `String` | ||
| > Add this when in production, otherwise the requests won't send to correct route. | ||
| > Default `http://localhost:8000`. | ||
| * `max_tokens`: `Integer` | ||
| > Add this when you want to limit tokens can be generated, is useful in production. | ||
| > Default `128` | ||
| To build docker containers |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.