Skip to content

Commit

Permalink
0.11.0: Added optional password authentication
Browse files Browse the repository at this point in the history
  • Loading branch information
flavienbwk committed Sep 8, 2024
1 parent de3859b commit ba464c2
Show file tree
Hide file tree
Showing 16 changed files with 315 additions and 66 deletions.
1 change: 1 addition & 0 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ REPO_NAME=MyRepo
REPO_URL=https://github.com/user/MyRepo.git
REPO_PATH=./examples
INGEST_SECRET=9640f515-fd2c-4403-845f-71b25fc9086c
INTERFACE_PASSWORD=f4bccbf2-cfdc-452d-b39a-fea61b1687fe # leave empty for no password on interface
OPENAI_API_KEY=XXX
MODEL_TYPE_INFERENCE=gpt-4o-mini
MODEL_TYPE_EMBEDDING=text-embedding-3-small
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@ jobs:
name: 'Deploy Repochat'
id: deploy_repochat
with:
dirs_to_scan: "README.md,package.json,Dockerfile,api,action,app" # comma-separated glob dirs to analyze from this repo
dirs_to_scan: "README.md,package.json,Dockerfile,api,action,app"
interface_password: ${{ secrets.INTERFACE_PASSWORD }}
openai_api_key: ${{ secrets.OPENAI_API_KEY }}
openai_model_type_inference: "gpt-4o-mini"
openai_model_type_embedding : "text-embedding-3-small"
cloud_provider: 'scaleway'
provider_name: 'scaleway'
provider_key_id: ${{ secrets.PROVIDER_KEY_ID }}
provider_key_secret: ${{ secrets.PROVIDER_KEY_SECRET }}
provider_project_id: ${{ secrets.PROVIDER_PROJECT_ID }}
Expand Down
77 changes: 54 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@
[![GitHub release](https://img.shields.io/github/v/release/flavienbwk/repochat-action)](https://github.com/flavienbwk/repochat-action/releases/latest)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

> [!CAUTION]
> This project is currently NOT ready.
> This project is in early development and may contain bugs or incomplete features. Use with caution in production environments.
> [!WARNING]
> This project is in early development and may contain bugs. Use with caution in production environments.
Chat with your repo in under 2 minutes using GitHub Actions on supported Cloud providers.
Chat with your repo in under 2 minutes using GitHub Actions on [supported Cloud providers](#supported-cloud-providers).

Repochat is a LLM chatbot with stateless data ingestion capabilities through its API.
Features:

- :sparkles: A nice web UI
- :octocat: Available as a GitHub Action
- :floppy_disk: Stateful ingestion mode ([directory mode](./dir.docker-compose.yml#L6))
- :arrows_counterclockwise: Sateless ingestion via API ([api mode](./api.docker-compose.yml#L6))
- :closed_lock_with_key: Optional password on web UI

## Requirements

Expand All @@ -19,26 +24,29 @@ Repochat is a LLM chatbot with stateless data ingestion capabilities through its

## Usage for GitHub Actions

Easily add RepoChat to your project using GitHub Actions:

```yaml
jobs:
steps:
- name: Deploy a convenient chatbot for your repo
id: deploy_repochat
uses: flavienbwk/repochat-action@v0
if: github.ref == 'refs/heads/main'
with:
dirs_to_scan: "./example,README.md" # comma-separated glob dirs to analyze from this repo (required)
openai_api_key: ${{ secrets.OPENAI_API_KEY }} # (required)
openai_model_type_inference: "gpt-4o-mini" # (required)
openai_model_type_embedding : "text-embedding-3-small" # (required)
cloud_provider: 'scaleway' # (required)
# All parameters not explicitly marked as "optional" are required
dirs_to_scan: "./example,README.md" # comma-separated glob dirs to analyze
interface_password: ${{ secrets.INTERFACE_PASSWORD }} # optional
openai_api_key: ${{ secrets.OPENAI_API_KEY }}
openai_model_type_inference: "gpt-4o-mini"
openai_model_type_embedding : "text-embedding-3-small"
provider_name: 'scaleway'
provider_key_id: ${{ secrets.PROVIDER_KEY_ID }}
provider_key_secret: ${{ secrets.PROVIDER_KEY_SECRET }}
provider_project_id: ${{ secrets.PROVIDER_PROJECT_ID }}
provider_default_region: 'fr-par'
provider_default_zone: 'fr-par-2'

- name: Get container domain from precedent step
- name: Get RepoChat domain
run: echo "DOMAIN=${{ steps.deploy_repochat.outputs.domain }}" >> $GITHUB_OUTPUT
id: repochat_domain
```
Expand All @@ -48,12 +56,15 @@ jobs:
- **[Scaleway](https://www.scaleway.com/en/)**
- Refer to [Scaleway's documentation to generate API keys](https://www.scaleway.com/en/docs/identity-and-access-management/iam/how-to/create-api-keys/).
- Additional required parameters:
- `cloud_provider`: 'scaleway'
- `provider_key_id`: ${{ secrets.PROVIDER_KEY_ID }}
- `provider_key_secret`: ${{ secrets.PROVIDER_KEY_SECRET }}
- `provider_project_id`: ${{ secrets.PROVIDER_PROJECT_ID }}
- `provider_default_region`: 'fr-par' # example
- `provider_default_zone`: 'fr-par-2' # example
```txt
provider_name: 'scaleway'
provider_key_id: ${{ secrets.PROVIDER_KEY_ID }}
provider_key_secret: ${{ secrets.PROVIDER_KEY_SECRET }}
provider_project_id: ${{ secrets.PROVIDER_PROJECT_ID }}
provider_default_region: 'fr-par' # example
provider_default_zone: 'fr-par-2' # example
```
## Other deployments
Expand Down Expand Up @@ -105,9 +116,28 @@ jobs:

## Development

```bash
make dev
```
<details>
<summary>👉 Run RepoChat for development...</summary>

1. Clone this repo

```bash
[email protected]:flavienbwk/repochat-action.git
```

2. Copy and update env variables

```bash
cp .env.example .env
```

3. Run the local stack

```bash
make dev
```

</details>

## Release Action

Expand All @@ -130,8 +160,9 @@ make dev
- [x] API data chat
- [x] GitHub Actions release
- [x] Secure ingestion endpoint (/api/ingest)
- [ ] Secure RepoChat with optional password
- [x] Secure RepoChat with optional password
- [ ] Maintain state over S3

## Why not use Vercel ?

Vercel is very limited when deploying everything but JS. First, ChromaDB (and any sqlite-based library) [is not supported in Vercel](https://vercel.community/t/is-vercel-incompatible-with-chromadb-sqlite/787). Then, this project uses a FastAPI Python API that requires more storage than [Vercel's 250MB bundle limit](https://vercel.com/docs/functions/runtimes#bundle-size-limits).
Vercel is very limited when it comes to deploying everything but JS. First, ChromaDB (and any sqlite-based library) [is not supported in Vercel](https://vercel.community/t/is-vercel-incompatible-with-chromadb-sqlite/787). Then, this project uses a FastAPI Python API that requires more storage than [Vercel's 250MB bundle limit](https://vercel.com/docs/functions/runtimes#bundle-size-limits).
6 changes: 5 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ inputs:
dirs_to_scan:
description: 'Comma-separated list of directories that you want to chat about'
required: true
interface_password:
description: 'Password to access the interface'
required: false
default: ''
openai_api_key:
description: 'Your OpenAI API key'
required: true
Expand All @@ -16,7 +20,7 @@ inputs:
description: 'OpenAI model type for embedding'
default: 'text-embedding-3-small'
required: false
cloud_provider:
provider_name:
description: 'Cloud provider'
default: 'scaleway'
required: false
Expand Down
16 changes: 9 additions & 7 deletions action/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,11 @@ async function ingestFiles(directoryPath, apiUrl, ingestSecret, excludeFiles = [

try {
const dirsToScan = core.getInput('dirs_to_scan');
const interfacePassword = core.getInput('interface_password');
const openaiApiKey = core.getInput('openai_api_key');
const openaiModelTypeInference = core.getInput('openai_model_type_inference');
const openaiModelTypeEmbedding = core.getInput('openai_model_type_embedding');
const cloudProvider = core.getInput('cloud_provider');
const cloudProvider = core.getInput('provider_name');
const providerKeyId = core.getInput('provider_key_id');
const providerKeySecret = core.getInput('provider_key_secret');
const providerProjectId = core.getInput('provider_project_id');
Expand Down Expand Up @@ -188,8 +189,12 @@ try {
console.log('Namespace is ready');


const openaiApiKeySecret = { key: 'OPENAI_API_KEY', value: openaiApiKey }
const parIngestSecretSecret = { key: 'INGEST_SECRET', value: parIngestSecret }
let listOfSecrets = [];
listOfSecrets.push({ key: 'OPENAI_API_KEY', value: openaiApiKey })
listOfSecrets.push({ key: 'INGEST_SECRET', value: parIngestSecret })
if (interfacePassword) {
listOfSecrets.push({ key: 'INTERFACE_PASSWORD', value: interfacePassword })
}
const containerConfig = {
name: containerName,
namespaceId: namespace.id,
Expand All @@ -207,10 +212,7 @@ try {
REPO_URL: `https://github.com/${process.env.GITHUB_REPOSITORY}`,
MODE: 'api'
},
secretEnvironmentVariables: [
openaiApiKeySecret,
parIngestSecretSecret
]
secretEnvironmentVariables: listOfSecrets
};

try {
Expand Down
3 changes: 2 additions & 1 deletion api.docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@ services:
repochat:
build: .
environment:
MODE: api
REPO_NAME: ${REPO_NAME}
REPO_URL: ${REPO_URL}
OPENAI_API_KEY: ${OPENAI_API_KEY}
INGEST_SECRET: ${INGEST_SECRET}
MODEL_TYPE_INFERENCE: ${MODEL_TYPE_INFERENCE}
MODEL_TYPE_EMBEDDING: ${MODEL_TYPE_EMBEDDING}
CLEAR_DB_AT_RESTART: ${CLEAR_DB_AT_RESTART}
MODE: api
INTERFACE_PASSWORD: ${INTERFACE_PASSWORD}
ports:
- "3001:80"
1 change: 1 addition & 0 deletions api/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
REPO_URL = os.getenv('REPO_URL', '').strip()
REPO_PATH = os.getenv('REPO_PATH', '').strip()
INGEST_SECRET = os.environ['INGEST_SECRET'].strip()
INTERFACE_PASSWORD = os.getenv('INTERFACE_PASSWORD', '').strip()
OPEN_AI_API_KEY = os.environ['OPENAI_API_KEY'].strip()
MODEL_TYPE_INFERENCE = os.environ['MODEL_TYPE_INFERENCE'].strip()
MODEL_TYPE_EMBEDDING = os.environ['MODEL_TYPE_EMBEDDING'].strip()
Expand Down
Loading

0 comments on commit ba464c2

Please sign in to comment.