transition from serverless framework to terraform (#194)

.github/workflows/ci.yaml

@@ -0,0 +1,86 @@
+name: deploy-probot-terraform
+
+on:
+  pull_request:
+  workflow_dispatch:
+  push:
+    branches:
+      - "pull-request/[0-9]+"
+      - "main"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: false
+
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  deploy:
+    name: Deploy Probot Application
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Get AWS credentials
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ vars.SERVERLESS_AWS_ROLE_ARN }}
+          aws-region: ${{ vars.AWS_REGION }}
+
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '18'
+
+      - name: Install npm dependencies
+        run: npm ci
+
+      - name: Test Probot
+        run: npm run test
+
+      - name: Build Probot
+        run: npm run build
+
+      - name: Copy release draft template
+        run: cp src/plugins/ReleaseDrafter/draft_template.njk dist/plugins/ReleaseDrafter
+
+      - name: Package Lambda functions
+        run: |
+          zip -r probot.zip .
+          zip -r authorizer.zip . -x "probot.zip"
+
+      - name: Setup Terraform
+        uses: hashicorp/setup-terraform@v3
+        with:
+          terraform_version: "1.9.2"
+
+      - name: Terraform Format Check
+        working-directory: terraform
+        run: terraform fmt -check
+
+      - name: Terraform Init
+        working-directory: terraform
+        run: terraform init
+
+      - name: Terraform Validate
+        working-directory: terraform
+        run: terraform validate
+
+      - name: Terraform Plan
+        id: plan
+        working-directory: terraform
+        run: terraform plan -out tfplan
+        env:
+          TF_VAR_app_id: ${{ secrets.APP_ID }}
+          TF_VAR_webhook_secret: ${{ secrets.WEBHOOK_SECRET }}
+          TF_VAR_private_key: ${{ secrets.PRIVATE_KEY }}
+          TF_VAR_gputester_pat: ${{ secrets.GPUTESTER_PAT }}
+
+      - name: Terraform Apply
+        if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
+        working-directory: terraform
+        run: terraform apply -auto-approve tfplan

.gitignore

@@ -6,3 +6,9 @@ npm-debug.log
 coverage
 dist
 .serverless
+.terraform/
+*.tfstate
+*.tfstate.*
+*.tfplan
+.terraform.lock.hcl
+*.tfvars

CONTRIBUTING.md

@@ -0,0 +1,33 @@
+# Contributing
+
+Any new functionality should be introduced as a new plugin in the [src/plugins](./src/plugins) directory. New plugins should make use of the shared `featureIsDisabled` function so that repositories can disable the feature if they desire. New plugins should also have an entry added in [config.ts](./src/config.ts)
+
+## Making Infrastructure Changes
+
+The project uses Terraform to manage AWS infrastructure. The configuration files are located in the `terraform/` directory.
+
+### Structure
+
+- `main.tf`: Provider configuration and backend setup
+- `lambda.tf`: Lambda function definitions
+- `iam.tf`: IAM roles and policies
+- `api_gateway.tf`: API Gateway configuration
+- `cloudwatch.tf`: CloudWatch log groups
+- `variables.tf`: Input variables
+- `outputs.tf`: Output values
+
+### Testing Changes
+
+1. Make your changes to the Terraform files
+2. Run `terraform fmt` to ensure consistent formatting
+3. Run `terraform validate` to check for configuration errors
+4. Create a PR - the GitHub Actions workflow will automatically:
+   - Check formatting
+   - Validate configuration
+   - Generate and post a plan to the PR
+
+### Deployment
+
+Infrastructure changes are automatically deployed when merged to `main`. The deployment:
+- Packages and uploads Lambda functions to S3
+- Applies Terraform changes with the new configuration

README.md

@@ -14,9 +14,55 @@ The plugins are listed in the [src/plugins](./src/plugins) folder.
 - **Branch Checker** - Set a status on PRs that checks whether they are targeting either the repo's _default branch_ or _default branch + 1_
 - **Recently Updated** - Sets a status on PRs based on whether a PR is `X` commits out-of-date compared to the based branch. `X` defaults to `5`, but is configurable via the `recently_updated_threshold` option in the `.github/ops-bot.yaml` configuration file.
 
-## Deployment
+## Infrastructure
 
-The _Serverless_ framework is used to deploy the Probot application to an AWS Lambda instance. The deployment configuration can be seen in the [serverless.yaml](./serverless.yaml) file. A deployment will happen automatically anytime a change is merged to the `main` branch affecting any of the following files: source code files, `package.json` file, or `serverless.yaml` file. See the [deploy.yaml](/.github/workflows/deploy.yaml) GitHub Action for more details.
+The project's infrastructure is managed using Terraform. Key components include:
+
+- AWS Lambda functions for the Probot handler and authorizer
+- API Gateway with custom authorizer
+- IAM roles and policies
+- CloudWatch log groups
+- S3 bucket for deployment artifacts
+
+### Prerequisites
+
+- Terraform v1.9.2 or later
+- AWS CLI configured with appropriate credentials
+- Node.js 18.x
+
+### Deployment
+
+The deployment is automated via GitHub Actions. For manual deployment:
+
+1. Build the application:
+```bash
+npm install
+npm run build
+```
+2. Package Lambda functions:
+```bash
+zip -r probot-{version}.zip dist
+zip -r authorizer-{version}.zip dist/authorizer.js
+```
+3. Upload to S3:
+```bash
+aws s3 cp probot-{version}.zip s3://rapidsai-serverless-deployments/serverless/ops-bot/prod/
+aws s3 cp authorizer-{version}.zip s3://rapidsai-serverless-deployments/serverless/ops-bot/prod/
+```
+4. Deploy infrastructure:
+```bash
+cd terraform
+terraform init
+terraform plan
+terraform apply
+```
+
+### Required Environment Variables
+
+- `APP_ID`: GitHub App ID
+- `WEBHOOK_SECRET`: GitHub Webhook Secret
+- `PRIVATE_KEY`: GitHub App Private Key
+- `GPUTESTER_PAT`: GPU Tester Personal Access Token
 
 ## npm Scripts
 
@@ -30,7 +76,3 @@ npm run test
 # Deploy
 npm run deploy
 ```
-
-## Contributing
-
-Any new functionality should be introduced as a new plugin in the [src/plugins](./src/plugins) directory. New plugins should make use of the shared `featureIsDisabled` function so that repositories can disable the feature if they desire. New plugins should also have an entry added in [config.ts](./src/config.ts)

terraform/api_gateway.tf

@@ -0,0 +1,43 @@
+resource "aws_api_gateway_rest_api" "ops_bot" {
+  name = "ops-bot"
+}
+
+resource "aws_api_gateway_method" "root_post" {
+  rest_api_id = aws_api_gateway_rest_api.ops_bot.id
+  # resource_id   = aws_api_gateway_resource.proxy.id
+  resource_id   = aws_api_gateway_rest_api.ops_bot.root_resource_id
+  http_method   = "POST"
+  authorization = "NONE"
+}
+
+resource "aws_api_gateway_integration" "lambda" {
+  rest_api_id = aws_api_gateway_rest_api.ops_bot.id
+  resource_id = aws_api_gateway_rest_api.ops_bot.root_resource_id
+  http_method = aws_api_gateway_method.root_post.http_method
+  credentials = aws_iam_role.api_gateway_authorizer.arn
+
+  integration_http_method = "POST"
+  type                    = "AWS_PROXY"
+  uri                     = aws_lambda_function.authorizer.invoke_arn
+}
+
+resource "aws_api_gateway_deployment" "ops_bot" {
+  rest_api_id = aws_api_gateway_rest_api.ops_bot.id
+  triggers = {
+    redeployment = sha1(jsonencode([
+      aws_api_gateway_rest_api.ops_bot.root_resource_id,
+      aws_api_gateway_method.root_post.id,
+      aws_api_gateway_integration.lambda.id,
+    ]))
+  }
+
+  lifecycle {
+    create_before_destroy = true
+  }
+}
+
+resource "aws_api_gateway_stage" "ops_bot" {
+  deployment_id = aws_api_gateway_deployment.ops_bot.id
+  rest_api_id   = aws_api_gateway_rest_api.ops_bot.id
+  stage_name    = "prod"
+}

terraform/cloudwatch.tf

@@ -0,0 +1,9 @@
+resource "aws_cloudwatch_log_group" "probot_handler" {
+  name              = "/aws/lambda/ops-bot-handleProbot"
+  retention_in_days = 60
+}
+
+resource "aws_cloudwatch_log_group" "authorizer" {
+  name              = "/aws/lambda/ops-bot-authorizerFn"
+  retention_in_days = 60
+}