Documenting the env variables namespaces feature

modified: docs/cloud/reference/manifest.mdx modified: docs/cloud/resources/env-variables.md
subsquid · Dec 6, 2023 · 6881b77 · 6881b77
1 parent 7c7dfc0
commit 6881b77
Show file tree

Hide file tree

Showing 2 changed files with 74 additions and 46 deletions.
diff --git a/docs/cloud/reference/manifest.mdx b/docs/cloud/reference/manifest.mdx
@@ -44,10 +44,6 @@ Even though the squid services (`api`, `processor`, `migrate`) use the same sing
 
 The `deploy` section may define:
 
-### `secrets:`
-
-A list of secret names that must be added in the [Cloud app](https://app.subsquid.io). If any of the secrets is not defined, the deployment will fail.
-
 ### `addons:`
 
 A list of add-on services to be deployed along the squid services.
@@ -103,6 +99,14 @@ For the multiprocessor case:
 | `cmd`       | GraphQL API server exec command passed to the squid container. The server must be listening at port `GRAPHQL_SERVER_PORT`. |  `string[]` |    |   **Required**     |
 | `env`       | A key-value list of env variables to be set                             |  `{k:v}[]`    |  []                   |   Optional         |
 
+### `env:`
+
+**Optional** A key-value list of deployment-wide (i.e. visible to all services of the squid) env variables to be set
+
+### (deprecated) `secrets:` {#secrets}
+
+A list of secret names that must be added in the [Cloud app](https://app.subsquid.io). If any of the secrets is not defined, the deployment will fail.
+
 ## `scale:`
 
 See the [Scale the deployment](/cloud/reference/scale) section.

diff --git a/docs/cloud/resources/env-variables.md b/docs/cloud/resources/env-variables.md
@@ -2,66 +2,90 @@
 sidebar_position: 20
 title: Environment variables
 description: |- 
-  Add secrets and custom env variables
+  Variables, namespaces, secrets
 ---
 
-# Environment variables
+:::info
+Store all your sensitive inputs (API keys, passwords etc) as [secrets](#secrets).
+:::
+
+Subsquid Cloud supports adding environment variables to squid deployments. The variables can be defined as key-value pairs at any of the `env:` sections of the [manifest](/cloud/reference/manifest).
+
+For example, here is how to add a variable that will be visible only to the [squid processor](/cloud/reference/manifest/#processor):
+```yaml title="squid.yaml"
+deploy:
+  processor:
+    env:
+      MY_PROCESSOR_VAR: string_value
+```
+You can also add variables visible only to the [GraphQL server](/cloud/reference/manifest/#api) or to the [migration script](/cloud/reference/manifest/#migrate).
+
+There is also an option to add variables [for all services](/cloud/reference/manifest/#env):
+```yaml title="squid.yaml"
+deploy:
+  env:
+    MY_SQUIDWIDE_VAR: string_value
+```
+
+Variables can be assigned either to strings, or to member variables of [namespaces](#namespaces) provided by the service. For example, to make a processor-scoped `API_KEY` variable and populate it with the value of `secrets.API_KEY`, do this:
+```yaml title="squid.yaml"
+deploy:
+  processor:
+    env:
+      RPC_ENDPOINT: ${{ secrets.API_KEY }}
+```
 
-Subsquid Cloud supports adding environment variables to the squid deployments. There are two kinds: **secrets** and **environment variables**. The crucial difference is that the secrets are injected to all squids, while environments variables are set only to a specific version deployment. Environment variables are stored in plain text so **all the sensitive input (e.g. API keys) must be set as a secret**.
+## Database variables shadowing
 
-Secrets are set using the special command [`sqd secrets`](/squid-cli/secrets), while environment variables are set in the [deployment manifest](/cloud/reference/manifest).
+There is one special case in which the variables defined in the manifest will get overwritten by the Cloud: [database connection settings](/sdk/reference/store/typeorm/#database-connection-parameters) are shadowed by the system-defined values whenever the [`postgres` addon](/cloud/reference/pg) is enabled. For example, in the snippet below all the `DB_*` variable definitions will be ignored:
+```yaml title="squid.yaml"
+deploy:
+  addons:
+    postgres:
+  env:
+    DB_HOST: mypostgreshost.xyz
+    DB_PORT: 5432
+    DB_NAME: squid-tests
+    DB_USER: me
+    DB_PASS: ${{ secrets.DATABASE_PASSWORD }}
+    DB_SSL: true
+```
 
-## Secrets 
+## Namespaces
 
-Secrets are designed to store sensitive data that all the squids can access as an environment variable with the given name. A typical usage is an API key or a private URL for a chain gRPC endpoint. The secrets are defined at the Cloud [organization](/cloud/resources/organizations) level and are exposed to all organization squids that [request them in their manifest](/cloud/reference/manifest/#secrets).
+### Secrets
+
+Secrets are designed to store sensitive data, such as API keys or private URLs for chain RPC endpoints. They are defined at the [organization](/cloud/resources/organizations) level and are exposed to all organization squids that request them in their environment variable definitions.
 
 To add a secret:
-1. Create it in the Cloud:
+
+1. Create it in the Cloud. You can do it at the [secrets page](https://app.subsquid.io/secrets) or with [`sqd secrets`](/squid-cli/secrets):
    ```bash
    sqd secrets set MOONRIVER_GRPC_ENDPOINT wss://moonriver.my-endpoint.com/ws/my-secret-key
    ```
-2. Add it to the [manifest](/cloud/reference/manifest):
+2. At your squid's [manifest](/cloud/reference/manifest), add an environment variable and assign it to the secret:
    ```yaml
    deploy:
-     secrets:
-       - MOONRIVER_GRPC_ENDPOINT
+     env:
+       RPC_ENDPOINT: ${{ secrets.MOONRIVER_GRPC_ENDPOINT }}
    ```
-   Note: a deployment requesting a secret unknown to the Cloud will fail.
-3. Access in the squid with `process.env`:
+   Note: **a deployment requesting a secret unknown to the Cloud will fail**.
+3. Access the value in the squid with `process.env`, e.g.
    ```ts
    const processor = new EvmBatchProcessor()
-       .setDataSource({
-           archive: lookupArchive("moonriver", { type: 'EVM' }),
-           chain: {
-             url: process.env.MOONRIVER_GRPC_ENDPOINT,
-             rateLimit: 1000rps
-           }
-       })
+     .setDataSource({
+       archive: lookupArchive('moonriver', { type: 'EVM' }),
+       chain: {
+         url: process.env.RPC_ENDPOINT,
+         rateLimit: 1000rps
+       }
+     })
    ```
-
 Inspect, remove and update the secrets using the [`sqd secrets`](/squid-cli/secrets) command.
 
 :::info
-Any changes to secrets will take effect only when the squid is restarted with [`sqd deploy`](/squid-cli/deploy).
-:::
-
-## Environment variables
-
-Squid-specific environment variables should be defined in the [deployment manifest](/cloud/reference/manifest).
-
-**Example**
-
-```yaml title="squid.yaml"
-# ...
-deploy:
-  # ...
-  processor:
-    env:
-      SQD_DEBUG: 
-        sqd:mapping
-    cmd: [ "sqd", "process:prod" ]
-# ....
+Any changes to secrets will take effect only when the squid is restarted, e.g. with
+```bash
+sqd deploy .
 ```
-
-
-
+:::