diff --git a/CHANGELOG.md b/CHANGELOG.md index 10f0199..5bda3ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,21 @@ -# Changelog +## [20200424](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) + +### Added + +- [Swagger `security` requirements](/reference/OpenAPI/security.md) can now also be specified at the path level. +- [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies) to configure proxies without a flow +- Enhanced [`proxy-request` action](/reference/actions/proxy-request.md) with `origin`, `query`, `stripEndpoint` and `addPrefix` properties + +### Fixed + +- If a client URL path is below the API base path, does not match any defined route, and a path is defined which equals the API base path, so that a matching client URL path is the concatenation of the API base path with itself (e.g. `/api/api` if the `basePath` is `/api`), the [fallback flow](/reference/OpenAPI/routing.md#fallback-flow) is now properly executed. +- Some PEM formatted keys could not be recognized during [JWT processing](/cookbook/jwt.md). +- Multi-line values for [environment variables](/cookbook/envvars.md) are now supported. + +### Changed + +- If the `definition` [request option](/reference/actions/request.md#options) is given with either a [`proxy-request` action](/reference/actions/proxy-request.md) or [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies), the defaults for the `exit-on-error`, `validate-request` and `validate-response` request options are changed to `true`. + ## [20200409](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) diff --git a/cookbook/README.md b/cookbook/README.md index e081160..a3da9ac 100644 --- a/cookbook/README.md +++ b/cookbook/README.md @@ -4,7 +4,9 @@ ## [How can I pass an arbitrary header field to an upstream system?](pass-header-field-upstream.md) -## [Forwarding a Request to an Upstream API](forward-request-upstream.md) +## [Proxying Requests to Upstream APIs (Swagger)](proxy-requests.md) + +## [Forwarding a Request to an Upstream API (Flow)](forward-request-upstream.md) ## [How can I pass response headers to the client?](pass-header-field-downstream.md) diff --git a/cookbook/forward-request-upstream.md b/cookbook/forward-request-upstream.md index cb422ef..f2fb2cb 100644 --- a/cookbook/forward-request-upstream.md +++ b/cookbook/forward-request-upstream.md @@ -22,8 +22,8 @@ The following [flow](/reference/flow.md) shows a more advanced example: { - {{// Replace the hostname of the incoming request with the UPSTREAM_URI environment variable }} - "url": {{ replace($request/url, "^http://[^/]+", $env/UPSTREAM_URI) }}, + {{// Replace the origin of the incoming request with the UPSTREAM_ORIGIN environment variable }} + "origin": {{ $env/UPSTREAM_ORIGIN }}, "headers": { {{// Set X-tra, drop X-Remove, copy Authorization }} @@ -42,7 +42,7 @@ The following [flow](/reference/flow.md) shows a more advanced example: ``` -The `proxy-request` action lets you set the `url` and modify the `headers` of the request. +The `proxy-request` action lets you set the `origin` and modify the `headers` of the request. Everything else is set up automatically: The client request body will be forwarded as-is, any headers not intended for upstream will be dropped. @@ -103,7 +103,31 @@ status code of the upstream response and possibly additional header fields are c The forwarded request will only have a body if the incoming request does. The `Content-Type` request and response headers are passed automatically with the body, therefore we don't have to copy them explicitly. +If the client and the upstream request URL paths do not share a common prefix, you can easily adjust the path by using `stripEndpoint` and `addPrefix`: + +```json +{ + "origin": {{ $env/UPSTREAM_ORIGIN }}, + "stripEndpoint: true, + "addPrefix": {{ $env/UPSTREAM_PATH_PREFIX }}, + … +} +``` + +Assuming the following `swagger.yaml` for `https://client.example.com/` + +```yaml +basePath: /api +paths: + /users/**: + … +``` + +, `https://users.upstream.example.com` for `UPSTREAM_ORIGIN`, and `/v4` for `UPSTREAM_PATH_PREFIX`, a client request for `https://client.example.com/api/users/profile` will be forwarded to `https://users.upstream.example.com/v4/profile`. + + ## See also +* [Proxying requests to Upstream APIs](proxy-requests.md) (cookbook) * [`proxy-request` action](/reference/actions/proxy-request.md) (reference) * [`request` action](/reference/actions/request.md) (reference) diff --git a/cookbook/proxy-requests.md b/cookbook/proxy-requests.md new file mode 100644 index 0000000..b705856 --- /dev/null +++ b/cookbook/proxy-requests.md @@ -0,0 +1,54 @@ +# Proxying requests to Upstream APIs + +To proxy requests to an upstream API, you can simply use a combination of two FLAT Swagger enhancements: + +* [wildcard paths](/reference/OpenAPI/differences.md#wildcard-paths), and +* [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies). + +Imagine, the upstream API, you want to use, is located at `https://upstream.api/api/docs` and provides a route to get a document with a certain ID (`/get-doc/{docid}`). To set FLAT as a proxy to this API, the following `swagger.yaml` snippet will do the job: + +```yaml +… +basePath: /api +… +paths: + /docs/**: + x-flat-proxy: + origin: https://upstream.api # replaces the origin +… +``` + + +Assuming a client origin of `https://client.example.com`, a client request to e.g. `https://client.example.com/api/docs/get-doc/42` will be proxied to `https://upstream.api/api/docs/get-doc/42`. + +Note, that only the origin is replaced, the path is not modified. + +If the upstream API for docs is located at `https://docs.upstream.api/v4` instead of `https://upstream.api/api/docs`, the path has to be adjusted, too: + +```yaml +… +basePath: /flat +… +paths: + /docs/**: + x-flat-proxy: + origin: https://docs.upstream.api + stripEndpoint: true # strips /flat/docs from the path + addPrefix: /v4 # inserts /v4 before the stripped path +… +``` + +The client request will be proxied to `https://docs.upstream.api/v4/get-doc/42`. + +Assuming you want to plug-in a different API (`https://users.upstream/v3`), just add the following: + +```yaml +… + /users/**: + x-flat-proxy: + origin: https://users.upstream.api + stripEndpoint: true # strips /flat/users from the path + addPrefix: /v3 # inserts /v3 before the stripped path +``` + +A client request to `https://client.example.com/flat/users/profile/4711` will be proxied to `https://users.upstream.api/v3/profile/4711`. diff --git a/reference/OpenAPI/differences.md b/reference/OpenAPI/differences.md index 385ea1e..ac1add1 100644 --- a/reference/OpenAPI/differences.md +++ b/reference/OpenAPI/differences.md @@ -9,6 +9,7 @@ First of all, several extensions named `x-flat-…` are recognized on different * `x-flat-flow`: [flow](routing.md#assigning-flat-flows) to be started. Recognized at top-level and below `paths`, `paths/` and `paths//`. * `x-flat-init`: [init flow](routing.md#init-flow) (top-level) * `x-flat-error`: [error flow](routing.md#error-flow) (top-level) +* `x-flat-proxy`: [proxy configuration](routing.md#assigning-flat-proxies) (below `paths`, `paths/` and `paths//`) * `x-flat-cors`: [CORS configuration](cors.md) (top-level) * `x-flat-validate`: [validation](validation.md) (top-level, below `paths/` and `paths//`) * `x-flat-jwt`: [expected JWT](security.md#the-x-flat-jwt-field) (in a [security scheme object](https://swagger.io/specification/v2/#securitySchemeObject)) @@ -85,3 +86,7 @@ paths: The longest matching wildcard path wins. The position of a wildcard path in the definition is irrelevant. Note that path parameters (i.e. sections enclosed in curly braces) cannot be combined with wildcards. + +## Security + +[Security schemes](/reference/OpenAPI/security.md#applying-security-schemes) can also be applied to specific paths, not only to specific operations or top-level. diff --git a/reference/OpenAPI/routing.md b/reference/OpenAPI/routing.md index bced919..e3dd9c4 100644 --- a/reference/OpenAPI/routing.md +++ b/reference/OpenAPI/routing.md @@ -100,8 +100,7 @@ paths: ### Init Flow -An _init flow_ is a separate flow file that is executed before the regular [flow](/reference/flow.md) -that is defined for an API path. It is specified by setting `x-flat-init` on +An _init flow_ is a separate flow file that is executed before the regular [flow](/reference/flow.md) or [configured proxy](routing.md#assigning-flat-proxies) defined for an API path. It is specified by setting `x-flat-init` on the top level in the OpenAPI definition: ```yaml @@ -129,7 +128,7 @@ flow from being executed, too. ### Error Flow -An _error flow_ is an optional separate flow file that is executed if a client request or response validation error has occurred, or if the `exit-on-error` option was set for a [request](/reference/actions/request.md) that has failed. +An _error flow_ is an optional separate flow file that is executed if a client request or response validation error has occurred, or if the `exit-on-error` option was set for a [request](/reference/actions/request.md), [proxy-request](/reference/actions/proxy-request.md) or [configured proxy](routing.md#assigning-flat-proxies) that has failed. It is specified by setting the `flow` property of `x-flat-error` on the top level in the OpenAPI definition: ```yaml @@ -147,6 +146,39 @@ Requests to resources outside the `basePath` are handled by the default flow def in `conf/flow.xml`. This allows for [serving HTML, images, JavaScript](/cookbook/file-serving.md) and the like. +## Assigning FLAT Proxies + +If FLAT acts as a proxy for an upstream API on a specific route, you could assign a flow containing a [`proxy-request` action](/reference/actions/proxy-request.md). + +A simpler way to achieve this is by using `x-flat-proxy` in the `swagger.yaml`: + +```yaml +basePath: /api +paths: + /users/**: + x-flat-proxy: + origin: https://users.upstream.example.com + stripEndpoint: true + addPrefix: /v4 + headers: + Correlation-ID: 42 + options: + timeout: 2 + definition: users-upstream.yaml + validate-response: report-only +``` + +A client request to `https://client.example.com/api/users/profile` will be proxied to `https://users.upstream.example.com/v4/profile`. See [wildcard paths](differences.md#wildcard-paths) for more information about `/**`. + +`x-flat-proxy` can be used below `paths`, `paths/` and `paths//`. + +The configuration for `x-flat-proxy` is the same as that for a [`proxy-request` action](/reference/actions/proxy-request.md) (translated from JSON to YAML syntax). + +If configured, the [init flow](routing.md#init-flow) is executed before the proxy request. If configured, the [error flow](routing.md#error-flow) is executed if the `exit-on-error` option was set and the proxy request fails. + +`x-flat-proxy` and `x-flat-flow` are alternatives and cannot be used in combination. + + ## Path Parameters Swagger paths can define path parameters: diff --git a/reference/OpenAPI/security.md b/reference/OpenAPI/security.md index 3da8618..6238fd6 100644 --- a/reference/OpenAPI/security.md +++ b/reference/OpenAPI/security.md @@ -43,7 +43,7 @@ securityDefinitions: ``` The code in this example defines a security scheme named `JWTHeaderAuth`. The token is expected to be a bearer token in the `Authorization` header. -The key is read from a file named `secret.pem` relative to the swagger.yaml. +The key is read from a file named `secret.pem` relative to the `swagger.yaml`. The signing algorithm is read from the `FLAT_JWT_ALG` environment variable. The JWT will be stored in the `$header_token` variable. The JWT payload is expected to contain an `aud` claim with a value read from the `FLAT_JWT_AUDIENCE` environment variable. @@ -92,18 +92,24 @@ paths: ## Applying Security Schemes -In Swagger, Security Schemes can be specified at the top level (default security) or for specific operations. +In Swagger, security schemes can be specified at the top level (default security) or for specific operations. +With FLAT, you can also specify a security scheme for a specific path (default security for all operations on the path). -In the following example, a `GET` request to `foo` must satisfy the security scheme named `JWTHeaderAuth`. +In the following example, a `GET` request to `/foo` must satisfy the security scheme named `JWTHeaderAuth`, while e.g. a `POST` or `PUT` request to `/foo` must satisfy the security scheme named `JWTCookieAuth`. All other requests must satisfy either the `JWTHeaderAuth` **or** `JWTCookieAuth` security schemes. ```yaml +# default security: - JWTHeaderAuth: [] - JWTCookieAuth: [] paths: /foo: + # default for all operations on /foo + security: + - JWTCookieAuth: [] get: + # specific for GET on /foo security: - JWTHeaderAuth: [] ``` diff --git a/reference/actions/README.md b/reference/actions/README.md index 71feb69..79d84b4 100644 --- a/reference/actions/README.md +++ b/reference/actions/README.md @@ -8,6 +8,8 @@ * [`eval`](eval.md) * [`log`](log.md) * [`nameshave`](nameshave.md) +* [`pass-body`](pass-body.md) +* [`proxy-request`](proxy-request.md) * [`regex`](regex.md) * [`request`](request.md) * [`requests`](requests.md) diff --git a/reference/actions/proxy-request.md b/reference/actions/proxy-request.md index 379a7da..bd65033 100644 --- a/reference/actions/proxy-request.md +++ b/reference/actions/proxy-request.md @@ -18,20 +18,86 @@ Just like an ordinary [`request`](request.md) the `proxy-request` can be configured using a [JSON template](/reference/templating/README.md) with the following properties: +### `origin` + +Sets the origin of the upstream system. Either `origin` or `url` is required. Type: `string`. + +### `stripEndpoint` + +To be used in connection with `origin`. If `true`, strips the endpoint path from the client request URL path before it is added to the upstream origin. Type: `boolean`, default: `false` + +E.g. + +```yaml +basePath: /api +paths: + /foo/bar: + /wildcard/** +``` + +For the client request URL `https://client.example.com/api/foo/bar`, the path is stripped to `/`. + +For the client request URL `https://client.example.com/api/wildcard/path/to`, the path is stripped to `/path/to`. + +### `addPrefix` + +Inserts a path prefix before the given (client request URL) path, after possible endpoint stripping (see `stripEndpoint`). Type: `string`. + ### `url` -Sets the URL to the upstream system. Required. +Sets the URL to the upstream system. Either `url` or `origin` is required. + +### `query` + +Overrides the query part of the URL. See the [`request` action](request.md#query) for the `query` syntax. ### `headers` -Sets or removes request header fields. The syntax is the same as in the [request action](request.md#headers). +Sets or removes request header fields. The syntax is the same as in the [`request` action](request.md#headers). To remove a header, set its value to `""`. ### `options` Sets request options. See the [`request` action options](request.md#options) for valid options. -## Example +**Note**: that, with `proxy-request`, in contrast to `request`, the defaults for `exit-on-error`, `validate-request` and `validate-response` are `true`, if a `definition` is configured. + +## Examples + +Using `origin`: + +```xml + + + { + "origin": "https://example.com", + "stripEndpoint": true, + "addPrefix": "/path/to/api", + + "headers": { + "X-API-Key": "foo42bar" + }, + + "options": { + "definition": "upstream.yaml", + "validate-response": "report-only" + } + } + + +``` + +With the client request URL `http://client.example.com/my/api/foo/bar` matching the swagger definition path + +```yaml +basePath: /my/api +paths: + /foo/**: + x-flat-flow: proxy.xml +``` +the upstream request URL will be `https://example.com/path/to/api/bar`. + +Using `url`: ```xml @@ -44,10 +110,8 @@ Sets request options. See the [`request` action options](request.md#options) for }, "options": { - "exit-on-error": true, "definition": "upstream.yaml", - "validate-request": true, - "validate-response": true + "validate-response": "report-only" } } diff --git a/reference/variables.md b/reference/variables.md index 8498f90..7d5244d 100644 --- a/reference/variables.md +++ b/reference/variables.md @@ -172,8 +172,8 @@ Example: localhost 12345 XeeSVJ5AFt8VyXYagp3lvgAAACc - http://localhost:12345/api/proxy?a=b&c=d - /api/proxy + http://localhost:12345/api/proxy/foo?a=b&c=d + /api/proxy/foo a=b&c=d localhost:12345 @@ -193,6 +193,7 @@ Example: VALUE1 VALUE2 + /api/proxy ``` @@ -202,6 +203,30 @@ As HTTP request headers are defined to be case-insensitive, their names are lowe $request/headers/user-agent ``` +If a client URL path is matched by a [wildcard path](/reference/OpenAPI/differences.md#wildcard-paths), `$request/endpoint` is the path part preceding the part matched by `/**`. Otherwise, `$request/endpoint` is the same as `$request/path`. + +```yaml +… +basePath: /api +… +paths: + /**: + … + /foo/**: + … + /foo/qux: + … + /foo/{p1}: + … +``` + +| Client URL | matches | `$request/path` | `$request/endpoint` | +| --- | --- | --- | --- | +| https://example.com/api/foo/qux | /foo/qux | /api/foo/qux | /api/foo/qux | +| https://example.com/api/foo/quuux | /foo/{p1} | /api/foo/quuux | /api/foo/quuux | +| https://example.com/api/foo/bar/qux | /foo/** | /api/foo/bar/qux | /api/foo | +| https://example.com/api/bar | /** | /api/bar | /api | + ## `$error`