Merge pull request #79 from sevenval/develop

Release 20200424

CHANGELOG.md

@@ -1,4 +1,21 @@
-# Changelog
+## [20200424](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags)
+
+### Added
+
+- [Swagger `security` requirements](/reference/OpenAPI/security.md) can now also be specified at the path level.
+- [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies) to configure proxies without a flow
+- Enhanced [`proxy-request` action](/reference/actions/proxy-request.md) with `origin`, `query`, `stripEndpoint` and `addPrefix` properties
+
+### Fixed
+
+- If a client URL path is below the API base path, does not match any defined route, and a path is defined which equals the API base path, so that a matching client URL path is the concatenation of the API base path with itself (e.g. `/api/api` if the `basePath` is `/api`), the [fallback flow](/reference/OpenAPI/routing.md#fallback-flow) is now properly executed.
+- Some PEM formatted keys could not be recognized during [JWT processing](/cookbook/jwt.md).
+- Multi-line values for [environment variables](/cookbook/envvars.md) are now supported.
+
+### Changed
+
+- If the `definition` [request option](/reference/actions/request.md#options) is given with either a [`proxy-request` action](/reference/actions/proxy-request.md) or [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies), the defaults for the `exit-on-error`, `validate-request` and `validate-response` request options are changed to `true`.
+
 
 ## [20200409](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags)
 

cookbook/README.md

@@ -4,7 +4,9 @@
 
 ## [How can I pass an arbitrary header field to an upstream system?](pass-header-field-upstream.md)
 
-## [Forwarding a Request to an Upstream API](forward-request-upstream.md)
+## [Proxying Requests to Upstream APIs (Swagger)](proxy-requests.md)
+
+## [Forwarding a Request to an Upstream API (Flow)](forward-request-upstream.md)
 
 ## [How can I pass response headers to the client?](pass-header-field-downstream.md)
 

cookbook/forward-request-upstream.md

@@ -22,8 +22,8 @@ The following [flow](/reference/flow.md) shows a more advanced example:
 <flow>
   <proxy-request>
   {
-    {{// Replace the hostname of the incoming request with the UPSTREAM_URI environment variable }}
-    "url": {{ replace($request/url, "^http://[^/]+", $env/UPSTREAM_URI) }},
+    {{// Replace the origin of the incoming request with the UPSTREAM_ORIGIN environment variable }}
+    "origin": {{ $env/UPSTREAM_ORIGIN }},
 
     "headers": {
       {{// Set X-tra, drop X-Remove, copy Authorization }}
@@ -42,7 +42,7 @@ The following [flow](/reference/flow.md) shows a more advanced example:
 </flow>
 ```
 
-The `proxy-request` action lets you set the `url` and modify the `headers` of the request.
+The `proxy-request` action lets you set the `origin` and modify the `headers` of the request.
 Everything else is set up automatically: The client request body will be forwarded
 as-is, any headers not intended for upstream will be dropped.
 
@@ -103,7 +103,31 @@ status code of the upstream response and possibly additional header fields are c
 The forwarded request will only have a body if the incoming request does.
 The `Content-Type` request and response headers are passed automatically with the body, therefore we don't have to copy them explicitly.
 
+If the client and the upstream request URL paths do not share a common prefix, you can easily adjust the path by using `stripEndpoint` and `addPrefix`:
+
+```json
+{
+  "origin": {{ $env/UPSTREAM_ORIGIN }},
+  "stripEndpoint: true,
+  "addPrefix": {{ $env/UPSTREAM_PATH_PREFIX }},
+  …
+}
+```
+
+Assuming the following `swagger.yaml` for `https://client.example.com/`
+
+```yaml
+basePath: /api
+paths:
+  /users/**:
+    …
+```
+
+, `https://users.upstream.example.com` for `UPSTREAM_ORIGIN`, and `/v4` for `UPSTREAM_PATH_PREFIX`, a client request for `https://client.example.com/api/users/profile` will be forwarded to `https://users.upstream.example.com/v4/profile`.
+
+
 ## See also
 
+* [Proxying requests to Upstream APIs](proxy-requests.md) (cookbook)
 * [`proxy-request` action](/reference/actions/proxy-request.md) (reference)
 * [`request` action](/reference/actions/request.md) (reference)

cookbook/proxy-requests.md

@@ -0,0 +1,54 @@
+# Proxying requests to Upstream APIs
+
+To proxy requests to an upstream API, you can simply use a combination of two FLAT Swagger enhancements:
+
+* [wildcard paths](/reference/OpenAPI/differences.md#wildcard-paths), and
+* [`x-flat-proxy`](/reference/OpenAPI/routing.md#assigning-flat-proxies).
+
+Imagine, the upstream API, you want to use, is located at `https://upstream.api/api/docs` and provides a route to get a document with a certain ID (`/get-doc/{docid}`). To set FLAT as a proxy to this API, the following `swagger.yaml` snippet will do the job:
+
+```yaml
+…
+basePath: /api
+…
+paths:
+  /docs/**:
+    x-flat-proxy:
+      origin: https://upstream.api    # replaces the origin
+…
+```
+
+
+Assuming a client origin of `https://client.example.com`, a client request to e.g. `https://client.example.com/api/docs/get-doc/42` will be proxied to `https://upstream.api/api/docs/get-doc/42`.
+
+Note, that only the origin is replaced, the path is not modified.
+
+If the upstream API for docs is located at `https://docs.upstream.api/v4` instead of `https://upstream.api/api/docs`, the path has to be adjusted, too:
+
+```yaml
+…
+basePath: /flat
+…
+paths:
+  /docs/**:
+    x-flat-proxy:
+      origin: https://docs.upstream.api
+      stripEndpoint: true             # strips /flat/docs from the path
+      addPrefix: /v4                  # inserts /v4 before the stripped path
+…
+```
+
+The client request will be proxied to `https://docs.upstream.api/v4/get-doc/42`.
+
+Assuming you want to plug-in a different API (`https://users.upstream/v3`), just add the following:
+
+```yaml
+…
+  /users/**:
+    x-flat-proxy:
+      origin: https://users.upstream.api
+      stripEndpoint: true             # strips /flat/users from the path
+      addPrefix: /v3                  # inserts /v3 before the stripped path
+```
+
+A client request to `https://client.example.com/flat/users/profile/4711` will be proxied to `https://users.upstream.api/v3/profile/4711`.

reference/OpenAPI/differences.md

@@ -9,6 +9,7 @@ First of all, several extensions named `x-flat-…` are recognized on different
 * `x-flat-flow`: [flow](routing.md#assigning-flat-flows) to be started. Recognized at top-level and below `paths`, `paths/<path>` and `paths/<path>/<operation>`.
 * `x-flat-init`: [init flow](routing.md#init-flow) (top-level)
 * `x-flat-error`: [error flow](routing.md#error-flow) (top-level)
+* `x-flat-proxy`: [proxy configuration](routing.md#assigning-flat-proxies) (below `paths`, `paths/<path>` and `paths/<path>/<operation>`)
 * `x-flat-cors`: [CORS configuration](cors.md) (top-level)
 * `x-flat-validate`: [validation](validation.md) (top-level, below `paths/<path>` and `paths/<path>/<operation>`)
 * `x-flat-jwt`: [expected JWT](security.md#the-x-flat-jwt-field) (in a [security scheme object](https://swagger.io/specification/v2/#securitySchemeObject))
@@ -85,3 +86,7 @@ paths:
 The longest matching wildcard path wins. The position of a wildcard path in the definition is irrelevant.
 
 Note that path parameters (i.e. sections enclosed in curly braces) cannot be combined with wildcards.
+
+## Security
+
+[Security schemes](/reference/OpenAPI/security.md#applying-security-schemes) can also be applied to specific paths, not only to specific operations or top-level.

reference/OpenAPI/routing.md

@@ -100,8 +100,7 @@ paths:
 
 ### Init Flow
 
-An _init flow_ is a separate flow file that is executed before the regular [flow](/reference/flow.md)
-that is defined for an API path. It is specified by setting `x-flat-init` on
+An _init flow_ is a separate flow file that is executed before the regular [flow](/reference/flow.md) or [configured proxy](routing.md#assigning-flat-proxies) defined for an API path. It is specified by setting `x-flat-init` on
 the top level in the OpenAPI definition:
 
 ```yaml
@@ -129,7 +128,7 @@ flow from being executed, too.
 
 ### Error Flow
 
-An _error flow_ is an optional separate flow file that is executed if a client request or response validation error has occurred, or if the `exit-on-error` option was set for a [request](/reference/actions/request.md) that has failed.
+An _error flow_ is an optional separate flow file that is executed if a client request or response validation error has occurred, or if the `exit-on-error` option was set for a [request](/reference/actions/request.md), [proxy-request](/reference/actions/proxy-request.md) or [configured proxy](routing.md#assigning-flat-proxies) that has failed.
 It is specified by setting the `flow` property of `x-flat-error` on the top level in the OpenAPI definition:
 
 ```yaml
@@ -147,6 +146,39 @@ Requests to resources outside the `basePath` are handled by the default flow def
 in `conf/flow.xml`. This allows for [serving HTML, images, JavaScript](/cookbook/file-serving.md) and the like.
 
 
+## Assigning FLAT Proxies
+
+If FLAT acts as a proxy for an upstream API on a specific route, you could assign a flow containing a [`proxy-request` action](/reference/actions/proxy-request.md).
+
+A simpler way to achieve this is by using `x-flat-proxy` in the `swagger.yaml`:
+
+```yaml
+basePath: /api
+paths:
+  /users/**:
+    x-flat-proxy:
+      origin: https://users.upstream.example.com
+      stripEndpoint: true
+      addPrefix: /v4
+      headers:
+        Correlation-ID: 42
+      options:
+        timeout: 2
+        definition: users-upstream.yaml
+        validate-response: report-only
+```
+
+A client request to `https://client.example.com/api/users/profile` will be proxied to `https://users.upstream.example.com/v4/profile`. See [wildcard paths](differences.md#wildcard-paths) for more information about `/**`.
+
+`x-flat-proxy` can be used below `paths`, `paths/<path>` and `paths/<path>/<operation>`.
+
+The configuration for `x-flat-proxy` is the same as that for a [`proxy-request` action](/reference/actions/proxy-request.md) (translated from JSON to YAML syntax).
+
+If configured, the [init flow](routing.md#init-flow) is executed before the proxy request. If configured, the [error flow](routing.md#error-flow) is executed if the `exit-on-error` option was set and the proxy request fails.
+
+`x-flat-proxy` and `x-flat-flow` are alternatives and cannot be used in combination.
+
+
 ## Path Parameters
 
 Swagger paths can define path parameters:

reference/OpenAPI/security.md

@@ -43,7 +43,7 @@ securityDefinitions:
 ```
 The code in this example defines a security scheme named `JWTHeaderAuth`.
 The token is expected to be a bearer token in the `Authorization` header.
-The key is read from a file named `secret.pem` relative to the swagger.yaml.
+The key is read from a file named `secret.pem` relative to the `swagger.yaml`.
 The signing algorithm is read from the `FLAT_JWT_ALG` environment variable.
 The JWT will be stored in the `$header_token` variable.
 The JWT payload is expected to contain an `aud` claim with a value read from the `FLAT_JWT_AUDIENCE` environment variable.
@@ -92,18 +92,24 @@ paths:
 
 ## Applying Security Schemes
 
-In Swagger, Security Schemes can be specified at the top level (default security) or for specific operations.
+In Swagger, security schemes can be specified at the top level (default security) or for specific operations.
+With FLAT, you can also specify a security scheme for a specific path (default security for all operations on the path).
 
-In the following example, a `GET` request to `foo` must satisfy the security scheme named `JWTHeaderAuth`.
+In the following example, a `GET` request to `/foo` must satisfy the security scheme named `JWTHeaderAuth`, while e.g. a `POST` or `PUT` request to `/foo` must satisfy the security scheme named `JWTCookieAuth`.
 All other requests must satisfy either the `JWTHeaderAuth` **or** `JWTCookieAuth` security schemes.
 
 ```yaml
+# default
 security:
   - JWTHeaderAuth: []
   - JWTCookieAuth: []
 paths:
   /foo:
+    # default for all operations on /foo
+    security:
+      - JWTCookieAuth: []
     get:
+      # specific for GET on /foo
       security:
         - JWTHeaderAuth: []
 ```

reference/actions/README.md

@@ -8,6 +8,8 @@
 * [`eval`](eval.md)
 * [`log`](log.md)
 * [`nameshave`](nameshave.md)
+* [`pass-body`](pass-body.md)
+* [`proxy-request`](proxy-request.md)
 * [`regex`](regex.md)
 * [`request`](request.md)
 * [`requests`](requests.md)

reference/actions/proxy-request.md

@@ -18,20 +18,86 @@ Just like an ordinary [`request`](request.md) the `proxy-request`
 can be configured using a [JSON template](/reference/templating/README.md)
 with the following properties:
 
+### `origin`
+
+Sets the origin of the upstream system. Either `origin` or `url` is required. Type: `string`.
+
+### `stripEndpoint`
+
+To be used in connection with `origin`. If `true`, strips the endpoint path from the client request URL path before it is added to the upstream origin. Type: `boolean`, default: `false`
+
+E.g.
+
+```yaml
+basePath: /api
+paths:
+  /foo/bar:
+  /wildcard/**
+```
+
+For the client request URL `https://client.example.com/api/foo/bar`, the path is stripped to `/`.
+
+For the client request URL `https://client.example.com/api/wildcard/path/to`, the path is stripped to `/path/to`.
+
+### `addPrefix`
+
+Inserts a path prefix before the given (client request URL) path, after possible endpoint stripping (see `stripEndpoint`). Type: `string`.
+
 ### `url`
 
-Sets the URL to the upstream system. Required.
+Sets the URL to the upstream system. Either `url` or `origin` is required.
+
+### `query`
+
+Overrides the query part of the URL. See the [`request` action](request.md#query) for the `query` syntax.
 
 ### `headers`
 
-Sets or removes request header fields. The syntax is the same as in the [request action](request.md#headers).
+Sets or removes request header fields. The syntax is the same as in the [`request` action](request.md#headers).
 To remove a header, set its value to `""`.
 
 ### `options`
 
 Sets request options. See the [`request` action options](request.md#options) for valid options.
 
-## Example
+**Note**: that, with `proxy-request`, in contrast to `request`, the defaults for `exit-on-error`, `validate-request` and `validate-response` are `true`, if a `definition` is configured.
+
+## Examples
+
+Using `origin`:
+
+```xml
+<flow>
+  <proxy-request>
+  {
+    "origin": "https://example.com",
+    "stripEndpoint": true,
+    "addPrefix": "/path/to/api",
+
+    "headers": {
+      "X-API-Key": "foo42bar"
+    },
+
+    "options": {
+      "definition": "upstream.yaml",
+      "validate-response": "report-only"
+    }
+  }
+  </proxy-request>
+</flow>
+```
+
+With the client request URL `http://client.example.com/my/api/foo/bar` matching the swagger definition path
+
+```yaml
+basePath: /my/api
+paths:
+  /foo/**:
+    x-flat-flow: proxy.xml
+```
+the upstream request URL will be `https://example.com/path/to/api/bar`.
+
+Using `url`:
 
 ```xml
 <flow>
@@ -44,10 +110,8 @@ Sets request options. See the [`request` action options](request.md#options) for
     },
 
     "options": {
-      "exit-on-error": true,
       "definition": "upstream.yaml",
-      "validate-request": true,
-      "validate-response": true
+      "validate-response": "report-only"
     }
   }
   </proxy-request>