Merge pull request #87 from sevenval/develop

Release 20200519

CHANGELOG.md

@@ -1,3 +1,23 @@
+# Changelog
+
+## Unreleased
+
+## [20200519](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags)
+
+### Added
+
+- [Beta image](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags?name=beta) now publicly available. [More about Docker images…](/administration/docker.md)
+- Warnings in [debug log](/reference/debugging.md) about invalid [Swagger definitions](/reference/openapi/differences.md)
+- Validation for the [`assert`](/reference/actions/assert.md) and [`set-env`](/reference/actions/set-env.md) test action configurations.
+- The [`error` action](/reference/actions/error.md)
+- additional configuration options for the [PHP-FPM process management](/administration/configuration.md)
+- [`out-header`](/reference/OpenAPI/security.md#forwarding-jwt-upstream) property for easy JWT forwarding
+
+## Fixed
+
+-  Calls to the [`content()` function](/reference/functions/content.md) affecting the result of the [`body()` function](/reference/functions/body.md)
+
+
 ## [20200424](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags)
 
 ### Added

administration/README.md

@@ -1,7 +1,6 @@
 # Administration
 
-_FLAT Server_ is publicly available as a [Docker](https://www.docker.com) image: https://hub.docker.com/r/sevenvaltechnologies/flatrunner
-
+_FLAT Server_ is publicly available as a [Docker](docker.md) image.
 Therefore, running FLAT requires a Docker setup of your choice.
 
 ## [Configuration](configuration.md)

administration/configuration.md

@@ -10,11 +10,12 @@ The chapter [Defining Env Vars](/cookbook/envvars.md#defining-env-vars) in the C
 ### Miscellaneous
 
 * `FLAT_SERVER_ROLE`: This setting is typically used to distinguish production systems from staging or development servers. Its value is accessible in the [`$server` variable](/reference/variables.md#predefined-variables) as `$server/role`.
+* `FLAT_STATUS_AUTH`: Username and password, separated by a `:` for access to the `php-fpm` and `httpd` status pages. The pages are disabled entirely if `FLAT_STATUS_AUTH` is not set. If enabled, the `httpd` status can be requested via HTTP at `/ServerStatus`, and the php-fpm status at `/FPMStatus?full`.
 
 
 ### Request Timeouts
 
-Note that the maximum allowed overall time for a given request is always limited by FLAT_MAX_TIMEOUT. All other timeouts can be individually adjusted for each request using the appropriate [request option](/reference/actions/request.md#options).
+Note that the maximum allowed overall time for a given request is always limited by `FLAT_MAX_TIMEOUT`. All other timeouts can be individually adjusted for each request using the appropriate [request option](/reference/actions/request.md#options).
 
 Use the following environment variables to configure the timeouts used during requests:
 
@@ -28,8 +29,11 @@ Use the following environment variables to configure the timeouts used during re
 
 Some parameters used for PHP-FPM process management can be adjusted using environment variables. Refer to the [PHP-FPM documentation](https://www.php.net/manual/en/install.fpm.configuration.php) for more information.
 
-* `FLAT_FPM_MAX_PROCESSES`: Controls `pm.max_children`. Default `100`. The maximum number of child processes to be created.
+* `FLAT_FPM_MAX_PROCESSES`: Controls `pm.max_children`. Default `100`.
 * `FLAT_FPM_MAX_REQUESTS`: Controls `pm.max_requests`. Default `500`. Limits the number of requests each child process executes before it is replaced by a new process.
+* `FLAT_FPM_MIN_SPARE`: Controls `pm.min_spare_servers`. Default `5`. The desired minimum number of idle child processes to be created.
+* `FLAT_FPM_MAX_SPARE`: Controls `pm.max_spare_servers`. Default `7`. The desired maximum number of idle child processes.
+* `FLAT_FPM_START_SERVERS`: Controls `pm.start_servers`. The default is the average of `pm.min_spare_servers` and `pm.min_spare_servers`. The initial number of child processes to be started.
 
 ## See also
 

administration/docker.md

@@ -0,0 +1,48 @@
+# Docker
+
+The _FLAT Server_ is publicly available as a [Docker](https://www.docker.com) image:
+https://hub.docker.com/r/sevenvaltechnologies/flatrunner
+So, make sure you have Docker installed on your system or an alternative means to
+download and run Docker images off Docker Hub.
+
+## Images
+
+The [`latest`](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags?name=latest)
+image provides you with the most recent stable official release.
+A [`beta`](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags?name=beta)
+image with brand-new [features](/CHANGELOG.md) is also available.
+
+You can use the [`flat`](/reference/flat-cli.md) command line tool
+to pull the latest release or beta image:
+
+```sh
+$ flat pull
+```
+
+or
+
+```sh
+$ flat pull -b
+```
+
+## Container
+
+To run the server, simply start the Docker container with the
+[`flat`](/reference/flat-cli.md) CLI:
+
+```sh
+$ flat start
+```
+
+Add the `-b` option to start a container based on the `beta` release:
+
+```sh
+$ flat start -b
+```
+
+To use a specific image, provide the respective release number as `TAG`
+environment variable, for example
+
+```sh
+$ TAG=20200409 flat start
+```

administration/logging.md

@@ -27,7 +27,7 @@ Client requests received by FLAT are recorded as type `flat_access` and have the
   "method": "GET",
   "agent": "curl/7.54.0",
   "referrer": "",
-  "mime": "application/json"
+  "mime": "application/json",
   "realtime": 0.019845,
   "bytes": 563,
   "requestbytes": 0,

cookbook/envvars.md

@@ -47,7 +47,7 @@ The `inline` sink interweaves the debug output with your API response. It will d
 
 ### Docker
 
-If you control the docker setup yourself (e.g. `docker-compose`, Kubernetes, …) you can use its built-in support for environment variables. In this case, you are not bound to variable names starting with `FLAT_`.
+If you control the Docker setup yourself (e.g. `docker-compose`, Kubernetes, …) you can use its built-in support for environment variables. In this case, you are not limited to variables with names starting with `FLAT_`.
 
 Your `docker-compose.yaml` could look like this:
 ```yaml
@@ -179,7 +179,7 @@ You could read Basic Auth credentials from an env var in your [request](/referen
 </flow>
 ```
 
-Or you need to include a token in your requests:
+Or if you need to include a token in your requests:
 
 ```xml
 <flow>

cookbook/error-flow.md

@@ -1,9 +1,11 @@
 
 # Handling Errors with an Error Flow
 
-The [_error flow_](/reference/OpenAPI/routing.md#error-flow) is typically used to 
+The [_error flow_](/reference/OpenAPI/routing.md#error-flow) is triggered when certain
+severe errors occur while processing the flow, or when the
+[`error` action](/reference/actions/error.md) is invoked. It is typically used to
 produce custom error messages or headers, or set the HTTP status. Another example
-would be to use it to log additional information using [`fit-log()`](/reference/functions/fit-log.md)
+is augmenting the [access log](/administration/logging.md) with [custom log fields](/cookbook/custom-logging.md).
 
 In your `swagger.yaml`, point the top-level property `x-flat-error` to a flow
 file:
@@ -38,6 +40,15 @@ In this example, we use `error.xml` to format a custom error message:
 
 This will ensure that all errors that trigger the error flow will produce a consistent status, error message and headers.
 
+## Triggers
+
+Error triggers are
+
+* [schema violations](/reference/OpenAPI/validation.md),
+* [security violations (JWT)](/reference/OpenAPI/security.md),
+* [request errors](/reference/actions/request.md#options),
+* or the [`error` action](/reference/actions/error.md)
+
 ## See also
 
 * [Error Flow](/reference/OpenAPI/routing.md#error-flow)

reference/OpenAPI/security.md

@@ -12,6 +12,7 @@ The `x-flat-jwt` field references an object with fields describing the expected
 * `key` - REQUIRED. The key to decode the JSON Web Signature (JWS). This can either be specified with a value, or by referencing a file (`file`) or an environment variable (`env`).
 * `alg` - The signing algorithm the JWS is expected to be created with. This can either be specified with a value, or by referencing a file (`file`) or an environment variable (`env`). See the [`algorithm` parameter for `jwt-decode()`](/reference/functions/jwt-decode.md) for more information.
 * `out-var` - The name of the variable in which the JWT is stored (must be a proper variable name, starting with `$`; default: `"$jwt"`).
+* `out-header` - The name of the HTTP header that shall carry the JWT
 * `claims` - An object with claims the JWT payload is expected to contain. The field names are the claim names, the expected claim value is specified either with a value, or by referencing a file (`file`) or an environment variable (`env`).
 
 The token is considered valid if all of the following are true:
@@ -155,3 +156,73 @@ paths:
         - JWTHeaderAuth: []
 ```
 
+## Forwarding JWT Upstream
+
+The claims of an incoming JWT are stored in `$jwt` – or in any other global
+[variable](/reference/variables.md) that you specify in the `out-var`
+property of `x-flat-jwt`.
+In a [`request`](/reference/actions/request.md) or
+[`proxy-request`](/reference/actions/proxy-request.md) action you can send
+these claims upstream, for example using a query parameter or a header:
+
+```xml
+<flow>
+  <proxy-request>
+  {
+    "url": "https://httpbin.org/get",
+    "query": {{ $jwt }},                  <!-- ⬅ as query parameters -->
+    "headers": {
+      "Token": {{ json-stringify($jwt) }} <!-- ⬅ as 'Token' header -->
+    }
+  }
+  </proxy-request>
+</flow>
+```
+
+If we assume the value of `$jwt` being `{"id":42,"name":"John Doe"}`, the resulting
+upstream request URL would be `https://httpbin.org/get?id=42&name=John%20Doe`
+and the request would include a `Token` header: `Token: {"id":42,"name":"John Doe"}`.
+
+To just forward the received JWT token upstream as a header, you can also specify
+the desired header name with the `out-header` property of `x-flat-jwt`:
+
+```yaml
+securityDefinitions:
+  JWTHeaderAuth:
+    …
+    x-flat-jwt:
+      key: …
+      out-header: Token # ⬅ forwards this JWT in the 'Token' header
+```
+
+The `headers` section in `proxy-request` is then no longer needed:
+
+```xml
+<flow>
+  <proxy-request>
+  {
+    "url": "https://httpbin.org/get"
+  }
+  </proxy-request>
+</flow>
+```
+
+Moreover, if you just want to proxy requests, there's no need to define a [flow](routing.md#assigning-flat-flows)
+and you can simply set up your requests using [`x-flat-proxy`](routing.md#assigning-flat-proxies):
+
+```yaml
+…
+securityDefinitions:
+  JWTHeaderAuth:
+    …
+    x-flat-jwt:
+      key: …
+      out-header: Token # ⬅ forwards the JWT in the 'Token' header
+paths:
+  /:
+    get:
+      x-flat-proxy: # ⬅ no more x-flat-flow
+        url: https://httpbin.org/get
+      security:
+        - JWTHeaderAuth: []
+```

reference/actions/README.md

@@ -5,6 +5,7 @@
 * [`debug`](debug.md)
 * [`dump`](dump.md)
 * [`echo`](echo.md)
+* [`error`](error.md)
 * [`eval`](eval.md)
 * [`log`](log.md)
 * [`nameshave`](nameshave.md)

reference/actions/error.md

@@ -0,0 +1,92 @@
+# `error` Action
+
+The `error` action provides a simple and consistent way to handle errors throughout the configuration.
+
+It terminates the flow, sets its body template result as [`$error`](/reference/variables.md#usderror) and calls the [error flow](/reference/OpenAPI/routing.md#error-flow), if configured, or sends the [system error document](/reference/OpenAPI/validation.md#system-error-document) with the body template result as the value of the `error` property.
+
+In its simplest form, an error is triggered like this:
+
+```xml
+<error/>
+```
+
+To make errors more useful, they should contain a message:
+
+```xml
+<error>
+{
+  "message": "Upstream system sent unusable data"
+}
+</error>
+```
+
+## Error Properties
+
+The action body contains a constant JSON string or a [JSON template](/reference/templating/README.md) representing a JSON object.
+The object will be assigned to the `$error` variable.
+
+The following properties of the `$error` object are optional, but must have the specified 
+type if set. If not set, they will receive the following default values:
+
+* `message`: `string`, default: `'FlowError'`
+* `status`: `integer` between 100 and 599, default: `500`
+* `code`: `integer` between 0 and 9999, default: `5000`
+* `info`: `array` of `string`, default: `["Flow Error triggered"]`
+
+Additional properties are allowed. They will be accessible in the Error Flow. The following example includes the URL in `$error/url`:
+
+```xml
+<error>
+{
+  {{: $request/url }}
+  "message": "Upstream system sent unusable data",
+  "status": 502,
+  "code": 4711
+}
+</error>
+```
+
+## Status Code
+
+The `status` property will be used as the HTTP response status code. The status code may alternatively be defined with the optional `status` attribute.
+
+```xml
+<error status="403"/>
+```
+
+```xml
+<error status="502">
+{
+  "message": "Upstream system sent unusable data"
+}
+</error>
+```
+
+## Fixed message shortcut
+
+If the action body is a string constant (it must be enclosed in double quotes), its value will be
+assigned to the `$error/message` property. This is a handy shortcut to trigger simple errors with fixed messages:
+
+```xml
+<error status="502"> "Upstream system sent unusable data" </error>
+```
+
+## Logging
+
+The `error` emits an [error log](/administration/logging.md#error-log) event with the `error` topic and the given `message`:
+
+```json
+{
+  "timestamp": "2020-05-19T16:27:46+00:00",
+  "type": "flat_alert",
+  "requestID": "XsQJAMvNgp1P2qKAxKmdaAAAAAI",
+  "topic": "error",
+  "message": "Upstream system sent unusable data"
+}
+```
+
+## See also
+
+* [Error Flow](/reference/OpenAPI/routing.md#error-flow)
+* [`$error`](/reference/variables.md#usderror)
+* [Handling Errors](/cookbook/error-flow.md) (Cookbook)

reference/flat-cli.md

@@ -1,20 +1,35 @@
 # `flat` CLI
 
 Get the `flat` CLI from [sevenval/flat-cli](https://github.com/sevenval/flat-cli).
+See the [tutorial](/tutorial/README.md#tools) for installation instructions.
 
-Start the local FLAT server with:
+You can start the local FLAT server with
 
 ```sh
 $ flat start
 ```
 
-Type
+The option `-b` allows you to run the current beta version instead of the stable version:
+
+```sh
+$ flat start -b
+```
+
+FLAT is provided as a [Docker](/administration/docker.md) image. Use
+
+```sh
+$ flat pull
+```
+
+to keep your local FLAT image up to date.
+
+Run
 
 ```sh
 $ flat -h
 ```
 
-for all available commands and options of the FLAT cli.
+for a list of  all available commands and options of the FLAT CLI.
 
 
 See also