From ef6fa6700e3a3950569cbf0bafb6227be069cbac Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Fri, 2 Oct 2020 15:18:48 +0200 Subject: [PATCH 01/43] uuid3() and uuid4() #47606-uuid-xpath-function --- reference/functions/README.md | 1 + reference/functions/uuid.md | 14 ++++++++++++++ 2 files changed, 15 insertions(+) create mode 100644 reference/functions/uuid.md diff --git a/reference/functions/README.md b/reference/functions/README.md index 3d4c572..00cf19a 100644 --- a/reference/functions/README.md +++ b/reference/functions/README.md @@ -93,4 +93,5 @@ * [`name()` ↗](https://developer.mozilla.org/en/XPath/Functions/name) * [`namespace-uri()` ↗](https://developer.mozilla.org/en/XPath/Functions/namespace-uri) * [`position()` ↗](https://developer.mozilla.org/en/XPath/Functions/position) +* [`uuid3()` and `uuid4()` ↗](uuid.md) * [`xml-parse()`](xml-parse.md) diff --git a/reference/functions/uuid.md b/reference/functions/uuid.md new file mode 100644 index 0000000..0d72aee --- /dev/null +++ b/reference/functions/uuid.md @@ -0,0 +1,14 @@ +# `uuid3()` and `uuid4()` + +``` +string uuid3(string namespace_and_name) +string uuid4() +``` + +The functions `uuid3()` and `uuid4()` generate a [UUID](https://tools.ietf.org/html/rfc4122) version 3 or 4, respectively. + + +## Examples + +* `uuid4()` returns `da7e73a7-8dca-4123-b671-7997d7fd2fb8` or a similar, random UUID +* `uuid3("foobar")` is not really a call that conforms to the standard, since no namespace (a binary UUID) was included, but returns `3858f622-30ac-3c91-9f30-0c664312c63f` anyway. From bfa8852d5fcfd3785199dc7b59a7e6bdadc501a9 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Fri, 2 Oct 2020 18:51:10 +0200 Subject: [PATCH 02/43] document FIT_FLAT_UID_FORMAT env var #47606-uuid-xpath-function --- administration/configuration.md | 1 + 1 file changed, 1 insertion(+) diff --git a/administration/configuration.md b/administration/configuration.md index 49245af..659f3e6 100644 --- a/administration/configuration.md +++ b/administration/configuration.md @@ -13,6 +13,7 @@ The chapter [Defining Env Vars](/cookbook/envvars.md#defining-env-vars) in the C * `FLAT_STATUS_AUTH`: Username and password, separated by a `:` for access to the `php-fpm` and `httpd` status pages. The pages are disabled entirely if `FLAT_STATUS_AUTH` is not set. If enabled, the `httpd` status can be requested via HTTP at `/ServerStatus`, and the php-fpm status at `/FPMStatus?full`. * `FLAT_DEBUG_ALLOW_HEADER`: enable [per request debugging](/reference/debugging.md#request-debugging), defaults to `false` unless `FLAT_DEBUG_AUTH` is set. * `FLAT_DEBUG_AUTH`: sets a password to protect [per request debugging](/reference/debugging.md#request-debugging). +* `FIT_FLAT_UID_FORMAT`: use `uuid3` or `uuid4` to switch the format of the `requestID` generated in the logs and the `id` in the DC to UUID version 3 or 4 respectively instead of the Apache httpd unique_id used by default. ### Request Timeouts From 01c151adac48a6a679197b87f660e420afd026e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 5 Oct 2020 10:56:57 +0200 Subject: [PATCH 03/43] Mentioned order of custom log fields in access_log. --- reference/actions/log.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/reference/actions/log.md b/reference/actions/log.md index 8d558dd..dc2b1ee 100644 --- a/reference/actions/log.md +++ b/reference/actions/log.md @@ -22,6 +22,7 @@ All name/value pairs of the object are registered for logging. When the system [System log fields](/administration/logging.md#fields) like `timestamp` cannot be overriden. You can call the action multiple times. Fields of the same name are overwritten. However, nested fields are merged into the previously registered log fields. +The order of fields is maintained. ```xml @@ -50,6 +51,18 @@ The merged custom log fields are: } ``` +They would appear in the [access log](/administration/logging.md#access-log) like this: + +```json +{ + "timestamp": "2019-10-15T15:49:13+00:00", + "type": "flat_access", + … + "name": "alice", + "role": "admin" +} +``` + Fields with `null` values are not included in the log event. In order to remove a previously registered field, you can unset it with `null` value. The `user` field of the previous example can be unregistered like this: From e554d71dbc678ea71e002c848cd0c387b5a8eeca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 5 Oct 2020 12:45:56 +0200 Subject: [PATCH 04/43] Fixes: `name` and `role` are actually parts of `user`, log line is a one-liner --- reference/actions/log.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/reference/actions/log.md b/reference/actions/log.md index dc2b1ee..f62fbed 100644 --- a/reference/actions/log.md +++ b/reference/actions/log.md @@ -54,13 +54,7 @@ The merged custom log fields are: They would appear in the [access log](/administration/logging.md#access-log) like this: ```json -{ - "timestamp": "2019-10-15T15:49:13+00:00", - "type": "flat_access", - … - "name": "alice", - "role": "admin" -} +{"timestamp": …,"user":{"name":"alice","role":"admin"}} ``` Fields with `null` values are not included in the log event. In order to remove a previously registered field, you can unset it with `null` value. From 31850194fc9a3f28f81675970d3b3c82e3a5e1fa Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Mon, 5 Oct 2020 15:16:39 +0200 Subject: [PATCH 05/43] document uuid3() properly #47606-uuid-xpath-function --- reference/functions/uuid.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/functions/uuid.md b/reference/functions/uuid.md index 0d72aee..ba536c9 100644 --- a/reference/functions/uuid.md +++ b/reference/functions/uuid.md @@ -1,14 +1,14 @@ # `uuid3()` and `uuid4()` ``` -string uuid3(string namespace_and_name) +string uuid3(string namespace, string name) string uuid4() ``` The functions `uuid3()` and `uuid4()` generate a [UUID](https://tools.ietf.org/html/rfc4122) version 3 or 4, respectively. - ## Examples * `uuid4()` returns `da7e73a7-8dca-4123-b671-7997d7fd2fb8` or a similar, random UUID -* `uuid3("foobar")` is not really a call that conforms to the standard, since no namespace (a binary UUID) was included, but returns `3858f622-30ac-3c91-9f30-0c664312c63f` anyway. +* `uuid3('6ba7b810-9dad-11d1-80b4-00c04fd430c8', 'example.com')` returns `9073926b-929f-31c2-abc9-fad77ae3e8eb`. +* `uuid3("foobar")` is not really a call that conforms to the standard, since no namespace (a UUID) was included, but returns `3858f622-30ac-3c91-9f30-0c664312c63f` anyway. From 9a7b5c0978cbb3962f757bf2409e1d7bae3b2cc8 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Mon, 5 Oct 2020 17:18:37 +0200 Subject: [PATCH 06/43] documentation of uuid3() and uuid4() is not remote --- reference/functions/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/functions/README.md b/reference/functions/README.md index 00cf19a..a2f6a55 100644 --- a/reference/functions/README.md +++ b/reference/functions/README.md @@ -93,5 +93,5 @@ * [`name()` ↗](https://developer.mozilla.org/en/XPath/Functions/name) * [`namespace-uri()` ↗](https://developer.mozilla.org/en/XPath/Functions/namespace-uri) * [`position()` ↗](https://developer.mozilla.org/en/XPath/Functions/position) -* [`uuid3()` and `uuid4()` ↗](uuid.md) +* [`uuid3()` and `uuid4()`](uuid.md) * [`xml-parse()`](xml-parse.md) From e263343efc93adee3bb41daa9234e43f89c7b116 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Mon, 5 Oct 2020 17:21:52 +0200 Subject: [PATCH 07/43] changelog for uuid3() and uuid4() --- CHANGELOG.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index eccfe44..1701b97 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## Unreleased + +### Added + +- The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) + + ## [20200828](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) ### Added From 4689d3beed9d34e76fd2da35fb6165c27e06dffb Mon Sep 17 00:00:00 2001 From: Felix Hassert Date: Thu, 8 Oct 2020 09:21:08 +0200 Subject: [PATCH 08/43] add see-alsos for uuid functions --- reference/functions/uuid.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/reference/functions/uuid.md b/reference/functions/uuid.md index ba536c9..2b3c233 100644 --- a/reference/functions/uuid.md +++ b/reference/functions/uuid.md @@ -12,3 +12,7 @@ The functions `uuid3()` and `uuid4()` generate a [UUID](https://tools.ietf.org/h * `uuid4()` returns `da7e73a7-8dca-4123-b671-7997d7fd2fb8` or a similar, random UUID * `uuid3('6ba7b810-9dad-11d1-80b4-00c04fd430c8', 'example.com')` returns `9073926b-929f-31c2-abc9-fad77ae3e8eb`. * `uuid3("foobar")` is not really a call that conforms to the standard, since no namespace (a UUID) was included, but returns `3858f622-30ac-3c91-9f30-0c664312c63f` anyway. + +## See also + +* [`FIT_FLAT_UID_FORMAT`](/administration/configuration#miscellaneous) From 2fc7678f54b228a3c7fdd354ad2f49352f7472dc Mon Sep 17 00:00:00 2001 From: Felix Hassert Date: Thu, 8 Oct 2020 09:22:50 +0200 Subject: [PATCH 09/43] typo --- reference/functions/uuid.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/functions/uuid.md b/reference/functions/uuid.md index 2b3c233..44c6056 100644 --- a/reference/functions/uuid.md +++ b/reference/functions/uuid.md @@ -15,4 +15,4 @@ The functions `uuid3()` and `uuid4()` generate a [UUID](https://tools.ietf.org/h ## See also -* [`FIT_FLAT_UID_FORMAT`](/administration/configuration#miscellaneous) +* [`FIT_FLAT_UID_FORMAT`](/administration/configuration.md#miscellaneous) From eb6535e2c50f372c1259b6a4009cdbf05290487d Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Tue, 27 Oct 2020 11:45:04 +0100 Subject: [PATCH 10/43] Changelog for path parameters in error flows --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1701b97..9c9dc42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) +### Fixed + +- [Path parameters](/reference/openapi/routing.md#path-parameters) were not usable in [error flows](/reference/openapi/routing.md#error-flow) + ## [20200828](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) From f86cc204ecfecef8f437b2f05bb2bc5e59d0730c Mon Sep 17 00:00:00 2001 From: johakoch <53434855+johakoch@users.noreply.github.com> Date: Wed, 28 Oct 2020 08:10:38 +0100 Subject: [PATCH 11/43] Fix path in CHANGELOG.md Co-authored-by: Felix Hassert --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c9dc42..6986d07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ ### Fixed -- [Path parameters](/reference/openapi/routing.md#path-parameters) were not usable in [error flows](/reference/openapi/routing.md#error-flow) +- [Path parameters](/reference/OpenAPI/routing.md#path-parameters) were not usable in [error flows](/reference/OpenAPI/routing.md#error-flow) ## [20200828](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) From 3c2829999c64449d5b2bb5746499fc6e1d510de9 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Wed, 28 Oct 2020 15:32:05 +0100 Subject: [PATCH 12/43] XPath function ldap-query() --- reference/functions/README.md | 1 + reference/functions/ldap-query.md | 59 +++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 reference/functions/ldap-query.md diff --git a/reference/functions/README.md b/reference/functions/README.md index a2f6a55..55aab62 100644 --- a/reference/functions/README.md +++ b/reference/functions/README.md @@ -86,6 +86,7 @@ * [`html-parse()`](html-parse.md) * [`id()` ↗](https://developer.mozilla.org/en/XPath/Functions/id) * [`ldap-lookup()`](ldap-lookup.md) +* [`ldap-query()`](ldap-query.md) * [`lang()` ↗](https://developer.mozilla.org/en/XPath/Functions/lang) * [`last()` ↗](https://developer.mozilla.org/en/XPath/Functions/last) * [`local-name()` ↗](https://developer.mozilla.org/en/XPath/Functions/local-name) diff --git a/reference/functions/ldap-query.md b/reference/functions/ldap-query.md new file mode 100644 index 0000000..9519d85 --- /dev/null +++ b/reference/functions/ldap-query.md @@ -0,0 +1,59 @@ +# `ldap-query()` + +``` +OXN-node-set ldap-query(string url, string rdn, string rdnPassword, string base_dn, string search, string attributes) +``` + +The `ldap-query()` function connects to an LDAP server with the given `url`, `rdn` and `rdnPassword`. +It then performs a query by the given `search`. +An [OXN](/reference/templating/oxn.md) JSON array is returned with objects containing the found entities' `dn` and additional attributes given by `attributes`. +If no entities match the query an empty node-set is returned. + +## Parameters + +* `url` The ldap URL (string) +* `rdn` The (relative) distinguished name of the connecting user (string) +* `rdnPassword` The password of the connecting user (string) +* `base_dn` The base distinguished name for the directory, used for the search (string) +* `search` The filter for searching entities (string) +* `attributes` A comma-separated list of attributes to return (string) + + +## Example + +In the following example, the LDAP server is connected with the DN given in the `rdn` and `rdnPassword` POST parameters. +The given filter is used to search for an entry of a person which is a member of a group `Users` and sAMAccountName containg `doe`. +In addition to the (default) `dn`, the `sAMAccountName`, `displayName` and `mail` from the found entries are added to the results. + +```xml + + concat("(&(objectClass=person)(memberOf=CN=Users,ou=People,dc=example,dc=com)(sAMAccountName=*doe*))") + "sAMAccountName,displayName,mail" + + ldap-lookup($ldap_settings/url, $request/post/rdn, $request/post/rdnPassword, "dc=example,dc=com", $search, $attributes) + + { + "status": 403, + "message": "ldap-lookup() failed" + } + + +``` + +The result is +```json +[ + { + "dn": "cn=John Doe,ou=People,dc=example,dc=com", + "sAMAccountName": "john.doe", + "displayName": "John Doe", + "mail": "john.doe@example.com" + }, + { + "dn": "cn=Joseph Adoell,ou=People,dc=example,dc=com", + "sAMAccountName": "joseph.adoell", + "displayName": "Joseph Adoell", + "mail": "joseph.adoell@example.com" + } +] +``` From 200266ea6df70a24fea3bf645c894ffaa61206a6 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Wed, 28 Oct 2020 15:33:25 +0100 Subject: [PATCH 13/43] Changelog entry for ldap-query() --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6986d07..6aa1eab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ ### Added - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) +- The [`ldap-query()` function](/reference/functions/ldap-query.md) ### Fixed From 3dac7026e4582f7e344833cfb02df8901021b017 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Wed, 28 Oct 2020 21:24:44 +0100 Subject: [PATCH 14/43] write PHP_INI_* environment variables in php.ini.d/from_env.ini #47662 --- administration/configuration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/administration/configuration.md b/administration/configuration.md index 659f3e6..2336c25 100644 --- a/administration/configuration.md +++ b/administration/configuration.md @@ -27,6 +27,10 @@ Use the following environment variables to configure the timeouts used during re * `FLAT_MAX_TIMEOUT`: Maximum allowed time in seconds for outgoing HTTP requests. * `FLAT_MAX_TIMEOUT_PROCESSES`: Specifies how many PHP-FPM processes are allowed to wait for slow sources at any one time. The lower the threshold value, the earlier slow sources will be blocked. +### php.ini + +Arbitrary directives can be merged into `php.ini` by setting environment variables starting with `PHP_INI_`. For example, to set `post_max_size = 100M`, just export `PHP_INI_post_max_size=50M` in your shell before starting `flat` CLI. Refer to the [PHP documentation](https://www.php.net/manual/en/ini.list.php) for a list of directives. + ### PHP-FPM Some parameters used for PHP-FPM process management can be adjusted using environment variables. Refer to the [PHP-FPM documentation](https://www.php.net/manual/en/install.fpm.configuration.php) for more information. From f888f0f11808eb20461668712192204cac6b204e Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 29 Oct 2020 12:58:04 +0100 Subject: [PATCH 15/43] CA cert file for ldaps URIs --- reference/configuration.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/reference/configuration.md b/reference/configuration.md index be97fff..3e25d11 100644 --- a/reference/configuration.md +++ b/reference/configuration.md @@ -31,3 +31,19 @@ The difference is, that you may use _Dynamic Attribute Values_ and if-clauses as ``` + +## LDAP TLS Configuration + +If you use the [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or +[`ldap-query()`](/reference/functions/ldap-query.md) function and connect to the LDAP server via TLS +(`ldaps://...` URL), add the following config setting to your config file: + +```xml + + + + + +``` + +The path is resolved relative to the config.xml file. From 328bdc99dd81eb90e20b71464cde9f38cf206217 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 29 Oct 2020 13:01:41 +0100 Subject: [PATCH 16/43] Changelog for LDAP TLS config --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6aa1eab..82ef81b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) - The [`ldap-query()` function](/reference/functions/ldap-query.md) +- [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) ### Fixed From cd0605106767eae9bdd90b61ea153e5a427cf509 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Wed, 4 Nov 2020 15:52:07 +0100 Subject: [PATCH 17/43] minor fixes in documentation of XPath function ldap-query() --- reference/functions/ldap-query.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/reference/functions/ldap-query.md b/reference/functions/ldap-query.md index 9519d85..452f4fc 100644 --- a/reference/functions/ldap-query.md +++ b/reference/functions/ldap-query.md @@ -1,4 +1,3 @@ -# `ldap-query()` ``` OXN-node-set ldap-query(string url, string rdn, string rdnPassword, string base_dn, string search, string attributes) @@ -6,8 +5,8 @@ OXN-node-set ldap-query(string url, string rdn, string rdnPassword, string base_ The `ldap-query()` function connects to an LDAP server with the given `url`, `rdn` and `rdnPassword`. It then performs a query by the given `search`. -An [OXN](/reference/templating/oxn.md) JSON array is returned with objects containing the found entities' `dn` and additional attributes given by `attributes`. -If no entities match the query an empty node-set is returned. +An [OXN](/reference/templating/oxn.md) JSON array is returned with objects containing `dn` and additional attributes given by `attributes` of all the entities that were found. +If no entities match the query, an empty node-set is returned. ## Parameters @@ -21,16 +20,17 @@ If no entities match the query an empty node-set is returned. ## Example -In the following example, the LDAP server is connected with the DN given in the `rdn` and `rdnPassword` POST parameters. -The given filter is used to search for an entry of a person which is a member of a group `Users` and sAMAccountName containg `doe`. -In addition to the (default) `dn`, the `sAMAccountName`, `displayName` and `mail` from the found entries are added to the results. +In the following example, FLAT connects to the LDAP server with the DN given in the `rdn` and `rdnPassword` POST parameters. +The given filter is used to search for an entry of a person which is a member of a group `Users` and sAMAccountName containing `doe`. +In addition to the (default) `dn`, the `sAMAccountName`, `displayName` and `mail` from the found entries are included to the results. ```xml concat("(&(objectClass=person)(memberOf=CN=Users,ou=People,dc=example,dc=com)(sAMAccountName=*doe*))") "sAMAccountName,displayName,mail" + "ldap://ad.example.com" - ldap-lookup($ldap_settings/url, $request/post/rdn, $request/post/rdnPassword, "dc=example,dc=com", $search, $attributes) + ldap-lookup($ldap_url, $request/post/rdn, $request/post/rdnPassword, "dc=example,dc=com", $search, $attributes) { "status": 403, From 1a2810848bf0fa7a64f2fbbb1b8bc2b721917843 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Wed, 4 Nov 2020 15:54:16 +0100 Subject: [PATCH 18/43] minor beautification for `ldap cacert-src=`... --- reference/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/configuration.md b/reference/configuration.md index 3e25d11..34e845c 100644 --- a/reference/configuration.md +++ b/reference/configuration.md @@ -36,7 +36,7 @@ The difference is, that you may use _Dynamic Attribute Values_ and if-clauses as If you use the [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or [`ldap-query()`](/reference/functions/ldap-query.md) function and connect to the LDAP server via TLS -(`ldaps://...` URL), add the following config setting to your config file: +(`ldaps://...` URL), you may have to provide the corresponding CA certificate using the following config setting in your config file: ```xml From 596b379ed495a937e9e3d4b4b35915fa0ecb1361 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Wed, 4 Nov 2020 16:00:53 +0100 Subject: [PATCH 19/43] ldap-query(): fix to/in --- reference/functions/ldap-query.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/functions/ldap-query.md b/reference/functions/ldap-query.md index 452f4fc..457fd10 100644 --- a/reference/functions/ldap-query.md +++ b/reference/functions/ldap-query.md @@ -22,7 +22,7 @@ If no entities match the query, an empty node-set is returned. In the following example, FLAT connects to the LDAP server with the DN given in the `rdn` and `rdnPassword` POST parameters. The given filter is used to search for an entry of a person which is a member of a group `Users` and sAMAccountName containing `doe`. -In addition to the (default) `dn`, the `sAMAccountName`, `displayName` and `mail` from the found entries are included to the results. +In addition to the (default) `dn`, the `sAMAccountName`, `displayName` and `mail` from the found entries are included in the results. ```xml From 20e37f1d1dea17fd8f18d45da512f85d4a12f29d Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Tue, 10 Nov 2020 17:48:58 +0100 Subject: [PATCH 20/43] Add documentation for token scope --- reference/OpenAPI/security.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/reference/OpenAPI/security.md b/reference/OpenAPI/security.md index c3f8e1e..32fdcac 100644 --- a/reference/OpenAPI/security.md +++ b/reference/OpenAPI/security.md @@ -14,12 +14,16 @@ The `x-flat-jwt` field references an object with fields describing the expected * `out-var` - The name of the variable in which the JWT is stored (must be a proper variable name, starting with `$`; default: `"$jwt"`). * `out-header` - The name of an HTTP request header that shall carry the JWT in upstream requests. * `claims` - An object with claims the JWT payload is expected to contain. The field names are the claim names, the expected claim value is specified either with a value, or by referencing a file (`file`) or an environment variable (`env`). +* `scope-claim` - The name of the claim, storing the scope of the token (string; default: `scope`). The claim value must either be + * a string containing a whitespace-separated list of scopes (`"scope1 scope2 scope3"`) or + * an array with a string entry for each scope (`["scope1", "scope2", "scope3"]`). The token is considered valid if all of the following are true: * the JWS can be decoded, * the JWS has a proper JWT, * the JWT is not expired, * the JWT contains the expected claims, if any are configured, +* the scope of the token contains all required scopes as specified in the security requirement, * the JWT can be stored in a variable. `$jwt` or the alternative variable specified in `out-var` and the header specified in `out-header` will be unset if the token is not valid. @@ -40,6 +44,7 @@ securityDefinitions: alg: env: FLAT_JWT_ALG out-var: $header_token + scope-claim: permission claims: aud: env: FLAT_JWT_AUDIENCE @@ -49,6 +54,7 @@ The token is expected to be a bearer token in the `Authorization` header. The key is read from a file named `secret.pem` relative to the `swagger.yaml`. The signing algorithm is read from the `FLAT_JWT_ALG` environment variable. The JWT will be stored in the `$header_token` variable. +The scope of the token is specified in a `permission` claim. The JWT payload is expected to contain an `aud` claim with a value read from the `FLAT_JWT_AUDIENCE` environment variable. If the request does not contain an `Authorization` header with the proper bearer structure, or the token is invalid, this security scheme will fail. @@ -98,9 +104,13 @@ paths: In Swagger, security schemes can be specified at the top level (default security) or for specific operations. With FLAT, you can also specify a security scheme for a specific path (default security for all operations on the path). -In the following example, a `GET` request to `/foo` must satisfy the security scheme named `JWTHeaderAuth`, while e.g. a `POST` or `PUT` request to `/foo` must satisfy the security scheme named `JWTCookieAuth`. +In the following example, a `GET` request to `/foo` must satisfy the security scheme named `JWTHeaderAuth`, +a `POST` request to `/foo` must satisfy the security scheme named `JWTHeaderAuth` and the token must have both `write:foo` and `read:bar` scopes, +while e.g. a `DELETE` or `PUT` request to `/foo` must satisfy the security scheme named `JWTCookieAuth`. All other requests must satisfy either the `JWTHeaderAuth` **or** `JWTCookieAuth` security schemes. +**Note:** While in Swagger 2.0, specifying scopes in security requirements is limited to OAuth2 security definitions, FLAT has no such limitation. + ```yaml # default security: @@ -115,6 +125,10 @@ paths: # specific for GET on /foo security: - JWTHeaderAuth: [] + post: + # specific for POST on /foo + security: + - JWTHeaderAuth: [ write:foo, read:bar ] ``` If a request does not pass the security check, it is rejected with status code `403` and error code `3206`, and the [error flow](/reference/OpenAPI/routing.md#error-flow) is run, if configured. @@ -135,6 +149,7 @@ securityDefinitions: alg: env: FLAT_JWT_ALG out-var: $header_token + scope-claim: permission claims: aud: env: FLAT_JWT_AUDIENCE @@ -156,6 +171,10 @@ paths: get: security: # token must be in Authorization header - JWTHeaderAuth: [] + post: + # specific for POST on /foo + security: + - JWTHeaderAuth: [ write:foo, read:bar ] ``` ## Forwarding JWT Upstream From 6800da7e3264a4153a9b15468299b62e463aacc1 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Tue, 10 Nov 2020 17:54:05 +0100 Subject: [PATCH 21/43] Changelog for token scope --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 82ef81b..3cce603 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) - The [`ldap-query()` function](/reference/functions/ldap-query.md) - [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) +- The [`scope-claim` property](/reference/OpenAPI/security.md#the-x-flat-jwt-field) +- Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) ### Fixed From 1931c92b1a6605a8362167589326847a686bf8d9 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Fri, 13 Nov 2020 19:10:42 +0100 Subject: [PATCH 22/43] actually Trigger an upstream-response-validation-error in the tutorial --- tutorial/README.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/tutorial/README.md b/tutorial/README.md index aadd160..7a08052 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -815,12 +815,13 @@ To abort the flow in case the upstream request or response is invalid or the req ``` -To see the effect, change the parameter name `language` to `lang` in upstream_request.xml: +To see the effect, shorten the `hello-world` of the repository name to just `hello` in upstream_request.xml: ```xml … - {{ concat("lang:", $request/params/language) }} - + "repo:leachim6/hello", + + … ] ``` @@ -847,7 +848,7 @@ HTTP/1.1 400 Bad Request … ``` ```json -{"error":{"message":"Upstream Request Validation Failed","status":400,"requestID":"main","info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."],"code":3202}} +{"error":{"message":"Upstream Response Validation Failed","status":502,"requestID":"main","info":["No definition for status code 422 and no 'default'.","Upstream status: 422 Unprocessable Entity"],"code":3203}} ``` If you prefer to provide a custom error document, you can configure an error flow. Just create `error.xml`: @@ -889,15 +890,16 @@ Error-Code: 3202 … ``` ```json -{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."]}} +{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."] ``` Now revert the change to upstream_request.xml: ```xml … - {{ concat("language:", $request/params/language) }} - + "repo:leachim6/hello-world", + + … ] ``` From 39cb8819c29883aa95389b26a3d786f1caf09aa8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 16 Nov 2020 08:51:47 +0100 Subject: [PATCH 23/43] Markdown --- tutorial/README.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/tutorial/README.md b/tutorial/README.md index 7a08052..2d0f0d9 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -815,7 +815,7 @@ To abort the flow in case the upstream request or response is invalid or the req ``` -To see the effect, shorten the `hello-world` of the repository name to just `hello` in upstream_request.xml: +To see the effect, shorten the `hello-world` of the repository name to just `hello` in `upstream_request.xml`: ```xml … @@ -827,8 +827,9 @@ To see the effect, shorten the `hello-world` of the repository name to just `hel ``` If we request our API -``` -curl -si localhost:8080/html + +```bash +$ curl -si localhost:8080/html ``` instead of the output @@ -836,8 +837,6 @@ instead of the output ``` HTTP/1.1 404 Not Found … -``` -``` {"error": "Unknown language"} ``` @@ -846,8 +845,6 @@ we now get ``` HTTP/1.1 400 Bad Request … -``` -```json {"error":{"message":"Upstream Response Validation Failed","status":502,"requestID":"main","info":["No definition for status code 422 and no 'default'.","Upstream status: 422 Unprocessable Entity"],"code":3203}} ``` @@ -888,12 +885,10 @@ HTTP/1.1 400 Bad Request … Error-Code: 3202 … -``` -```json {"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."] ``` -Now revert the change to upstream_request.xml: +Now revert the change to `upstream_request.xml`: ```xml … From 610965a286fdcdd01ca04ce5e4793000084e0745 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 16 Nov 2020 08:52:49 +0100 Subject: [PATCH 24/43] Re-insert removed braces --- tutorial/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorial/README.md b/tutorial/README.md index 2d0f0d9..d2ae07f 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -885,7 +885,7 @@ HTTP/1.1 400 Bad Request … Error-Code: 3202 … -{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."] +{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."]}} ``` Now revert the change to `upstream_request.xml`: From f8a3dc4d29cdd52059546efc3f554726f4b3c7b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 16 Nov 2020 08:59:59 +0100 Subject: [PATCH 25/43] Correct error message --- tutorial/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorial/README.md b/tutorial/README.md index d2ae07f..727fdb5 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -885,7 +885,7 @@ HTTP/1.1 400 Bad Request … Error-Code: 3202 … -{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello-world filename:html lang:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."]}} +{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello filename:html language:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."]}} ``` Now revert the change to `upstream_request.xml`: From c50d4b989382eec5404b8133127f9415a9002d98 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 16 Nov 2020 09:00:57 +0100 Subject: [PATCH 26/43] missing error.xml --- tutorial/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tutorial/README.md b/tutorial/README.md index 727fdb5..eb10e97 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -1192,3 +1192,24 @@ paths: items: type: array ``` + +### `error.xml` + +```xml + + + + { + "Status": {{ $error/status }}, + "Error-Code": {{ $error/code }} + } + + +``` From 54bd0fe4a7eb5c49150727fbabc0e469deb9010d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Afflerbach?= Date: Mon, 16 Nov 2020 09:05:19 +0100 Subject: [PATCH 27/43] Better code context --- tutorial/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tutorial/README.md b/tutorial/README.md index eb10e97..2c132c2 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -818,12 +818,12 @@ To abort the flow in case the upstream request or response is invalid or the req To see the effect, shorten the `hello-world` of the repository name to just `hello` in `upstream_request.xml`: ```xml - … + … + [ + "q=hello", "repo:leachim6/hello", … - ] - ``` If we request our API @@ -891,12 +891,12 @@ Error-Code: 3202 Now revert the change to `upstream_request.xml`: ```xml - … + … + [ + "q=hello", "repo:leachim6/hello-world", … - ] - ``` From 384f626516414c5e23c5d37e2b7b2da988aaf9b8 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Mon, 16 Nov 2020 15:14:16 +0100 Subject: [PATCH 28/43] move examples to github.com/sevenval/flat-tutorial-files, mention docker-compose --- tutorial/README.md | 158 ++------------------------------------------- 1 file changed, 7 insertions(+), 151 deletions(-) diff --git a/tutorial/README.md b/tutorial/README.md index 2c132c2..1e33691 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -33,6 +33,11 @@ $ sudo mv flat /usr/local/bin > You could also put `flat` into your `~/bin/` directory. If that is not > already in your `$PATH`, you can add it with `export PATH="$PATH:~/bin"`. +If you do not have `bash` installed on your system, you can use `docker-compose` as an alternative to the flat-cli. +In addition to all the configuration files used in this tutorial, the repository +[FLAT tutorial files](https://github.com/sevenval/flat-tutorial-files) also provides `docker-compose.yml` files to +get you started. Refer to the (accompanying documentation)[https://github.com/sevenval/flat-tutorial-files/blob/main/README.md] for more information. + ## Getting Started Let's create a workspace for our little project. We call it `hello-world` and create a directory with that name: @@ -1061,155 +1066,6 @@ Besides the built-in debug information, custom debug output can help understand … ``` -## Complete Configuration - -Here are the complete configuration files: - -### `hello.xml` - -```xml - - - - - - - - - - - - - {"error": "Unknown language"} - - - - -``` - -### `upstream_request.xml` - -```xml - - - - - { - "url": "https://api.github.com/search/code", - "query": {{ join(" ", $query_parameters) }}, - "options": { - "definition": "upstream.yaml", - "validate-request": true, - "validate-response": true, - "exit-on-error": true - } - } - - -``` - -### `swagger.yaml` +## Configuration Files -```yaml -swagger: "2.0" -info: - version: "1.0" - title: Hello World! -x-flat-validate: - request: true - response: true -x-flat-error: - flow: error.xml -paths: - /{language}: - get: - x-flat-flow: hello.xml - parameters: - - name: language - in: path - required: true - type: string - pattern: ^[a-zA-Z0-9]+$ - produces: - - application/json - responses: - 200: - description: URL to a "Hello World" snippet - schema: - type: object - required: - - url - properties: - url: - type: string - pattern: ^https://raw.githubusercontent.com/ -``` - -### `upstream.yaml` - -```yaml -swagger: "2.0" -info: - version: "1.0" - title: GitHub Search API -host: api.github.com -schemes: - - https -paths: - /search/code: - get: - parameters: - - name: q - in: query - required: true - type: string - pattern: ^hello repo:leachim6/hello-world filename:\w+ language:\w+$ - produces: - - application/json - responses: - 200: - description: GitHub code search results - schema: - type: object - required: - - total_count - - items - properties: - total_count: - type: integer - items: - type: array -``` - -### `error.xml` - -```xml - - - - { - "Status": {{ $error/status }}, - "Error-Code": {{ $error/code }} - } - - -``` +The complete configuration files can be found in the companion repository [FLAT tutorial files](https://github.com/sevenval/flat-tutorial-files). From d48e7b864f3d38d5af19a505a94d6be782713a6c Mon Sep 17 00:00:00 2001 From: Felix Hassert Date: Mon, 23 Nov 2020 12:32:11 +0100 Subject: [PATCH 29/43] request options: add link to timeout env vars --- reference/actions/request.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/actions/request.md b/reference/actions/request.md index d710ce3..8880ca9 100644 --- a/reference/actions/request.md +++ b/reference/actions/request.md @@ -289,9 +289,9 @@ Example: `uploads` with `src` or `value` The `options` property sets the request options. Its value must be a JSON object. The following options are valid: -* `timeout` - Maximum time in seconds for processing a request (type: `number`, default: `$FLAT_FETCH_DEFAULT_TIMEOUT`) -* `connect-timeout` - Maximum time in seconds for establishing a connection to a server (type: `number`, default: `$FLAT_FETCH_DEFAULT_CONNECT_TIMEOUT)`) -* `time-to-first-byte-timeout` - Maximum time in seconds for receiving the first byte (type: `integer`, default: `$FLAT_DEFAULT_TTFB_TIMEOUT`) +* `timeout` - Maximum time in seconds for processing a request (type: `number`, default: [`$FLAT_FETCH_DEFAULT_TIMEOUT`](/administration/configuration.md#request-timeouts)) +* `connect-timeout` - Maximum time in seconds for establishing a connection to a server (type: `number`, default: [`$FLAT_FETCH_DEFAULT_CONNECT_TIMEOUT`](/administration/configuration.md#request-timeouts)) +* `time-to-first-byte-timeout` - Maximum time in seconds for receiving the first byte (type: `integer`, default: [`$FLAT_DEFAULT_TTFB_TIMEOUT`](/administration/configuration.md#request-timeouts)) * `send-x-forwarded-for` - Whether to include the client's IP address in an `X-Forwarded-For` header (type: `boolean`, default `false`) * `set-response-headers` - The HTTP response header fields to set in the response (type: `object` with field name as key and `string` or array of `string` as value) * `tls-version` - The TLS version (valid values: `"auto"`, `"TLSv1"`, `"TLSv1.0"`, `"TLSv1.1"`, `"TLSv1.2"`) From 810146fe28f5b990d08df59ffd36eac6eda2f767 Mon Sep 17 00:00:00 2001 From: Felix Hassert Date: Mon, 23 Nov 2020 12:56:51 +0100 Subject: [PATCH 30/43] cookbook: add "see also" links to timeout article --- cookbook/request-timeout.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/cookbook/request-timeout.md b/cookbook/request-timeout.md index fef5472..f4cf2e5 100644 --- a/cookbook/request-timeout.md +++ b/cookbook/request-timeout.md @@ -16,3 +16,8 @@ In case we have to deal with a slow upstream system, the [various timeout option ``` Without that `timeout` option the request will be aborted after three seconds – FLAT's default timeout setting. + +## See also + +* [`request` action timeouts](/reference/actions/request.md#options) (Reference) +* [default timeouts](/administration/configuration.md#request-timeouts) (Administration) From 86ad7044bf51da30f2260b01cd99de09b428ad45 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 17 Dec 2020 14:04:42 +0100 Subject: [PATCH 31/43] Documentation of post-check-flow --- reference/OpenAPI/security.md | 39 +++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/reference/OpenAPI/security.md b/reference/OpenAPI/security.md index 32fdcac..233d265 100644 --- a/reference/OpenAPI/security.md +++ b/reference/OpenAPI/security.md @@ -17,6 +17,7 @@ The `x-flat-jwt` field references an object with fields describing the expected * `scope-claim` - The name of the claim, storing the scope of the token (string; default: `scope`). The claim value must either be * a string containing a whitespace-separated list of scopes (`"scope1 scope2 scope3"`) or * an array with a string entry for each scope (`["scope1", "scope2", "scope3"]`). +* `post-check-flow` - The path to a flow executed after the JWT is checked and found valid. In this flow, e.g. additional checks can be implemented (use the [`error` action](/reference/actions/error.md) to throw errors). The token is considered valid if all of the following are true: * the JWS can be decoded, @@ -162,6 +163,16 @@ securityDefinitions: key: env: FLAT_COOKIE_SECRET out-var: $cookie_token + JWTCookieAuth2: + type: apiKey + in: header + name: Cookie + x-flat-cookiename: authtoken2 + x-flat-jwt: + key: + env: FLAT_COOKIE_SECRET + out-var: $cookie_token2 + post-check-flow: check-cookie-jwt.xml security: # alternatives: token in Authorization header or authtoken cookie - JWTHeaderAuth: [] - JWTCookieAuth: [] @@ -175,6 +186,34 @@ paths: # specific for POST on /foo security: - JWTHeaderAuth: [ write:foo, read:bar ] + /projects/{p}: + get: + security: + - JWTCookieAuth2: [] + parameters: + - name: p + in: path + type: string + required: true +``` + +check-cookie-jwt.xml (checking whether the value of the `pid` JWT claim equals the `p` request path parameter): + +```xml + + + { + "token_id": {{ $cookie_token2/pid }}, + "path_id": {{ $request/params/p }} + } + + + { + "status": 401, + "message": "Token is not applicable for this project." + } + + ``` ## Forwarding JWT Upstream From 88f7e160271303c7078701c2423f7bab9153bd29 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 17 Dec 2020 14:05:01 +0100 Subject: [PATCH 32/43] Changelog entry for post-check-flow --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cce603..b1828f4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) - The [`ldap-query()` function](/reference/functions/ldap-query.md) - [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) -- The [`scope-claim` property](/reference/OpenAPI/security.md#the-x-flat-jwt-field) +- The [`scope-claim`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) and [post-check-flow](/reference/OpenAPI/security.md#the-x-flat-jwt-field) properties - Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) ### Fixed From 3cae5c0e58de494f765b3ee0dea3373982e4c5b9 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 17 Dec 2020 15:20:24 +0100 Subject: [PATCH 33/43] Cookbook for claims, scopes and post-check-flow --- cookbook/README.md | 2 + cookbook/checking-jwt-tokens.md | 264 ++++++++++++++++++++++++++++++++ cookbook/post-check-flow.md | 32 ++++ cookbook/x-flat-jwt.md | 2 + 4 files changed, 300 insertions(+) create mode 100644 cookbook/checking-jwt-tokens.md create mode 100644 cookbook/post-check-flow.md diff --git a/cookbook/README.md b/cookbook/README.md index 86d55de..3e94a1b 100644 --- a/cookbook/README.md +++ b/cookbook/README.md @@ -2,6 +2,8 @@ ## [Protecting Access using JWT Tokens](x-flat-jwt.md) +## [Performing Additional Checks on JWT Access Tokens](checking-jwt-tokens.md) + ## [Proxying Requests to Upstream APIs (Swagger)](proxy-requests.md) ## [Testing Templates](test-templates.md) diff --git a/cookbook/checking-jwt-tokens.md b/cookbook/checking-jwt-tokens.md new file mode 100644 index 0000000..11fddb2 --- /dev/null +++ b/cookbook/checking-jwt-tokens.md @@ -0,0 +1,264 @@ +# Performing Additional Checks on JWT Access Tokens + +Before you start, make sure that you read the cookbook [Protecting Access using JWT Tokens](x-flat-jwt.md). + +Let's start with the following swagger definition: + +swagger.yaml: + +```yaml +swagger: "2.0" +host: my-api.com +schemes: + - https +basePath: / +securityDefinitions: + JWTCookie: + type: apiKey + in: header + name: Cookie + x-flat-cookiename: authtoken + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + out-var: $jwt +security: + - JWTCookie: [] +paths: + /projects/{p}: + x-flat-flow: … + get: + parameters: + - name: p + in: path + description: The project identifier + type: string + required: true + patch: + parameters: + - name: p + in: path + description: The project identifier + type: string + required: true +``` + +The API has one endpoint with a path parameter `p` indicating the project identifier and two operations (`GET` and `PATCH`). The whole API is secured with a security scheme labelled "JWTCookie". + +FLAT will make sure that every request to this endpoint +* has a `Cookie` header +* with a value for the `authtoken` cookie +* that is a JWT +* properly signed and +* not expired. + +In addition to this, FLAT provides features for further checks: + +* [Claims](#checking-claims) +* [Scopes](#checking-scopes) +* [Post-check flow](#the-post-check-flow) + +## Checking Claims + +You can e.g. make sure that the token was issued by a certain token provider (`iss` claim) + +```yaml +… + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + out-var: $jwt + claims: + iss: "https://trustworthy-token-provider.com" # ⬅ the mandatory value for the iss claim +… +``` + +and that your API is (one of) the intended audience(s) for the token (`aud` claim) + +```yaml +… + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + out-var: $jwt + claims: + iss: "https://trustworthy-token-provider.com" + aud: "https://my-api.com/" # ⬅ a mandatory value for the aud claim +… +``` + +A JWT with the following claims will pass the test: + +```json +{ + "iss": "https://trustworthy-token-provider.com", + "aud": [ "https://my-api.com/", "https://a-different-api.org/" ], + … +} +``` + +while + +```json +{ + "iss": "https://the-reckless-token-provider.com", + "aud": [ "https://my-api.com/", "https://a-different-api.org/" ], + … +} +``` +or +```json +{ + "iss": "https://trustworthy-token-provider.com", + "aud": [ "https://a-different-api.org/" ], + … +} +``` +will **not** pass. + +## Checking Scopes + +Let's restrict the use of the `PATCH` operation to specially authorized requests. We can use scopes to achieve this: + +```yaml +… + patch: + security: + - JWTCookie: [ write ] + parameters: +… +``` + +FLAT will now look for a scope claim (default claim name is `scope`) with a value of `write`. If the `write` scope is present (perhaps among others, like in `"scope": "read write create"`), the request passes, otherwise it is rejected. + +BTW, you can specify another claim name for scopes using the `scope-claim` property of `x-flat-jwt`: + +```yaml +… + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + scope-claim: sc # ⬅ look for scopes in the sc JWT claim +… +``` + +## The post-check flow + +Lastly, we want to check that a certain non-standard JWT claim `pid` matches the path param `p` (the project identifier). + +We use the post-check-flow feature: + +```yaml +… + x-flat-jwt: + key: + file: pubkey.pem + … + post-check-flow: check-jwt.xml +… +``` +with check-jwt.xml: +```xml + + + + { + "token_id": {{ $jwt/pid }}, + "path_id": {{ $request/params/p }} + } + + + { + "status": 401, + "message": "Token is not applicable for this project." + } + + +``` + +A JWT with the claim + +```json +{ + … + "pid": "ABC123", + … +} +``` +will give access to `https://my-api.com/projects/ABC123`, but **not** to `https://my-api.com/projects/DEF456`. + + +## All files together + +swagger.yaml: +```yaml +swagger: "2.0" +host: my-api.com +schemes: + - https +basePath: / +securityDefinitions: + JWTCookie: + type: apiKey + in: header + name: Cookie + x-flat-cookiename: authtoken + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + claims: + iss: "https://trustworthy-token-provider.com" + scope-claim: sc # default: scope + out-var: $the_claims + out-header: JWT + post-check-flow: check-jwt.xml +security: + - JWTCookie: [] +paths: + /projects/{p}: + x-flat-flow: ... + get: + parameters: + - name: p + in: path + description: The project identifier + type: string + required: true + patch: + security: + - JWTCookie: [ write ] + parameters: + - name: p + in: path + description: The project identifier + type: string + required: true +``` + +check-jwt.xml: +```xml + + + + { + "token_id": {{ $jwt/pid }}, + "path_id": {{ $request/params/p }} + } + + + { + "status": 401, + "message": "Token is not applicable for this project." + } + + +``` + +## See also + +* [FLAT Security](/reference/OpenAPI/security.md) (reference) diff --git a/cookbook/post-check-flow.md b/cookbook/post-check-flow.md new file mode 100644 index 0000000..39ed0cc --- /dev/null +++ b/cookbook/post-check-flow.md @@ -0,0 +1,32 @@ +# Performing Additional Checks on JWT Access Tokens + +swagger.yaml: + +```yaml +swagger: "2.0" +basePath: / +securityDefinitions: + JWTCookie: + type: apiKey + in: header + name: Cookie + x-flat-cookiename: authtoken + x-flat-jwt: + key: + file: pubkey.pem + alg: RS256 + claims: + iss: "The token provider" + scope-claim: sc # default: scope + out-var: $jwt + post-check-flow: check-jwt.xml +paths: + /projects/{p}: + x-flat-flow: ... + get: + security: + - JWTCookie: [ read ] + patch: + security: + - JWTCookie: [ write ] +``` \ No newline at end of file diff --git a/cookbook/x-flat-jwt.md b/cookbook/x-flat-jwt.md index 7bd4d5a..ffe16d3 100644 --- a/cookbook/x-flat-jwt.md +++ b/cookbook/x-flat-jwt.md @@ -347,6 +347,8 @@ init.xml: ``` +If you want to know, how to perform some additional checks on the JWT, visit the cookbook [Performing Additional Checks on JWT Access Tokens](checking-jwt-tokens.md). + ## See also * [FLAT Security](/reference/OpenAPI/security.md) (reference) From 5dfcc2e802e12f2e5384b452384960b81bea29d9 Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Mon, 4 Jan 2021 14:23:43 +0100 Subject: [PATCH 34/43] document ldap timeout behaviout/configuration #47701-ldap-timeouts --- reference/configuration.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/reference/configuration.md b/reference/configuration.md index 34e845c..8508d29 100644 --- a/reference/configuration.md +++ b/reference/configuration.md @@ -47,3 +47,17 @@ If you use the [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or ``` The path is resolved relative to the config.xml file. + +## LDAP Timeouts + +LDAP requests via [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or +[`ldap-query()`](/reference/functions/ldap-query.md) use `FLAT_MAX_TIMEOUT` as the default timeout. +If you want to set a lower timeout for LDAP requests, use the setting below in your config file: + +```xml + + + + + +``` From 13682de4914f627a8e6d4113a1d4c5a099d87abc Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Mon, 4 Jan 2021 16:05:50 +0100 Subject: [PATCH 35/43] Corrected some errors --- tutorial/README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tutorial/README.md b/tutorial/README.md index 1e33691..74f9f23 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -369,10 +369,10 @@ Let's try again: $ curl --silent localhost:8080/..%2fswagger.yaml%23 | jq { "error": { - "message": "Input Validation Failed", + "message": "Input validation failed", … "info": [ - "Pattern constraint violated in path for language: Does not match the regex pattern ^[a-zA-Z0-9]+$." + "Pattern constraint violated in path for language: '../swagger.yaml#' does not match the pattern '^[a-zA-Z0-9]+$'." ] } } @@ -644,14 +644,14 @@ x-flat-validate: If we now alter the domain name in the second template of our flow to `flaw.githubusercontent.com`, we get a validation error: ```bash -$ curl --silent localhost:8080/javascript | jq +$ curl --silent localhost:8080/json | jq { "error": { - "message": "Output Validation Failed", + "message": "Client response validation failed", "status": 500, "requestID": "XNrzofUrpUX@vMuN6J31EwAAADQ", "info": [ - "Pattern constraint violated in body for code: Does not match the regex pattern ^https://raw.githubusercontent.com/." + "Pattern constraint violated in body for url: 'https://flaw.githubusercontent.com/leachim6/hello-world/a5df0ebf101fbb762604717ad10165ad7d4a5317/j/json.json' does not match the pattern '^https://raw.githubusercontent.com/'." ] } } @@ -848,7 +848,7 @@ HTTP/1.1 404 Not Found we now get ``` -HTTP/1.1 400 Bad Request +HTTP/1.1 502 Bad Gateway … {"error":{"message":"Upstream Response Validation Failed","status":502,"requestID":"main","info":["No definition for status code 422 and no 'default'.","Upstream status: 422 Unprocessable Entity"],"code":3203}} ``` @@ -886,11 +886,11 @@ x-flat-error: # ⬅ We now get ``` -HTTP/1.1 400 Bad Request +HTTP/1.1 502 Bad Gateway … Error-Code: 3202 … -{"CustomError":{"Message":"Upstream Request Validation Failed","Info":["Pattern constraint violated in query for q: 'hello repo:leachim6\/hello filename:html language:html' does not match the pattern '^hello repo:leachim6\/hello-world filename:\\w+ language:\\w+$'."]}} +{"CustomError":{"Message":"Upstream Response Validation Failed","Info":["No definition for status code 422 and no 'default'.","Upstream status: 422 Unprocessable Entity"]}} ``` Now revert the change to `upstream_request.xml`: From b5bbb0ac78c1e1cdf1ce386929e8014bde0aab19 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Mon, 4 Jan 2021 16:27:57 +0100 Subject: [PATCH 36/43] Timeout (singular) --- reference/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/configuration.md b/reference/configuration.md index 8508d29..abbe271 100644 --- a/reference/configuration.md +++ b/reference/configuration.md @@ -48,7 +48,7 @@ If you use the [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or The path is resolved relative to the config.xml file. -## LDAP Timeouts +## LDAP Timeout LDAP requests via [`ldap-lookup()`](/reference/functions/ldap-lookup.md) or [`ldap-query()`](/reference/functions/ldap-query.md) use `FLAT_MAX_TIMEOUT` as the default timeout. From 289371bb8be4b9f298364617b56c9c5bee542d17 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Mon, 4 Jan 2021 16:28:14 +0100 Subject: [PATCH 37/43] Changelog entry --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cce603..a21b2c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) - The [`ldap-query()` function](/reference/functions/ldap-query.md) -- [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) +- [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) and [LDAP timeout](/reference/configuration.md#ldap-timeout) - The [`scope-claim` property](/reference/OpenAPI/security.md#the-x-flat-jwt-field) - Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) From 65f8c9861d08c17f0f8e1d3dba8604bd08c8dadb Mon Sep 17 00:00:00 2001 From: "Rainer M. Canavan" Date: Tue, 5 Jan 2021 16:03:42 +0100 Subject: [PATCH 38/43] minor improvements to checking-jwt-tokens #47661-post-check-flow --- cookbook/checking-jwt-tokens.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/cookbook/checking-jwt-tokens.md b/cookbook/checking-jwt-tokens.md index 11fddb2..43749f7 100644 --- a/cookbook/checking-jwt-tokens.md +++ b/cookbook/checking-jwt-tokens.md @@ -1,6 +1,6 @@ # Performing Additional Checks on JWT Access Tokens -Before you start, make sure that you read the cookbook [Protecting Access using JWT Tokens](x-flat-jwt.md). +Before you start, make sure that you have read the cookbook chapter [Protecting Access using JWT Tokens](x-flat-jwt.md). Let's start with the following swagger definition: @@ -61,7 +61,7 @@ In addition to this, FLAT provides features for further checks: ## Checking Claims -You can e.g. make sure that the token was issued by a certain token provider (`iss` claim) +For example, you can ensure that the token was issued by a specific token provider (`iss` claim) ```yaml … @@ -132,7 +132,7 @@ Let's restrict the use of the `PATCH` operation to specially authorized requests … ``` -FLAT will now look for a scope claim (default claim name is `scope`) with a value of `write`. If the `write` scope is present (perhaps among others, like in `"scope": "read write create"`), the request passes, otherwise it is rejected. +FLAT will now look for a scope claim (default claim name is `scope`) with a value of `write`. If the `write` scope is present (possibly along with further scopes, like in `"scope": "read write create"`), the request passes, otherwise it is rejected. BTW, you can specify another claim name for scopes using the `scope-claim` property of `x-flat-jwt`: @@ -148,7 +148,7 @@ BTW, you can specify another claim name for scopes using the `scope-claim` prope ## The post-check flow -Lastly, we want to check that a certain non-standard JWT claim `pid` matches the path param `p` (the project identifier). +Finally, we want to check that a certain non-standard JWT claim `pid` matches the path param `p` (the project identifier). We use the post-check-flow feature: @@ -189,7 +189,7 @@ A JWT with the claim … } ``` -will give access to `https://my-api.com/projects/ABC123`, but **not** to `https://my-api.com/projects/DEF456`. +will permit access to `https://my-api.com/projects/ABC123`, but **not** to `https://my-api.com/projects/DEF456`. ## All files together From 1412883d610220080aaeb36b5c0d20317b985380 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Wed, 6 Jan 2021 08:49:10 +0100 Subject: [PATCH 39/43] code markup for property --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b2573ec..4c408e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ - The [`uuid3()` and `uuid4()` functions](/reference/functions/uuid.md) - The [`ldap-query()` function](/reference/functions/ldap-query.md) - [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) and [LDAP timeout](/reference/configuration.md#ldap-timeout) -- The [`scope-claim`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) and [post-check-flow](/reference/OpenAPI/security.md#the-x-flat-jwt-field) properties +- The [`scope-claim`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) and [`post-check-flow`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) properties - Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) ### Fixed From 2e2bedb9bfe03d47104f3f9bb21c56c015c58247 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Wed, 6 Jan 2021 12:49:16 +0100 Subject: [PATCH 40/43] changelog entry for merging directives into php.ini --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c408e8..1fadf37 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ - [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) and [LDAP timeout](/reference/configuration.md#ldap-timeout) - The [`scope-claim`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) and [`post-check-flow`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) properties - Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) +- [Merging directives into `php.ini`](/administration/configuration.md#phpini) via environment variables ### Fixed From ec034258563f756977b15ad69ef731b5f7bf7152 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 7 Jan 2021 11:12:10 +0100 Subject: [PATCH 41/43] Release 20210107 --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fadf37..10446d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -## Unreleased +## [20210107](https://hub.docker.com/r/sevenvaltechnologies/flatrunner/tags) ### Added From 970071ac35a23ac18ea714ece0a5cf10ee719c94 Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 7 Jan 2021 11:21:56 +0100 Subject: [PATCH 42/43] Release 20210107 (2) --- CHANGELOG.md | 2 +- reference/functions/ldap-query.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10446d1..582d756 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ - [LDAP TLS configuration](/reference/configuration.md#ldap-tls-configuration) and [LDAP timeout](/reference/configuration.md#ldap-timeout) - The [`scope-claim`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) and [`post-check-flow`](/reference/OpenAPI/security.md#the-x-flat-jwt-field) properties - Specifying the [required token scope](/reference/OpenAPI/security.md#applying-security-schemes) -- [Merging directives into `php.ini`](/administration/configuration.md#phpini) via environment variables +- [Merging directives into `php.ini`](/administration/configuration.md#php-ini) via environment variables ### Fixed diff --git a/reference/functions/ldap-query.md b/reference/functions/ldap-query.md index 457fd10..c2b886e 100644 --- a/reference/functions/ldap-query.md +++ b/reference/functions/ldap-query.md @@ -1,3 +1,4 @@ +# `ldap-query()` ``` OXN-node-set ldap-query(string url, string rdn, string rdnPassword, string base_dn, string search, string attributes) From e96e5de3a5f5807552031836b42bce26970cca6b Mon Sep 17 00:00:00 2001 From: Johannes Koch Date: Thu, 7 Jan 2021 11:38:29 +0100 Subject: [PATCH 43/43] Release 20210107 (3) --- tutorial/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorial/README.md b/tutorial/README.md index 74f9f23..8c1a522 100644 --- a/tutorial/README.md +++ b/tutorial/README.md @@ -369,7 +369,7 @@ Let's try again: $ curl --silent localhost:8080/..%2fswagger.yaml%23 | jq { "error": { - "message": "Input validation failed", + "message": "Client request validation failed", … "info": [ "Pattern constraint violated in path for language: '../swagger.yaml#' does not match the pattern '^[a-zA-Z0-9]+$'."