Pantheon Secrets Support

[dustinleblanc]

Friends, Countryman, Beltalowda,

It is with great sadness that we see the sunsetting of the great and useful Lockr.io, and our team is beginning the transition to Pantheon's shiny new secrets system. Lockr was so well integrated into the Pantheon recipe due to the hard work of @pirog and Chris at Lockr, it would be great to reproduce such awesomeness with the new native secrets system.

Alas, I know time is money, and as Sweet Brown once said, Ain't nobody got time for that! So I figured as a first duty, lets get the issue open, and then see what folks at Pantheon (@greg.1.anderson or @kporras07?) might be willing to give us some hints on implementation. Source code for their module is here: https://git.drupalcode.org/project/pantheon_secrets

Lets see if the community can come together to get this feature handled.

[reynoldsalec]

Lol thanks @dustinleblanc for bumping this!

I would say with the Lando v4 service API nearing feature completeness we're on the cusp of redoing all services/recipes, so my personal excitement may be lower until that happens, but would love to get it moving.

[greg-1-anderson]

@dustinleblanc What is the goal here? The Pantheon Secrets service was not designed to manage local secrets. The only way that secrets could be visible to Lando would be if they were given the scope "user", or if you added your own "get secrets" REST endpoint that ran on your Pantheon environment, and returned values via pantheon_get_secret() (which only works when running on Pantheon).

We could consider simple adjustments to Pantheon secrets, e.g. perhaps automatically removing the "user" scope for environment overrides of the live environment. Then, Lando users would need to know to apply the "user" scope to secrets they wanted to share with a local development environment, and Lando would also need a Terminus machine token to fetch secrets via an alternate path when running locally.

Disclaimer: this is brainstorming on what is possible / desired, not a commitment for development.

[dustinleblanc]

@greg-1-anderson thanks for chiming in. The goal is the "most seamless possible experience for devs when setting up a site and pulling it down", which is more or less what the Pantheon recipe for Lando has been doing for years now.

I personally think that even if a dev had to do something like update their .lando.yml to specify the secrets and store them in an .env or something, maybe that would still be a fairly intuitive experience? I'd love to just help ease the support in whatever way requires the least changes outside of the plugin code that is possible and if there is a way this can be achieved without the end developer having to modify the site code (even via settings.local.php if it could be helped). Even if there were a terminus way to get the secrets values and exfiltrate them to local and Lando could hook into that and just setup the appserver container so that the pantheon_get_secret() function would work.

The existing Lockr integration worked by lando getting its hands on the binding.pem and then essentially spoofing that it was on a Pantheon server (as best as I remember the code), which allowed Lockr to accept requests from the dev's local machine. This made things fairly seamless once a dev authenticated with their Terminus token.

TLDR, there are probably a number of ways to tackle this, but I just suspect there are many who will appreciate the tackling, and therefore I am glad the discussion hath commenced.

[lwalley]

@dustinleblanc if the requirement is just to fake secrets locally then https://docs.pantheon.io/guides/secrets/local provides manual steps for integrating Pantheon Secrets PHP SDK with Lando.

Assuming the Drupal module you referenced uses the PHP SDK (seems to) and the factory method, then as long as CUSTOMER_SECRETS_FAKE_FILE is set in .lando.yml then you can fake the secrets locally, it works seamlessly, example usage:

$ lando drush php
>>> use PantheonSystems\CustomerSecrets\CustomerSecrets;
>>> $client = CustomerSecrets::create()->getClient();
=> PantheonSystems\CustomerSecrets\CustomerSecretsFakeClient {#18502}
>>> $secret = $client->getSecret('foo');
=> PantheonSystems\CustomerSecrets\Secret {#18496}
>>> $secret_value = $secret->getValue();
=> "bar"

In terms of Lando Pantheon Plugin adjustments to support the manual integration, it would be nice if we could:

Use lando pull --secrets instead of terminus secret:site:local-generate.

Use config instead of CUSTOMER_SECRETS_FAKE_FILE e.g.:

config:
  secrets: /app/secrets.json

Having said all that, it sounds like Lockr integration may have been allowing access to real secrets? If so, I don't think that requirement would be satisfied by the fake client, unless the secret is in the user scope; for the web scope the values are not included (set to null) when generating the local file.

For our use case we prefer not to have the actual web secrets in our local environments. Editing the secrets.json to add in some fake local values for development purposes, is straightforward. However, if you need to do this in bulk then it would be useful to have built in way to inject overrides for the local file e.g., values could be set in config or some other local override file, and injected into the secrets.json on lando pull --secrets e.g.,:

config:
  secrets:
    file: /app/secrets.json
    values:
      foo: bar

Outcome is the value bar (which would otherwise be null) is injected into secret foo in the generated secrets.json:

{
  "EntityType": "sites",
  "EntityId": "<site uuid>",
  "Secrets": {
    "foo": {
      "Value": "bar",
      "Type": "runtime",
      "Scopes": [
        "web"
      ]
    }
  }
}

The above approach is not secure for cases where values can't be faked e.g., API keys to third party services. To make it more robust, short of opening up Pantheon Secrets service to manage local secrets (which it was not designed to do as noted by @greg-1-anderson), it would good if the locally generated file could be encrypted at rest, and perhaps only available on the Lando container i.e., not in the local mount; similar to how the secrets cache works in temporary directory on the Pantheon containers. In this case having lando secrets edit similar to how Ansible vault works could be useful.

[dustinleblanc]

@lwalley Sorry just saw this today, very cool! Thanks for doing all that research. You mentioned Ansible Vault, we use 1password as a company so some kind of use of 1password secrets service would also be quite cool.