Skip to content

Commit

Permalink
add to usage help for all targets and subsystems
Browse files Browse the repository at this point in the history
  • Loading branch information
lilatomic committed Dec 25, 2024
1 parent 0c5b3aa commit 92c645e
Show file tree
Hide file tree
Showing 12 changed files with 39 additions and 31 deletions.
4 changes: 2 additions & 2 deletions docs/docs/jvm/java-and-scala.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -330,11 +330,11 @@ Test runs are _hermetic_, meaning that they are stripped of the parent `pants` p

To add any arbitrary environment variable back to the process, you can either add the environment variable to the specific tests with the `extra_env_vars` field on `junit_test` / `junit_tests` / `scala_junit_test` / `scala_junit_tests` / `scalatest_test` / `scalatest_tests` targets or to all your tests with the `[test].extra_env_vars` option. Generally, prefer the field `extra_env_vars` field so that more of your tests are hermetic.

With both `[test].extra_env_vars` and the `extra_env_vars` field, you can either hardcode a value or leave off a value to "allowlist" it and read from the parent `pants` process's environment.
With both `[test].extra_env_vars` and the `extra_env_vars` field, you can specify a value with the form `"VAR1=hardcoded_value"` or read it from the parent `pants` process's environment with the form `VAR2`. `fnmatch` globs like `"VAR_PREFIXED_*"` can be used to read multiple environment variables.

```toml tab={"label":"pants.toml"}
[test]
extra_env_vars = ["VAR1", "VAR2=hardcoded_value"]
extra_env_vars = ["VAR1=hardcoded_value", "VAR2", "VAR_PREFIXED_*"]
```

```python tab={"label":"project/BUILD"}
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/jvm/kotlin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -205,11 +205,11 @@ Test runs are _hermetic_, meaning that they are stripped of the parent `pants` p

To add any arbitrary environment variable back to the process, you can either add the environment variable to the specific tests with the `extra_env_vars` field on `kotlin_junit_test` / `kotlin_junit_tests` targets or to all your tests with the `[test].extra_env_vars` option. Generally, prefer the field `extra_env_vars` field so that more of your tests are hermetic.

With both `[test].extra_env_vars` and the `extra_env_vars` field, you can either hardcode a value or leave off a value to "allowlist" it and read from the parent `pants` process's environment.
With both `[test].extra_env_vars` and the `extra_env_vars` field, you can specify a value with the form `"VAR1=hardcoded_value"` or read it from the parent `pants` process's environment with the form `VAR2`. `fnmatch` globs like `"VAR_PREFIXED_*"` can be used to read multiple environment variables.

```toml tab={"label":"pants.toml"}
[test]
extra_env_vars = ["VAR1", "VAR2=hardcoded_value"]
extra_env_vars = ["VAR1=hardcoded_value", "VAR2", "VARS_PREFIXED_*"]
```

```python tab={"label":"project/BUILD"}
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/python/goals/test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,11 @@ Test runs are _hermetic_, meaning that they are stripped of the parent `pants` p

To add any arbitrary environment variable back to the process, you can either add the environment variable to the specific tests with the `extra_env_vars` field on `python_test` / `python_tests` targets or to all your tests with the `[test].extra_env_vars` option. Generally, prefer the field `extra_env_vars` field so that more of your tests are hermetic.

With both `[test].extra_env_vars` and the `extra_env_vars` field, you can either hardcode a value or leave off a value to "allowlist" it and read from the parent `pants` process's environment.
With both `[test].extra_env_vars` and the `extra_env_vars` field, you can specify a value with the form `"VAR1=hardcoded_value"` or read it from the parent `pants` process's environment with the form `VAR2`. `fnmatch` globs like `"VAR_PREFIXED_*"` can be used to read multiple environment variables.

```toml tab={"label":"pants.toml"}
[test]
extra_env_vars = ["VAR1", "VAR2=hardcoded_value"]
extra_env_vars = ["VAR1=hardcoded_value", "VAR2", "VAR_PREFIXED_*"]
```

```python tab={"label":"project/BUILD"}
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/shell/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -302,11 +302,11 @@ To force your tests to run again, rather than reading from the cache, run `pants

Test runs are _hermetic_, meaning that they are stripped of the parent `pants` process's environment variables. This is important for reproducibility, and it also increases cache hits.

To add any arbitrary environment variable back to the process, use the option `extra_env_vars` in the `[test]` options scope. You can hardcode a value for the option, or leave off a value to "allowlist" it and read from the parent `pants` process's environment.
To add any arbitrary environment variable back to the process, use the option `extra_env_vars` in the `[test]` options scope. You can specify a value with the form `"VAR1=hardcoded_value"` or read it from the parent `pants` process's environment with the form `VAR2`. `fnmatch` globs like `"VAR_PREFIXED_*"` can be used to read multiple environment variables.

```toml title="pants.toml"
[test]
extra_env_vars = ["VAR1", "VAR2=hardcoded_value"]
extra_env_vars = ["VAR1=hardcoded_value", "VAR2", "VAR_PREFIXED_*"]
```

Use `[bash-setup].executable_search_paths` to change the `$PATH` env var used during test runs. You can use the special string `"<PATH>"` to read the value from the parent `pants` process's environment.
Expand Down
2 changes: 1 addition & 1 deletion docs/notes/2.25.x.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Thank you to [Klayvio](https://www.klaviyo.com/) and [Normal Computing](https://
- Fixed a longstanding bug in the processing of [synthetic targets](https://www.pantsbuild.org/2.24/docs/writing-plugins/the-target-api/concepts#synthetic-targets-api). This fix has the side-effect of requiring immutability and hashability of scalar values in BUILD files, which was always assumed but not enforced. This may cause BUILD file parsing errors, if you have custom field types involving custom mutable data structures. See ([#21725](https://github.com/pantsbuild/pants/pull/21725)) for more.
- [Fixed](https://github.com/pantsbuild/pants/pull/21665) bug where `pants --export-resolve=<resolve> --export-py-generated-sources-in-resolve=<resolve>` fails (see [#21659](https://github.com/pantsbuild/pants/issues/21659) for more info).
- [Fixed](https://github.com/pantsbuild/pants/pull/21694) bug where an `archive` target is unable to produce a ZIP file with no extension (see [#21693](https://github.com/pantsbuild/pants/issues/21693) for more info).
- `[subprocess-environment].env_vars` and `extra_env_vars` (on many subsystems) now supports a generalised glob syntax using Python [fnmatch](https://docs.python.org/3/library/fnmatch.html) to construct patterns like `AWS_*`, `TF_*`, and `S2TESTS_*`.
- `[subprocess-environment].env_vars` and `extra_env_vars` (on many subsystems and targets) now supports a generalised glob syntax using Python [fnmatch](https://docs.python.org/3/library/fnmatch.html) to construct patterns like `AWS_*`, `TF_*`, and `S2TESTS_*`.

#### Remote Caching/Execution

Expand Down
6 changes: 3 additions & 3 deletions src/python/pants/backend/adhoc/target_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from pants.base.glob_match_error_behavior import GlobMatchErrorBehavior
from pants.core.util_rules.adhoc_process_support import PathEnvModifyMode
from pants.core.util_rules.environments import EnvironmentField
from pants.engine.env_vars import EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.fs import GlobExpansionConjunction
from pants.engine.process import ProcessCacheScope
from pants.engine.target import (
Expand Down Expand Up @@ -191,11 +192,10 @@ class AdhocToolTimeoutField(IntField):
class AdhocToolExtraEnvVarsField(StringSequenceField):
alias: ClassVar[str] = "extra_env_vars"
help = help_text(
"""
f"""
Additional environment variables to provide to the process.
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
{EXTRA_ENV_VARS_USAGE_HELP}
"""
)

Expand Down
5 changes: 4 additions & 1 deletion src/python/pants/backend/helm/subsystems/helm.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from pants.backend.helm.resolve.remotes import HelmRemotes
from pants.backend.helm.target_types import HelmChartTarget, HelmRegistriesField
from pants.core.util_rules.external_tool import TemplatedExternalTool
from pants.engine.env_vars import EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.platform import Platform
from pants.option.option_types import (
ArgsListOption,
Expand Down Expand Up @@ -153,9 +154,11 @@ class HelmSubsystem(TemplatedExternalTool):
)
extra_env_vars = StrListOption(
help=softwrap(
"""
f"""
Additional environment variables that would be made available to all Helm processes
or during value interpolation.
{EXTRA_ENV_VARS_USAGE_HELP}
"""
),
advanced=True,
Expand Down
6 changes: 3 additions & 3 deletions src/python/pants/backend/javascript/package_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
from pants.core.util_rules import stripped_source_files
from pants.engine import fs
from pants.engine.collection import Collection, DeduplicatedCollection
from pants.engine.env_vars import EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.fs import (
CreateDigest,
DigestContents,
Expand Down Expand Up @@ -416,11 +417,10 @@ class NodeBuildScriptExtraEnvVarsField(StringSequenceField):
required = False
default = ()
help = help_text(
"""
f"""
Additional environment variables to include in environment when running a build script process.
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
{EXTRA_ENV_VARS_USAGE_HELP}
"""
)

Expand Down
7 changes: 3 additions & 4 deletions src/python/pants/backend/python/providers/pyenv/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
TemplatedExternalTool,
)
from pants.core.util_rules.external_tool import rules as external_tools_rules
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest, EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.fs import CreateDigest, FileContent
from pants.engine.internals.native_engine import Digest, MergeDigests
from pants.engine.internals.selectors import Get, MultiGet
Expand Down Expand Up @@ -67,11 +67,10 @@ class PyenvPythonProviderSubsystem(TemplatedExternalTool):
class EnvironmentAware:
installation_extra_env_vars = StrListOption(
help=softwrap(
"""
f"""
Additional environment variables to include when running `pyenv install`.
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
{EXTRA_ENV_VARS_USAGE_HELP}
This is especially useful if you want to use an optimized Python (E.g. setting
`PYTHON_CONFIGURE_OPTS='--enable-optimizations --with-lto'` and
Expand Down
6 changes: 4 additions & 2 deletions src/python/pants/backend/terraform/tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
ExternalToolRequest,
TemplatedExternalTool,
)
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest, EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.fs import EMPTY_DIGEST, Digest
from pants.engine.internals.selectors import Get
from pants.engine.platform import Platform
Expand Down Expand Up @@ -359,8 +359,10 @@ def default_known_versions(cls):

extra_env_vars = StrListOption(
help=softwrap(
"""
f"""
Additional environment variables that would be made available to all Terraform processes.
{EXTRA_ENV_VARS_USAGE_HELP}
"""
),
advanced=True,
Expand Down
17 changes: 8 additions & 9 deletions src/python/pants/core/goals/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
from pants.engine.console import Console
from pants.engine.desktop import OpenFiles, OpenFilesRequest
from pants.engine.engine_aware import EngineAwareReturnType
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest
from pants.engine.env_vars import EnvironmentVars, EnvironmentVarsRequest, EXTRA_ENV_VARS_USAGE_HELP
from pants.engine.fs import EMPTY_FILE_DIGEST, Digest, FileDigest, MergeDigests, Snapshot, Workspace
from pants.engine.goal import Goal, GoalSubsystem
from pants.engine.internals.session import RunId
Expand Down Expand Up @@ -517,10 +517,10 @@ def activated(cls, union_membership: UnionMembership) -> bool:
class EnvironmentAware:
extra_env_vars = StrListOption(
help=softwrap(
"""
f"""
Additional environment variables to include in test processes.
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
{EXTRA_ENV_VARS_USAGE_HELP}
"""
),
)
Expand Down Expand Up @@ -745,13 +745,12 @@ def calculate_from_global_options(self, test: TestSubsystem) -> Optional[int]:
class TestExtraEnvVarsField(StringSequenceField, metaclass=ABCMeta):
alias = "extra_env_vars"
help = help_text(
"""
Additional environment variables to include in test processes.
f"""
Additional environment variables to include in test processes.
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
{EXTRA_ENV_VARS_USAGE_HELP}
This will be merged with and override values from `[test].extra_env_vars`.
This will be merged with and override values from `[test].extra_env_vars`.
"""
)

Expand Down
5 changes: 5 additions & 0 deletions src/python/pants/engine/env_vars.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@
name_value_re = re.compile(r"([A-Za-z_]\w*)=(.*)")
shorthand_re = re.compile(r"([A-Za-z_]\w*)")

EXTRA_ENV_VARS_USAGE_HELP = """\
Entries are strings in the form `ENV_VAR=value` to use explicitly; or just
`ENV_VAR` to copy the value of a variable in Pants's own environment.
`fnmatch` globs like `ENV_VAR_PREFIXED_*` can be used to copy multiple environment variables.
"""

class CompleteEnvironmentVars(FrozenDict):
"""CompleteEnvironmentVars contains all environment variables from the current Pants process.
Expand Down

0 comments on commit 92c645e

Please sign in to comment.