test run with hatch test fails for lack of dependency that is included in pyproject.toml

[ajprax]

I'm working on a library that includes some optional dependencies. At test time, these dependencies are required to test the optional code.

I have the dependencies included in pyproject.toml thusly:

[tool.hatch.envs.test]
dependencies = ["crcmod", "tqdm", "watchdog"]

(crcmod and tqdm are optional dependencies of the library, watchdog is used in a simple test watcher)

If I run hatch env show I see the dependencies properly included in the test environment, and when I run hatch -e test run python I can import them, but when I run hatch test they fail with an import error.

To add further confusion, tqdm works properly, while crcmod (and numpy from further testing on another branch) both fail to import during tests. tqdm was the first added, but I haven't been able to find any other differences between how it and the others were added.

This branch has the issue.

[jpgoldberg]

I believe that hatch test is going to run tests in the default environment. Your dependencies are specified for your test environment.

[ajprax]

Ah, I guess I forgot that the test environment was created as a step in a tutorial and not as a base part of hatch, otherwise it would be surprising for the test command to not use the test environment.

I just tried hatch -e test test and it still failed. I then tried adding crcmod to the base dependencies list in pyproject.toml after which hatch test and hatch -e test test both passed. I removed the crcmod dependency and both test commands still passed. I tried variations of hatch run python and hatch test which give inconsistent results

• hatch run python cannot import tqdm or crcmod
• hatch -e test run python can import tqdm and crcmod
• hatch test tests pass despite tqdm and crcmod both being imported by tests
• hatch -e test test tests pass now as expected, but seemingly because of tqdm and crcmod having been temporarily included in base dependencies rather than because the test environment is properly configured.

Somehow it seems like hatch test with or without an explicit environment uses a third hidden environment that is neither default nor test which adds dependencies that are included in base dependencies, but does not remove them. This could also explain why the tqdm test passed previously while crcmod failed, assuming I had temporarily included tqdm in base dependencies and removed it (I don't remember, but wouldn't rule it out).

[jpgoldberg]

@ajprax, I believe you are correct when you say

Somehow it seems like hatch test with or without an explicit environment uses a third hidden environment that is neither default nor test which adds dependencies that are included in base dependencies, but does not remove them.

Looking at pydoc hatch.cli.test I see

Run tests using the hatch-test environment matrix.

If no filtering options are selected, then tests will be run in the first compatible environment
found in the matrix with priority given to those matching the current interpreter.

And a whole lot more.

I suspect from that documentation, you can set your dependencies in [tool.hatch.env.hatch-test], but I have not tested.