Passing file descriptors to Eleventy

[thoughtsunificator]

Linux
Eleventy 3.0.0

npx @11ty/eleventy --input=<(echo "foo")

Yields:

[11ty] Wrote 0 files in 0.02 seconds (v3.0.0)

Which mostly translates to eleventy ignoring the input file.

As an analogy, the following works as is:
node -p 'fs.readFileSync(process.argv[1])' <(echo foo)

<Buffer 66 6f 6f 0a>

Looks like the programatic API has the same behavior, so it's most likely not related to Eleventy CLI.

[Ryuno-Ki]

I should also note that the package is question is not actively maintained.

I am noticing several of those (but this is an unrelated issue). Given that Eleventy tends to require fewer and fewer packages over the versions I'm positive we can find a solution to this.

[Ryuno-Ki]

On my Debian Linux, when I log process.argv before it is handed to minimist in

https://github.com/11ty/eleventy/blob/v3.0.0/cmd.cjs#L33

I can see that <(echo "foo()") is evaluated to /dev/fd/63, that is, a file descriptor.

So I created a small test file

var argv = require("minimist")(process.argv.slice(2));
console.log(argv);

in the spirit of the official example

and run it with node test.cjs --input=<(echo "foo()") to observe the same result.

I don't see anything specific to handle file descriptors in Minimist's codebase.

[Ryuno-Ki]

The Programmatic API shows the same behaviour:

import Eleventy from "@11ty/eleventy"

let elev = new Eleventy("/dev/fd/63");
await elev.write();

invoked with node test.mjs <(echo "foo()").

That's going to be a nice rabbit hole to go down to! (After dinner)

[Ryuno-Ki]

TL;DR: fast-glob cannot see the file descriptor.

import fg from "fast-glob";

const entries = await fg(["/dev/fd/63"]);
console.log(entries);

Long version

1. When constructing, Eleventy is holding the input path in rawInput:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L104

2. It is then calling its init method:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L104

3. This is invoking its config initialization (among other things)

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L483

4. Within here, the directories getter is called:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L234

5. This is setting the rawInput to a new ProjectDirectories instance:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L301

6. Internally this is deferring to a method:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Util/ProjectDirectories.js#L174

7. On Linux machines, everything is a file (but a file descriptor is not a directory) as asserted by

https://github.com/11ty/eleventy/blob/v3.0.0/src/Util/ProjectDirectories.js#L137

8. Now that the config is initialised it also initialises a writer. TemplateWriter, since we're outputting HTML:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L530

9. That one is holding a reference to its Eleventy Config:

https://github.com/11ty/eleventy/blob/v3.0.0/src/TemplateWriter.js#L31

Now we're done with the initialisation side.

Let's look at the write.

1. Our Eleventy instance is invoking the method:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L1256

2. This is deferring the call to the TemplateWriter:

https://github.com/11ty/eleventy/blob/v3.0.0/src/Eleventy.js#L1334

3. Which is collecting the paths:

https://github.com/11ty/eleventy/blob/v3.0.0/src/TemplateWriter.js#L428

4. Which defers to EleventyFiles:

https://github.com/11ty/eleventy/blob/v3.0.0/src/TemplateWriter.js#L140

5. Which is running a glob search:

https://github.com/11ty/eleventy/blob/v3.0.0/src/EleventyFiles.js#L400

6. Which is first querying the globs to search for (our fd string)

https://github.com/11ty/eleventy/blob/v3.0.0/src/EleventyFiles.js#L350

7. Before running a filesystem search using fast-glob:

https://github.com/11ty/eleventy/blob/v3.0.0/src/FileSystemSearch.js#L55

And here we receive an empty array as a result which leads to 0 files written in the end.

[thoughtsunificator]

Yes, exactly, and it looks like the glob I mentioned is not part of the issue, indeed. When passed a file descriptor through process substitution, we end up with the following TemplateWriter object:
{ passthroughCopy: Array(0), templates: Array(0) }
With README.md:
{ passthroughCopy: Array(0), templates: Array(1) }

Maybe we could do without all the globster action when a single file is passed?

[thoughtsunificator]

To help visualize, here's the call tree:

[thoughtsunificator]

I believe it's because https://github.com/mrmlnc/fast-glob/blob/e60a9f5f09bc58a3b22b1d7fb767c25f62df0d07/src/providers/filters/entry.ts#L55 checks if it's a file and Node.js fs API (isFile) returns false although it will return true when isFIFO is called. This seems to have changed in 3.3.3, although it's not out yet.

Here's the stacks:

#0 getFilter node_modules/fast-glob/out/providers/filters/entry.js:11:8
#1 _getReaderOptions node_modules/fast-glob/out/providers/provider.js:26:42
#2 read node_modules/fast-glob/out/providers/async.js:12:29
#3 getWorks node_modules/fast-glob/out/index.js:93:17
#4 FastGlob node_modules/fast-glob/out/index.js:10:18

#0 _onlyFileFilter node_modules/fast-glob/out/providers/filters/entry.js:40:8
#1 _filter node_modules/fast-glob/out/providers/filters/entry.js:20:17
#2 (anonymous) node_modules/fast-glob/out/providers/filters/entry.js:13:31
#3 (anonymous) node_modules/fast-glob/out/readers/stream.js:23:46

I have also filed a bug report in their repo: mrmlnc/fast-glob#469

EDIT: It seems that it was intentionally fixed in 3.3.3: mrmlnc/fast-glob@bcfd7bb#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R475