process: add execve #56496

ShogunPanda · 2025-01-07T06:26:52Z

This PR adds a new process.execve method (absolutely willing to change the name if you want), which is a wrapper for the execve UNIX function.

The function will never return and will swap the current process with a new one.
All memory and system resources are automatically collected from execve, except for std{in,out,err}.

The primary use of this function is in shell scripts to allow to setup proper logics and then spawn another command.

nodejs-github-bot · 2025-01-07T06:26:57Z

Review requested:

@nodejs/startup

nodejs-github-bot · 2025-01-07T06:34:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64381/

ljharb · 2025-01-07T06:35:37Z

What does it do on non-Unix systems?

ShogunPanda · 2025-01-07T07:18:59Z

@ljharb By non Unix we only mean Windows, isn't it? If that's the case, Windows also supports execve via _execve (doc here) so we should be good to go.

Once the CI ends I'll see which platform needs special assistance.

codecov · 2025-01-07T07:41:54Z

Codecov Report

Attention: Patch coverage is 88.05970% with 16 lines in your changes missing coverage. Please review.

Project coverage is 89.12%. Comparing base (afafee2) to head (e56bf2d).
Report is 28 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_process_methods.cc	79.48%	14 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #56496      +/-   ##
==========================================
+ Coverage   88.53%   89.12%   +0.58%     
==========================================
  Files         657      662       +5     
  Lines      190761   191709     +948     
  Branches    36616    36862     +246     
==========================================
+ Hits       168899   170861    +1962     
+ Misses      15048    13712    -1336     
- Partials     6814     7136     +322

Files with missing lines	Coverage Δ
lib/internal/bootstrap/node.js	`99.57% <100.00%> (+<0.01%)`	⬆️
lib/internal/process/per_thread.js	`99.39% <100.00%> (+0.06%)`	⬆️
src/node_errors.h	`88.00% <100.00%> (+3.00%)`	⬆️
src/node_process_methods.cc	`88.88% <79.48%> (+3.06%)`	⬆️

... and 97 files with indirect coverage changes

targos · 2025-01-07T07:45:44Z

On Windows:

D:\a\node\node\src\node_process_methods.cc(544,26): error C2065: 'F_GETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(544,17): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(546,17): error C2065: 'FD_CLOEXEC': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,16): error C2065: 'F_SETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,7): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')

targos

To fix https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/40120/

test/parallel/test-process-replace.js

test/parallel/test-process-replace-fail.js

test/parallel/test-process-replace-socket.js

src/node_process_methods.cc

ardinugrxha · 2025-01-07T08:02:50Z

src/node_process_methods.cc

+    node::Utf8Value pathname_string(env->isolate(), args[0].As<String>());
+
+    argv = new char*[2];
+    argv[0] = strdup(*pathname_string);


we need check not null also in here, maybe(?)

Good spot. Added.

I think we should use std::memcpy or a similar modern C++ function other than this.

mcollina

lgtm

test/parallel/test-process-replace-fail.js

ShogunPanda · 2025-01-07T14:04:11Z

@ljharb I was totally wrong. Despite of naming and similar signatures, the function _execve only creates a new process without replacing the old one.

I'll disable this function on Windows.

nodejs-github-bot · 2025-01-07T15:36:12Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64387/

doc/api/process.md

addaleax

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

This requires integration with the permissions API or is otherwise an immediate security hole.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

doc/api/errors.md

addaleax · 2025-01-07T17:16:37Z

doc/api/process.md

+resources from the current process are preserved, except for the standard input,
+standard output and standard error file descriptor.
+
+All other resources are discarded by system when the processes are swapped.


What if this isn't the a desirable behavior in a given situation?

Can you please expand this? What do you mean?

If there's any point in adding execve(), it's the customizability that the method brings with it. Leaving file descriptors open and/or redirecting them intentionally can be part of that -- just like you could in a bash script do exec bash 0<&4 1>&4 to create a shell that reads and writes to e.g. a pre-opened socket or something along those lines.

(With a similar reasoning, you could also allow users to actually specify argv[0] with a value they control -- it doesn't need to equal the filename that's being executed)

addaleax · 2025-01-07T17:18:52Z

src/node_process_methods.cc

+
+  THROW_ERR_PROCESS_REPLACE_FAILED(env, error_code);
+}
+#endif


This is quite hard-to-follow C++ with lots of unnecessary explicit memory management that Node.js has been doing a lot of effort to move away from. I'd recommend looking a bit a how other parts of the Node.js code base handle strings and conversion between C++ and JS values.

I'm absolutely willing to. Since I'm not familiar with the C++ codebase, do you have any suggestion of places I can look to?

@ShogunPanda Well, mostly anywhere else works. As a general rule, you'll want to get rid of new, new[], char*, memcpy() and strdup() as much as possible, and replace them with std::vector, std::string/Utf8Value as much as possible.

(You won't be entirely able to avoid something like std::vector<char*> because execve expects char** arguments, but the std::vector<char*>'s entries could point to the entries of a std::vector<std::string> or std::vector<Utf8Value> rather than having to manage memory manually).

ljharb · 2025-01-07T18:09:59Z

Why would we want to add a function that can't work on all tier 1 platforms?

jasnell · 2025-01-07T18:24:37Z

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

ljharb · 2025-01-07T18:25:44Z

Gotcha, i wasn't aware of that.

jasnell · 2025-01-07T18:29:57Z

I don't consider it to be ideal. I would have preferred a pattern like process.posix.getegid() similar to what we have with path.posix but these predate my involvement so they are what they are.

jasnell · 2025-01-07T18:32:10Z

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

lib/internal/process/per_thread.js

src/node_errors.h

anonrig · 2025-01-08T01:24:17Z

src/node_process_methods.cc

+  char** target = nullptr;
+  int length = js_array->Length();
+
+  CHECK_LT(length, INT_MAX);


Suggested change

CHECK_LT(length, INT_MAX);

CHECK_LT(length, INT_MAX);

CHECK_BT(length, 0);

I couldn't find CHECK_BT anywhere. Did you mean CHECK_GT or what?

src/node_process_methods.cc

anonrig · 2025-01-08T01:28:34Z

src/node_process_methods.cc

+    for (unsigned int i = 0; i < argv_array->Length(); i++) {
+      full_argv_array
+          ->Set(context, i + 1, argv_array->Get(context, i).ToLocalChecked())
+          .Check();
+    }


Does this make a copy? I didn't quite understand the goal here. Sorry for the questions!

Yes, that's the idea. execve wants a null-terminated char** while I have a JsArray.
On the Javascript side I have validated they are all strings with no null-bytes in the middle.
Any better idea on how to handle this?

My gut feeling would be that it's much easier to handle the conditional and array copying on the JS side of things and instead only pass a version of the array(s) down to C++ that's close to what the syscall expects 🙂

I see what you mean now.
I would love to make all the manipulation on the JS side, but it is then possible to pass the data from JS to be interpreted on C++ as char**. Out of my mind I'm thinking about using Uint8Array but I might be wrong.

You can still pass an array of strings, it's just the array copying/argument handling that would end up simplified (i.e. no need to check if (args.Length() > 1) {, no need to full_argv_array->Set(context, i + 1, argv_array->Get(context, i).ToLocalChecked()))

src/node_process_methods.cc

ShogunPanda · 2025-01-08T05:29:22Z

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

Thanks for involuntary hint. :)
I updated the code with #ifdef __POSIX__ and the docs to use the same wording already used there.

ShogunPanda · 2025-01-08T06:03:07Z

@addaleax

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

As requested, I've renamed it to execve. I didn't like the original name either :)

This requires integration with the permissions API or is otherwise an immediate security hole.

I just added integration with the permission API. It will require --allow-child-process.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

In the shell scripting context, there is no way to create a new process which would replace the current one. Think about using Node.js to build a complex command to run. After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

ShogunPanda · 2025-01-08T06:11:00Z

@jasnell

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

I added two tests which check that:

Test what happens if we leave a socket open and we try to reopen in the swapped process: the operation succeeds, which means the fd is destroyed. I also tried to install a on('close') but it is not invoked.
Test what happens if there is a process.on('exit') on the original process. It does not get invoked, which means that the system call immediately swap the programs without running any cleanup logic.

I've added this to the documentation to reflect that. Does it look good now?

addaleax · 2025-01-08T06:32:45Z

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

ShogunPanda · 2025-01-08T07:10:48Z

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

I kinda agree.
You still would have to handle the exit code to ensure proper broadcasting to the shell.
Also, I'm not sure what would happen with "TUI" (ncurses and similar) programs and so forth.

Is your objection a full block or just an "intent"?

addaleax · 2025-01-08T07:22:29Z

@ShogunPanda Just fyi, have you seen https://www.npmjs.com/package/foreground-child? That also works on Windows 🙂

My "request changes" marker only refers to the C++ memory management here, i.e. this comment specifically. I don't intend to block this feature, I've given my opinion but it's pretty clear overall that the Node.js project has a different stance from my own when it comes to 'what should go into core' 🙂

anonrig · 2025-01-08T16:43:09Z

src/node_errors.h

+  char message[128];
+  snprintf(message,
+           sizeof(message),
+           "process.execve failed with error code %s",
+           errors::errno_string(code));


Why not?

Suggested change

char message[128];

snprintf(message,

sizeof(message),

"process.execve failed with error code %s",

errors::errno_string(code));

auto message = std::string("process.execve failed with error code " + errors::errno_string(code);

anonrig · 2025-01-08T16:43:58Z

src/node_process_methods.cc

+      full_argv_array
+          ->Set(context, i + 1, argv_array->Get(context, i).ToLocalChecked())
+          .Check();


If you don't want this to throw, you can surround this with USE() and remove .Check()

?

Ideally, this should use neither of the two and just do proper error handling

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. process Issues and PRs related to the process subsystem. labels Jan 7, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

targos reviewed Jan 7, 2025

View reviewed changes

ardinugrxha reviewed Jan 7, 2025

View reviewed changes

mcollina approved these changes Jan 7, 2025

View reviewed changes

richardlau reviewed Jan 7, 2025

View reviewed changes

test/parallel/test-process-replace-fail.js Outdated Show resolved Hide resolved

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

jasnell reviewed Jan 7, 2025

View reviewed changes

doc/api/process.md Show resolved Hide resolved

addaleax requested changes Jan 7, 2025

View reviewed changes