Use workers to process files in parallel

[walkerburgin]

This PR adds a worker pool to process files in parallel.

Notes

The piscina typings reference EventEmitterAsyncResource, which was added in Node 16.14.0 but (apparently) wasn't exposed by @types/node until 20.x.x. I bumped @types/node and increased the engines check to 16.x, which feels reasonable given that Node 18 is already in maintenance and the current LTS is 20.x.x (docs). Not sure if you're willing to take that change or not.

Results

Here's the command I'm running to compare performance. We're currently using SWC for minification at the last stage of a build pipeline, so this test primarily reflects that use case. The fixtures directory here contains ~2400 JavaScript files.

time ../swc-cli/bin/swc.js --out-dir build/swc-cli -C minify=true -C jsc.minify.compress=true -C jsc.minify.mangle=true fixtures/**/*.js

Description	Time
With --sync (before this PR)	3:35.69
Without --sync (before this PR)	2:36.66
With --workers (after this PR)	1:18.58

As a sidenote, I was a little surprised that this is still meaningfully slower than replacing transformFile() with minify() (better emulating the custom code we're using internally at the moment). I'm not really familiar with SWCs implementation, so I'm not sure whether I should be surprised by this or not.

Description	Time
Replace transformFile() with minify() and --sync	43.552
Replace transformFile() with minify() without --sync	15.475
Replace transformFile() with minify() and add --workers	17.399

The minification docs imply that it's appropriate to run the CLI for minification like this. Maybe a dedicated minify command might make sense?

Fixes #274

[changeset-bot]

⚠️ No Changeset found

Latest commit: e50e2cf

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Walker Burgin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@types/[email protected]	None	+1	4.05 MB	types
npm/[email protected]	unsafe Transitive: environment, filesystem	+3	870 kB	metcoder95

🚮 Removed packages: npm/@types/[email protected]

View full report↗︎

[socket-security]

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Alert	Package	Note	Source
Native code	npm/[email protected]	Location: Package overview	examples/spack-basic/package.json examples/spack-multiple-entry/package.json examples/spack-node-modules/package.json package.json

View full report↗︎

Next steps

What's wrong with native code?

Contains native code which could be a vector to obscure malicious code, and generally decrease the likelihood of reproducible or reliable installs.

Ensure that native code bindings are expected. Consumers may consider pure JS and functionally similar alternatives to avoid the challenges and risks associated with native code bindings.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/[email protected] or ignore all packages with @SocketSecurity ignore-all

• @SocketSecurity ignore npm/[email protected]

[kdy1]

Thank you!