Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace knarc with libnarc #327

Merged
merged 1 commit into from
Jan 3, 2025
Merged

Replace knarc with libnarc #327

merged 1 commit into from
Jan 3, 2025

Conversation

lhearachel
Copy link
Collaborator

@lhearachel lhearachel commented Jan 1, 2025

This replaces the knarc utility subproject for packing NARCs with libnarc. libnarc is a C implementation (rather than C++) and comes with some additional features:

  1. It is shipped as a static CLI and a shareable library, which enables other tools to make direct use of the narc protocol, if they require it. This is technically usable by our current Python scripts, but not very practical due to the ability to use the subprocess module instead. However, I have plans for another tool which will be able to make use of this shared library.
  2. It comes with built-in filetype guessing. This is a feature that is straightforward to implement in knarc, but was just never done.
  3. It ships with an info command, which allows a user to inspect the internals of a NARC without needing to totally unpack it. This can be useful when debugging a mismatch.
  4. Performance with libnarc is higher; rudimentary tests show a 4-5x speedup when packing NARCs in the worst case over knarc:

Using pl_pokegra as a test, about 3000 member files to be packed, and using an order file of the whole NARC, just to simulate a "worst case" where either program has to verify every file name. Did 1000 iterations with each program. All numbers are in microseconds:

knarc:

Mean   - 222645.182
Median - 220329
Max    - 316445

libnarc:

Mean   - 47672.273
Median - 47221
Max    - 70227

Testing

This port has been tested under native builds on Arch Linux; the following platforms will also need to be tested:

  • WSL1
  • WSL2
  • macOS
  • MSYS2

@lhearachel
Copy link
Collaborator Author

WSL1 verified by @cbt6.

I have verified macOS as well.

@hedara90
Copy link

hedara90 commented Jan 3, 2025

Builds just fine under Msys2, and the speed improvements were very noticeable.
Tested on a fresh Msys2 install from a make clean state.

@lhearachel
Copy link
Collaborator Author

I am judging WSL2 as valid, given my testing results on Arch and the testing results of others on various platforms (two successful macOS builds, one MSYS2 build, and one WSL1 build).

@lhearachel lhearachel merged commit 568e212 into pret:main Jan 3, 2025
1 check passed
github-actions bot pushed a commit that referenced this pull request Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants