Move CDXJ file handling from bin/cli.js to WACZ class #120
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will allow other applications like Browsertrix Crawler that want to pass in existing CDXJ files be able to do so simply by setting the right values in the WACZ class initialization rather than having to duplicate code from the js-wacz CLI.
matteocargnelutti
force-pushed
the
cdxj-followups
branch
from
b857345 to
08b2571
Compare
matteocargnelutti
force-pushed
the
cdxj-followups
branch
from
08b2571 to
813e550
Compare
matteocargnelutti
approved these changes
Aug 15, 2024
There was a problem hiding this comment.
Hey there @tw4l ! Great PR, thank you.
I added docs for this new cdjx option in types.js and made a few other related edits - would you mind having a look?
Thank you 👋
tw4l
commented
Aug 15, 2024
types.js
Outdated
| * @property {?string} url - If set, will be added to datapackage.json as `mainPageUrl`. | ||
| * @property {?string} ts - If set, will be added to datapackage.json as `mainPageDate`. Can be any value that `Date()` can parse. | ||
| * @property {?string} title - If set, will be added to datapackage.json as `title`. | ||
| * @property {?string} description - If set, will be added to datapackage.json as `description`. | ||
| * @property {?string} signingUrl - If set, will be used to try and sign the resulting archive. | ||
| * @property {?string} signingToken - Access token to be used in combination with `signingUrl`. | ||
| * @property {?Object} datapackageExtras - If set, will be appended to datapackage.json under `extras`. | ||
| * @property {?string} cdxj - If set, skips indexing and allows for passing CDXJ files "as is". Path to a folder containing CDXJ files. Allows |
There was a problem hiding this comment.
Thanks for adding these! Looks like maybe some text got cut off at the end here?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up PR for issue #88
This change makes it so that other applications like Browsertrix Crawler that want to use js-wacz as a library can pass in existing CDXJ files simply by setting the right values during the WACZ class initialization rather than having to duplicate code from the js-wacz CLI module to add each line of the provided CDXJ files to the right b-tree manually.