style+perf: clean-up and optimize remove_empty_byte_from_padded_bytes_unchecked fn #41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…_unchecked function
There were a bunch of warnings that some of our set fmt properties were not being run: Warning: can't set `wrap_comments = true`, unstable features are only available in nightly channel. Warning: can't set `normalize_comments = true`, unstable features are only available in nightly channel.
Getting "error: toolchain 'nightly-x86_64-unknown-linux-gnu' is not installed" on github, and don't feel like debugging. Not even sure how cargo/rust are installed. Do they come preloaded by default? This reverts commit 6e87e0a.
|
Note: Apologies about the large number of edits that are just formatting.... applied |
bxue-l2
previously approved these changes
Jan 13, 2025
src/kzg.rs
Outdated
| /// Precompute the primitive roots of unity for binary powers that divide r - 1 | ||
| /// TODO(anupsv): Move this to the constants file. Ref: https://github.com/Layr-Labs/rust-kzg-bn254/issues/31 | ||
| /// Precompute the primitive roots of unity for binary powers that divide r | ||
| /// - 1 TODO(anupsv): Move this to the constants file. Ref: https://github.com/Layr-Labs/rust-kzg-bn254/issues/31 |
There was a problem hiding this comment.
argh that's what our linter when ran with nightly version does.... think I should just revert that commit?
@anupsv we'll need to look at that linter config at some point. It seems not that great.
There was a problem hiding this comment.
Reverted the cargo +nightly fmt commit and formatted with stable rust instead. PTAL
|
wait, exactly which approach you implemented? confusing to read " I do have to not however that the version with iterators (the one in this PR) is faster on 32KiB inputs but (slightly) slower on 32MiB." Code wise it looks right to mee |
Updated PR description, should have read "I do have to NOTE however" |
This reverts commit ae70bf5.
bxue-l2
approved these changes
Jan 14, 2025
samlaf
deleted the
perf--optimize-fn-remote-empty-byte-from-padded-bytes-unchecked
branch
anupsv
pushed a commit
that referenced
this pull request
Jan 15, 2025
…_unchecked fn (#41) * style+perf: clean-up and optimize remove_empty_byte_from_padded_bytes_unchecked function * ci: make cargo fmt use nightly There were a bunch of warnings that some of our set fmt properties were not being run: Warning: can't set `wrap_comments = true`, unstable features are only available in nightly channel. Warning: can't set `normalize_comments = true`, unstable features are only available in nightly channel. * style: cargo fmt * Revert "ci: make cargo fmt use nightly" Getting "error: toolchain 'nightly-x86_64-unknown-linux-gnu' is not installed" on github, and don't feel like debugging. Not even sure how cargo/rust are installed. Do they come preloaded by default? This reverts commit 6e87e0a. * Revert "style: cargo fmt" This reverts commit ae70bf5. * style: cargo fmt
samlaf
added a commit
that referenced
this pull request
Jan 19, 2025
* removing reading g2, fixing g2 tau points. clean up * fixing mainnet points * removing unwated files * style+perf: clean-up and optimize remove_empty_byte_from_padded_bytes_unchecked fn (#41) * style+perf: clean-up and optimize remove_empty_byte_from_padded_bytes_unchecked function * ci: make cargo fmt use nightly There were a bunch of warnings that some of our set fmt properties were not being run: Warning: can't set `wrap_comments = true`, unstable features are only available in nightly channel. Warning: can't set `normalize_comments = true`, unstable features are only available in nightly channel. * style: cargo fmt * Revert "ci: make cargo fmt use nightly" Getting "error: toolchain 'nightly-x86_64-unknown-linux-gnu' is not installed" on github, and don't feel like debugging. Not even sure how cargo/rust are installed. Do they come preloaded by default? This reverts commit 6e87e0a. * Revert "style: cargo fmt" This reverts commit ae70bf5. * style: cargo fmt * Cleanup/roots of unities setup functions (#37) * adding bare changes for batch verification * adding some comments * adding more comments * moving back to sha2 * removing a test which is no longer needed. Removing methods no longer needed * updates to method visibility, updating tests * fmt fixes * clean up * cleanup, optimization, inline docs * removing unwanted const * more docs and cleanup * formatting * removing unwanted comments * cargo fmt and clippy * adding test for point at infinity * cleaner errors, cleanup * adding another test case * removing unwanted errors * adding fixes per comments * adding 4844 spec references * comment fixes * formatting, adding index out of bound check, removing print statement * removing unwanted test, adding test for evaluate_polynomial_in_evaluation_form * moving test to bottom section * Update src/polynomial.rs Co-authored-by: Samuel Laferriere <[email protected]> * Update src/kzg.rs Co-authored-by: Samuel Laferriere <[email protected]> * Update src/kzg.rs Co-authored-by: Samuel Laferriere <[email protected]> * Update src/kzg.rs Co-authored-by: Samuel Laferriere <[email protected]> * Update src/helpers.rs Co-authored-by: Samuel Laferriere <[email protected]> * updating deps, and toolchain to 1.84 * removing errors test, no longer useful * adding to_byte_array arg explanation * fmt fixes * fmt and clippy fixes * fixing function names and fmt * clippy fixes * removing unwanted setup functions * removing vars from struct * fixing function name and comments * fixing naming for tests * fixing naming in benches * formatting * removing is_zero --------- Co-authored-by: anupsv <[email protected]> Co-authored-by: Samuel Laferriere * fixing endianess to big across functions (#39) * fixing endianess to big across functions * removing todo's * storing g2 tau as consts * cargo fmt * moving checker to test, adding explanation, cargo fmt * more cargo fmt --------- Co-authored-by: anupsv <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was a fun weekend. Got to learn a crap ton about rust iterators, assembly output, godbolt, llvm, etc.
I was just trying to make this function cleaner by adopting a functional iterator, but in doing so realized the code was then much slower (up to 7x depending on input size). With 2 small modifications, managed to get the output to use pre-allocated output vector and use simd instructions for copying, which made the code 2-7x FASTER depending on input size.
Benchmarks are available in master...perf--remove-empty-byte-from-padded-bytes-fn-benchmark. Here are the results (function_fast is the function implemented in this PR):
for 32B inputs
for 32KiB inputs
for 32MiB inputs
Note: I decided to implement the functional_fast function instead of the fast function (which contains the same logic but written without iterators), because I personally find it cleaner to read. I do have to note however that the version with iterators (the one in this PR) is faster on 32KiB inputs but (slightly) slower on 32MiB. If we ever have teams sending huge bytes in the future, we might want to implement both approaches and let them pick and choose? Or perhaps have a wrapper that dispatches to the correct implementation based on input size?