Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for multiple KZG backends #88

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

ArtiomTr
Copy link
Contributor

In this PR:

  • Added possibility to optionally compile blst or zkcrypto backends for kzg_utils package. Both can be included at the same time, and then selected at the runtime.
  • Added option to fork_choice_store::StoreConfig - kzg_backend
  • Added CLI option --kzg-backend.

Closes #74.

@sauliusgrigaitis
Copy link
Member

The build crashes. Is it the same issue as we have in #69 ? The issue in #69 is that there is no way to make a default backend as blst, and the only solution is to provide the backend every time explicitly.

@ArtiomTr
Copy link
Contributor Author

The build crashes. Is it the same issue as we have in #69 ? The issue in #69 is that there is no way to make a default backend as blst, and the only solution is to provide the backend every time explicitly.

Yes, currently build fails as no backends are specified. But I believe this could be fixed by specifying blst backend as default feature? I'll try to fix this

@sauliusgrigaitis
Copy link
Member

But I believe this could be fixed by specifying blst backend as default feature? I'll try to fix this

Seems there was some issue with setting a default feature. Let us know your findings.

@ArtiomTr ArtiomTr force-pushed the Add-support-for-multiple-kzg-backends branch 3 times, most recently from 8d5d9e0 to 34a8354 Compare January 16, 2025 10:49
@ArtiomTr
Copy link
Contributor Author

Not exactly sure why it works like that, but default feature needs to be specified in grandine crate, not in kzg_utils.

If blst feature is defined as default inside kzg_utils crate, then running cargo build --no-default-features still compiles kzg_utils with blst feature, and there is no other way of disabling it.

@ArtiomTr ArtiomTr force-pushed the Add-support-for-multiple-kzg-backends branch from 34a8354 to 4a34617 Compare January 16, 2025 17:43
@ArtiomTr ArtiomTr marked this pull request as ready for review January 16, 2025 17:44
blob_to_kzg_commitment::<Mainnet>(&blob).expect("should compute blob to kzg commitment");

assert_eq!(commitment, expected_commitment);
let backends = [
Copy link
Collaborator

@povi povi Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: Trying to run tests with zkcrypto backend results in compile errors. Try running:

cargo test --release --lib -p kzg_utils --features zkcrypto

results in

error[E0433]: failed to resolve: use of undeclared crate or module `rust_kzg_blst`
 --> kzg_utils/src/spec_tests/mod.rs:1:5
  |
1 | use rust_kzg_blst::types::{fr::FsFr, g1::FsG1};
  |     ^^^^^^^^^^^^^ use of undeclared crate or module `rust_kzg_blst`

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it would be great if build automatically ran tests for both backends

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've refactored tests - there was some additional checks before running actual eip_4844 functions, e.g.:

if G1::from_bytes(&test.input.get_commitment_bytes()).is_err() {
assert!(test.output.is_none());
return;
}

I think these checks were needed previously, as rust-kzg was panicking, when invalid parameters given. But now rust-kzg returns errors, so these checks are no longer required. But if you think that we should keep them, I can bring them back

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it would be great if build automatically ran tests for both backends

You mean when running cargo test all backends should be tested? Or adding --features zkcrypto to CI is enough?

kzg_utils/src/spec_tests/tests.rs Outdated Show resolved Hide resolved
@ArtiomTr ArtiomTr force-pushed the Add-support-for-multiple-kzg-backends branch from 4a34617 to 3637fc0 Compare January 22, 2025 16:56
@sauliusgrigaitis
Copy link
Member

I have a couple of comments:

  1. I suggest to add all the backends that rust-kzg supports (not only blst and zkcrypto) including GPU optimised and even mcl (it should be merged very soon I think).

  2. We now build on Linux, Windows and Macos on Gitlab CI. I suggest to enable different backends for those 3 different builds. But we need to choose which ones. My initial idea would be to use blst on Linux, and then choose a couple of other backends for Windows and Macos. zkcrypto would be a great candidate for Windows or Macos because we already have zkVM usecase for it. constantine or arkworks could be the third.

@sauliusgrigaitis
Copy link
Member

@ArtiomTr - can we also add GPU optimised backends?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for multiple rust-kzg backends
3 participants