Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[admin-tool] Venice-specific path filtering logic for ZooKeeper Snapshot and Restore tooling #1047

Merged
merged 9 commits into from
Jul 9, 2024

Conversation

bonytoni
Copy link
Collaborator

@bonytoni bonytoni commented Jun 24, 2024

Venice-specific path filtering logic for ZooKeeper Snapshot and Restore tooling

Overview: To remove ZooKeeper as a potential Single Point of Failure (SPoF) during an outage, we are cloning Venice-specific ZooKeeper paths in the parent region and restoring it in another region's local Zookeeper to continue serving as the primary parent controller and resuming Nuage, VPJ, and Samza operations offline where it left off.

This PR:

  • Implemented AdminTool command that extracts Venice-specific paths from a ZK snapshot input text file to an output text file
  • Implemented path filtering logic in ZkCopier.java
  • Refactored multiple files with Venice-managed ZK path constants from VeniceZkPaths.java

Next steps: AdminTool command that clones Venice-specific paths from a source ZK (local or live instance) to a destination ZK (local or live instance)

How was this PR tested?

Unit Testing, Manual Testing

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

@bonytoni bonytoni marked this pull request as ready for review June 26, 2024 18:03
Copy link
Contributor

@nisargthakkar nisargthakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My review is still in progress and I still need to go through the main filtering algorithm. But I wanted to leave a partial review so you can start addressing the preliminary comments

Copy link
Contributor

@nisargthakkar nisargthakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good. Good job

Copy link
Contributor

@nisargthakkar nisargthakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @bonytoni

@bonytoni bonytoni merged commit 4ba73c9 into linkedin:main Jul 9, 2024
32 checks passed
@bonytoni bonytoni deleted the ZNodePathFiltering branch July 9, 2024 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants