Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DVC] Add RequestBasedMetaRepository to enable metadata retrieval directly from server #1461

Closed

Conversation

pthirun
Copy link
Contributor

@pthirun pthirun commented Jan 21, 2025

Summary, imperative, start upper case, don't end with a period

This PR introduces the use of the /store_properties/{store_name} endpoint to retrieve and cache store metadata for the DaVinci client. The endpoint was implemented in the Venice Server in this PR: #1374.

The RequestBasedMetaRepository leverages this endpoint to maintain a cache of Store and SchemaData objects for requested stores. The cache is refreshed by the NativeMetadataRepository, which this new class extends.

By default, this feature is disabled in the ClientConfig passed to the NativeMetadataRepository. The default configuration preserves the current behavior of using the ThinClientMetaStoreBasedRepository. The relevant configuration field is ClientConfig.useRequestBasedMetaRepository. For more details, see NativeMetadataRepository.getInstance.

How was this PR tested?

Unit and Integration test included in PR.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

NOTE: behavior will not change by default, feature flag in ClientConfig must be turned on.

pthirun and others added 27 commits December 3, 2024 10:21
…stem store for forward compatibility support
@pthirun pthirun requested a review from xunyin8 January 21, 2025 23:45
@pthirun pthirun marked this pull request as draft January 21, 2025 23:45
@pthirun pthirun self-assigned this Jan 21, 2025
@pthirun pthirun force-pushed the feature/dvc-request-based-metadata branch 2 times, most recently from c3ab59e to ace1c85 Compare January 22, 2025 00:18
@pthirun pthirun force-pushed the feature/dvc-request-based-metadata branch from ace1c85 to dd6680f Compare January 23, 2025 00:30
@pthirun pthirun closed this Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant