Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Configs] Add foundation for RelayMiner operation configs. #284

Merged
merged 46 commits into from
Jan 4, 2024
Merged

[Configs] Add foundation for RelayMiner operation configs. #284

merged 46 commits into from
Jan 4, 2024

Conversation

red-0ne
Copy link
Contributor

@red-0ne red-0ne commented Dec 19, 2023 •
edited
Loading

Summary

Human Summary

  • Implement new RlayMiner configuration ([META] Add a new relayminer config #246) parser
  • Adapt RelayMiner server initialization to
    • Source from the config
    • Check if the config matches the on-chain Supplier's data
    • Support multiple proxy server instances
  • Adapt the RelayMiner config parser tests
  • Adapt the RelayMiner unit tests

This PR is based on #257 and deprecates the RelayMiner part of it.

Issue

Type of change

Select one or more:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Testing

  • Run all unit tests: make go_develop_and_test
  • Run E2E tests locally: make test_e2e
  • Run E2E tests on DevNet: Add the devnet-test-e2e label to the PR. This is VERY expensive, only do it after all the reviews are complete.

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have performed a self-review of my own code
  • I have commented my code, updated documentation and left TODOs throughout the codebase

red-0ne added 26 commits December 4, 2023 14:53
@red-0ne
feat: Working SDK with AppGateServer integration
13157a6
@red-0ne
chore: Address change requests
8c5587e
@red-0ne
fix: Chain depinject configs
f309253
@red-0ne
Merge remote-tracking branch 'origin/main' into sdk
d462807
@red-0ne
chore: Expose the full SessionSuppliers struct
cb999a0
@red-0ne
fix: Avoid nil sessions on GetSessionSupplierEndpoints
94f8f24
@red-0ne
chore: Explain SingleSupplierEndpoint struct usage
f3cc97e
@red-0ne
Merge remote-tracking branch 'origin/main' into sdk
42910bb
@red-0ne
fix: Rename client/event import as per main chainges
605e8ad
@red-0ne
fix: Add missing fmt package
a117bac
@red-0ne
Merge remote-tracking branch 'origin/main' into sdk
b11f705
@red-0ne
Merge remote-tracking branch 'origin/main' into sdk
bc3a0ce
@red-0ne
Merge remote-tracking branch 'origin/main' into sdk
99f0549
@red-0ne
feat: Enable gRPC client connection
62069d5
@red-0ne
feat: Have distinct JSON-RPC and gRPC urls
c0c73c9
@red-0ne
chore: Trigger e2e tests
824d2f6
@red-0ne
Merge branch 'feat/sdk-grpc-client' into feat/grpc-client-conn
c16387a
@red-0ne
chore: Rename config files entries and enable grpc for off-chain actors
4ed010a
@red-0ne
chore: Revert reviewdog
cad8f04
@red-0ne
chore: Address request changes
8d9588d
@red-0ne
chore: Fix temp assignment of FlagGRPC
8b1465c
@red-0ne
feat: [WIP] Implement new replayminer config
6c51872
@red-0ne
Merge remote-tracking branch 'origin/main' into feat/grpc-client-conn
c66d998
@red-0ne
feat: Adapt tests to new relay miner config
4b4df7a
@red-0ne
chore: Add relay miner config and full example
f6338ca
@red-0ne
Merge remote-tracking branch 'origin/main' into feat/relay-miner-config
831d353
@red-0ne red-0ne added relayminer Changes related to the Relayminer proxy Changes related to the Proxy tooling Tooling - CLI, scripts, helpers, off-chain, etc... labels Dec 19, 2023
@red-0ne red-0ne added this to the Shannon TestNet milestone Dec 19, 2023
Olshansk
Olshansk requested changes Dec 23, 2023
View reviewed changes
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this require additional changes if we go with proxy_name and service_id instead of name or would we want to do this in a followup PR?

localnet/poktrolld/config/relayminer_config_full_example.yaml Outdated Show resolved Hide resolved
localnet/poktrolld/config/relayminer_config_full_example.yaml Outdated Show resolved Hide resolved
localnet/poktrolld/config/relayminer_config_full_example.yaml Outdated Show resolved Hide resolved
localnet/poktrolld/config/relayminer_config_full_example.yaml Outdated Show resolved Hide resolved
@red-0ne red-0ne requested a review from Olshansk January 2, 2024 15:04
Olshansk
Olshansk approved these changes Jan 3, 2024
View reviewed changes
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the multiple iterations on this!

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 3, 2024

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

@github-actions github-actions bot added devnet push-image CI related - pushes images to ghcr.io labels Jan 3, 2024
okdas
okdas reviewed Jan 3, 2024
View reviewed changes
localnet/kubernetes/values-relayminer.yaml
proxy_names:
- http-proxy
hosts:
- tcp://relayminers:8545
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be just relayminers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't have any proxy/load banalncer yet, the port needs to be added here. Also the tcp:// scheme is here for url validation, it is then stripped to only take the host part.

@red-0ne red-0ne requested a review from Olshansk January 4, 2024 18:54
Olshansk
Olshansk approved these changes Jan 4, 2024
View reviewed changes
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's frikkin 🚢 this thing!

@red-0ne red-0ne merged commit bc55aa2 into main Jan 4, 2024
10 checks passed
@Olshansk Olshansk deleted the feat/relay-miner-config branch January 4, 2024 21:09
This was referenced Jan 4, 2024
Closed
Closed
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge remote-tracking branch 'pokt/main' into issues/141/refactor/cla…
f3e7866 
…im-proof

* pokt/main:
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/refactor/claim-proof' into issues/141/refact…
df9e3c2 
…or/supplier-keys

* issues/141/refactor/claim-proof:
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/refactor/supplier-keys' into issues/141/refa…
8ab13f0 
…ctor/supplier-errors

* issues/141/refactor/supplier-keys:
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/refactor/supplier-errors' into issues/141/pr…
b4ed47c 
…ep/in-memory-network

* issues/141/refactor/supplier-errors:
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/prep/in-memory-network' into issues/141/feat…
191c956 
…/in-memory-network

* issues/141/prep/in-memory-network:
  chore: add #GetConfig()
  chore: rename InMemoryCosmosNetwork to InMemoryNetwork
  chore: update comment
  chore: review feedback improvements
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/feat/in-memory-network' into issues/141/refa…
d1e68fe 
…ctor/in-memory-network

* issues/141/feat/in-memory-network:
  chore: fix comment typo
  chore: post-merge refactor
  chore: fix comment
  chore: add #GetConfig()
  chore: rename InMemoryCosmosNetwork to InMemoryNetwork
  chore: update comment
  chore: review feedback improvements
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
bryanchriswhite added a commit that referenced this pull request Jan 8, 2024
@bryanchriswhite
Merge branch 'issues/141/refactor/in-memory-network' into issues/141/…
973635c 
…refactor/proof-store-indices

* issues/141/refactor/in-memory-network:
  chore: post-merge refactor
  chore: review feedback improvements
  chore: fix comment typo
  chore: post-merge refactor
  chore: fix comment
  chore: add #GetConfig()
  chore: rename InMemoryCosmosNetwork to InMemoryNetwork
  chore: update comment
  chore: review feedback improvements
  chore: add TODOs
  [Docs] Load Test #1 - Plan (#286)
  [Bug] Fix observable error logging (#298)
  [docs] Relayminer config documentation (#288)
  [Configs] Add foundation for RelayMiner operation configs. (#284)
  [SMT] Update to use SMT v0.8.2 (#297)
  [EventsReplayClient] Fix Replay Client Bugs (#267)
@bryanchriswhite bryanchriswhite removed push-image CI related - pushes images to ghcr.io devnet-test-e2e labels May 16, 2024
@github-actions github-actions bot removed the devnet label May 16, 2024
okdas pushed a commit that referenced this pull request Nov 14, 2024
@red-0ne
[Configs] Add foundation for RelayMiner operation configs. (#284)
001a8a6 
* feat: Working SDK with AppGateServer integration

* chore: Address change requests

* fix: Chain depinject configs

* chore: Expose the full SessionSuppliers struct

* fix: Avoid nil sessions on GetSessionSupplierEndpoints

* chore: Explain SingleSupplierEndpoint struct usage

* fix: Rename client/event import as per main chainges

* fix: Add missing fmt package

* feat: Enable gRPC client connection

* feat: Have distinct JSON-RPC and gRPC urls

* chore: Trigger e2e tests

* chore: Rename config files entries and enable grpc for off-chain actors

* chore: Revert reviewdog

* chore: Address request changes

* chore: Fix temp assignment of FlagGRPC

* feat: [WIP] Implement new replayminer config

* feat: Adapt tests to new relay miner config

* chore: Add relay miner config and full example

* chore: add valid relay miner config

* chore: Address request changes

* fix: Correct tests error assertions

* chore: Address request changes

* refactor: Split relayminer config parsing and validation logic

* chore: Add TODO to make MarshalAndSend RPC-type agnostic

* chore: Strip-off comments form AppGateServer config, add documented example config

* chore: Strip comments out of relay miner config and reorder entries

* chore: Update flagNode variables and command documentation

* fix: Update sequencer host used in local net

* chore: Address change requests

* chore: Add UDP in TODO comment

* chore: Improve documentation about multiple hosts config

* chore: Add change to trigger e2e test

* chore: e2e bump

* fix: Use RPC url for tx broadcasting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@okdas okdas okdas left review comments

@Olshansk Olshansk Olshansk approved these changes

@bryanchriswhite bryanchriswhite Awaiting requested review from bryanchriswhite

Assignees

@red-0ne red-0ne

Labels
proxy Changes related to the Proxy relayminer Changes related to the Relayminer tooling Tooling - CLI, scripts, helpers, off-chain, etc...
Projects
Shannon
Status: ✅ Done
Milestone
Shannon Private TestNet
Development

Successfully merging this pull request may close these issues.
4 participants
@red-0ne @Olshansk @okdas @bryanchriswhite