-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Configs] Add foundation for RelayMiner operation configs. #284
Merged
Merged
Changes from all commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
13157a6
feat: Working SDK with AppGateServer integration
red-0ne 8c5587e
chore: Address change requests
red-0ne f309253
fix: Chain depinject configs
red-0ne d462807
Merge remote-tracking branch 'origin/main' into sdk
red-0ne cb999a0
chore: Expose the full SessionSuppliers struct
red-0ne 94f8f24
fix: Avoid nil sessions on GetSessionSupplierEndpoints
red-0ne f3cc97e
chore: Explain SingleSupplierEndpoint struct usage
red-0ne 42910bb
Merge remote-tracking branch 'origin/main' into sdk
red-0ne 605e8ad
fix: Rename client/event import as per main chainges
red-0ne a117bac
fix: Add missing fmt package
red-0ne b11f705
Merge remote-tracking branch 'origin/main' into sdk
red-0ne bc3a0ce
Merge remote-tracking branch 'origin/main' into sdk
red-0ne 99f0549
Merge remote-tracking branch 'origin/main' into sdk
red-0ne 62069d5
feat: Enable gRPC client connection
red-0ne c0c73c9
feat: Have distinct JSON-RPC and gRPC urls
red-0ne 824d2f6
chore: Trigger e2e tests
red-0ne c16387a
Merge branch 'feat/sdk-grpc-client' into feat/grpc-client-conn
red-0ne 4ed010a
chore: Rename config files entries and enable grpc for off-chain actors
red-0ne cad8f04
chore: Revert reviewdog
red-0ne 8d9588d
chore: Address request changes
red-0ne 8b1465c
chore: Fix temp assignment of FlagGRPC
red-0ne 6c51872
feat: [WIP] Implement new replayminer config
red-0ne c66d998
Merge remote-tracking branch 'origin/main' into feat/grpc-client-conn
red-0ne 4b4df7a
feat: Adapt tests to new relay miner config
red-0ne f6338ca
chore: Add relay miner config and full example
red-0ne 831d353
Merge remote-tracking branch 'origin/main' into feat/relay-miner-config
red-0ne 8f907f7
chore: add valid relay miner config
red-0ne d91fdd2
chore: Address request changes
red-0ne 496cde9
fix: Correct tests error assertions
red-0ne 36fcc43
Merge remote-tracking branch 'origin/feat/grpc-client-conn' into feat…
red-0ne 135dcd9
chore: Address request changes
red-0ne fab1dfb
refactor: Split relayminer config parsing and validation logic
red-0ne 6ea61bc
chore: Add TODO to make MarshalAndSend RPC-type agnostic
red-0ne f342d05
chore: Strip-off comments form AppGateServer config, add documented e…
red-0ne 22cd59e
chore: Strip comments out of relay miner config and reorder entries
red-0ne e9efcec
chore: Update flagNode variables and command documentation
red-0ne 9b80d76
Merge remote-tracking branch 'origin/feat/grpc-client-conn' into feat…
red-0ne b8847b5
fix: Update sequencer host used in local net
red-0ne 888cbb2
Merge remote-tracking branch 'origin/main' into feat/relay-miner-config
red-0ne 7fa3f51
chore: Address change requests
red-0ne 0b6bd0e
chore: Add UDP in TODO comment
red-0ne 272606b
chore: Improve documentation about multiple hosts config
red-0ne cd17a17
Merge remote-tracking branch 'origin/main' into feat/relay-miner-config
red-0ne 80c66d6
chore: Add change to trigger e2e test
red-0ne 6bc4394
chore: e2e bump
red-0ne e64c6e6
fix: Use RPC url for tx broadcasting
red-0ne File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,3 @@ | ||
config: | ||
query_node_url: tcp://sequencer-poktroll-sequencer:36657 | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,20 @@ | ||
config: | ||
query_node_url: tcp://sequencer-poktroll-sequencer:36657 | ||
network_node_url: tcp://sequencer-poktroll-sequencer:36657 | ||
signing_key_name: supplier1 | ||
smt_store_path: smt_stores | ||
pocket_node: | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
tx_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
proxies: | ||
- proxy_name: http-proxy | ||
type: http | ||
host: 0.0.0.0:8545 | ||
suppliers: | ||
- service_id: anvil | ||
type: http | ||
service_config: | ||
url: http://anvil:8547/ | ||
proxy_names: | ||
- http-proxy | ||
hosts: | ||
- tcp://relayminers:8545 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,8 +1,5 @@ | ||
# Whether the server should sign all incoming requests with its own ring (for applications) | ||
self_signing: true | ||
# The name of the key (in the keyring) that will be used to sign relays | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
signing_key: app1 | ||
# The host and port that the appgate server will listen on | ||
listening_endpoint: http://localhost:42069 | ||
# tcp://<host>:<port> to a full pocket node for reading data and listening for on-chain events | ||
query_node_url: tcp://127.0.0.1:36657 | ||
self_signing: true | ||
listening_endpoint: http://localhost:42069 |
11 changes: 11 additions & 0 deletions
11
localnet/poktrolld/config/appgate_server_config_example.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Pocket node URL that exposes CometBFT JSON-RPC API. | ||
# This can be used by the Cosmos client SDK, event subscriptions, etc... | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
# Pocket node URL that exposes the Cosmos gRPC service, dedicated to querying purposes. | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
# The name of the key (in the keyring) that will be used to sign relays | ||
signing_key: app1 | ||
# Whether the server should sign all incoming requests with its own ring (for applications) | ||
self_signing: true | ||
# The host and port that the appgate server will listen on | ||
listening_endpoint: http://localhost:42069 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,19 @@ | ||
# tcp://<host>:<port> to a full pocket node for reading data and listening for on-chain events | ||
query_node_url: tcp://localhost:36657 | ||
# tcp://<host>:<port> to a pocket node that gossips transactions throughout the network (may or may not be the sequencer) | ||
network_node_url: tcp://127.0.0.1:36657 | ||
# Name of the key (in the keyring) to sign transactions | ||
signing_key_name: supplier1 | ||
# TODO_TECHDEBT(#137, #130): Once the `relayer.json` config file is implemented AND a local LLM RPC service | ||
# is supported on LocalNet, this needs to be expanded to include more than one service. The ability to support | ||
# multiple services is already in place but currently (as seen below) is hardcoded. | ||
# TODO_UPNEXT(@okdas): this hostname should be updated to match that of the in-tilt anvil service. | ||
proxied_service_endpoints: | ||
anvil: http://anvil:8547 | ||
svc1: http://localhost:8082 | ||
# Path to where the data backing SMT KV store exists on disk | ||
smt_store_path: smt_stores | ||
pocket_node: | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
tx_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
proxies: | ||
- proxy_name: http-proxy | ||
type: http | ||
host: 0.0.0.0:8545 | ||
suppliers: | ||
- service_id: anvil | ||
type: http | ||
service_config: | ||
url: http://anvil:8547/ | ||
proxy_names: | ||
- http-proxy | ||
hosts: | ||
- tcp://relayminers:8545 |
126 changes: 126 additions & 0 deletions
126
localnet/poktrolld/config/relayminer_config_full_example.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
# TODO_CONSIDERATION: We don't need this now, but it would be beneficial if the | ||
# logic handling this config file could be designed in such a way that it allows for | ||
# "hot" config changes in the future, meaning changes without restarting a process. | ||
# This would be useful for adding a proxy or a supplier without interrupting the service. | ||
|
||
# Name of the key (in the keyring) to sign transactions. | ||
signing_key_name: supplier1 | ||
# Relative path (on the relayminer's machine) to where the data backing | ||
# SMT KV store exists on disk. | ||
smt_store_path: smt_stores | ||
|
||
pocket_node: | ||
# Pocket node URL that exposes CometBFT JSON-RPC API. | ||
# This can be used by the Cosmos client SDK, event subscriptions, etc... | ||
# If unspecified, defaults to `tx_node_rpc_url`. | ||
query_node_rpc_url: tcp://sequencer-poktroll-sequencer:36657 | ||
# Pocket node URL that exposes the Cosmos gRPC service, dedicated to querying purposes. | ||
query_node_grpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
# Pocket node URL that exposes the TendermintRPC service. | ||
tx_node_rpc_url: tcp://sequencer-poktroll-sequencer:36658 | ||
|
||
# Proxies are endpoints that expose different suppliers to the internet. | ||
proxies: | ||
# Name of the proxy. It will be used to reference in a supplier. | ||
# Must be unique. | ||
# Required. | ||
# TODO_CONSIDERATION: if we enforce DNS compliant names, it can potentially | ||
# become handy in the future. | ||
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-subdomain-names | ||
- proxy_name: http-example | ||
# Type of proxy: currently only http is supported but will support more | ||
# (https, tcp, quic ...) in the future. | ||
# MUST match the type of the supplier. | ||
# Required. | ||
type: http | ||
# Hostname to open port on. | ||
# Use 0.0.0.0 in containerized environments. | ||
# 127.0.0.1 with a reverse proxy when there's another process on localhost | ||
# that can be used as a reverse proxy (nginx, apache, traefik, etc.). | ||
# Required. | ||
host: 127.0.0.1:8080 | ||
|
||
# TODO_IMPROVE: https is not currently supported, but this is how it could potentially look. | ||
# - name: example-how-we-can-support-https | ||
# type: https | ||
# host: 0.0.0.0:8443 | ||
# tls: | ||
# enabled: true | ||
# certificate: /path/to/crt | ||
# key: /path/to/key | ||
|
||
# Suppliers are different services that can be offered through RelayMiner. | ||
# When a supplier is configured to use a proxy and staked appropriately, | ||
# the relays will start flowing through RelayMiner. | ||
suppliers: | ||
# The serviceId the supplier offered to the network . | ||
# It must match the Service.Id of the service that has been staked for. | ||
# Must be unique. | ||
# Required. | ||
- service_id: ethereum | ||
# Type of how the supplier offers service through the network. | ||
# Must match the type of the proxy the supplier is connected to. | ||
# Required. | ||
type: http | ||
# Configuration of the service offered through RelayMiner. | ||
service_config: | ||
# URL RelayMiner proxies the requests to. | ||
red-0ne marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# Also known as the data node, or service node in some cases. | ||
# Required. | ||
url: http://anvil.servicer:8545 | ||
# Authentication for the service. | ||
# HTTP Basic Auth: https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication | ||
# Optional. | ||
authentication: | ||
username: user | ||
password: pwd | ||
|
||
# TODO_IMPROVE: This is not supported in code yet, | ||
# but some services authenticate via a header. | ||
# Example, if the service requires a header like `Authorization: Bearer <PASSWORD>` | ||
# Authorization: Bearer <PASSWORD> | ||
# Optional. | ||
headers: {} | ||
|
||
# A list of hosts the proxy is accepting requests from. | ||
# When linked to the proxy, each host is going to be used to lookup the | ||
# the Supplier.Service in Pocket Network. | ||
# Each host in the list must match a Supplier.Service.Endpoint that the Supplier | ||
# has advertised on-chain when staking for that Service. | ||
# There are various reasons to having multiple hosts for the same supplier services, | ||
# - The on-chain Supplier may provide the same Service on multiple domains | ||
# (e.g. for different regions). | ||
# - The operator may want to route requests of different RPC types to | ||
# the same proxy. | ||
# - Migrating from one domain to another. Where the operator could still | ||
# accept requests on the old domain while the new domain is being propagated. | ||
# - The operator may want to have a different domain for internal requests. | ||
# - The on-chain Service configuration accepts multiple endpoints. | ||
# Must be unique within the proxy it is referenced in. | ||
# Required. | ||
hosts: | ||
- ethereum.devnet1.poktroll.com | ||
# The `service_id` of the supplier is automatically added to the hosts section | ||
# for potential troubleshooting/debugging purposes such as: | ||
# Having internal requests coming from non-FQDNs because of complex routing. | ||
# Sending requests from k8s pods. | ||
# Specify the `host` in curl requests when testing `curl -H "Host: ethereum" ...` | ||
# and make the proxy server process the request without the need for an | ||
# on-chain Endpoint entry. | ||
# - ethereum # <- this part is added automatically. | ||
|
||
# Names of proxies that this supplier is connected to. | ||
# This MUST correspond to the `proxy_name` entry in the `proxies` section | ||
# in order for the supplier to be available to the external network. | ||
# Required. | ||
proxy_names: | ||
- http-example # when the RelayMiner server builder runs. | ||
- service_id: 7b-llm-model | ||
type: http | ||
service_config: | ||
url: http://llama-endpoint | ||
hosts: | ||
- 7b-llm-model.devnet1.poktroll.com | ||
# - 7b-llm-model # <- this part is added automatically. | ||
proxy_names: | ||
- http-example |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be just
relayminers
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we don't have any proxy/load banalncer yet, the port needs to be added here. Also the
tcp://
scheme is here for url validation, it is then stripped to only take the host part.