fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel #2278

rymnc · 2024-10-02T10:21:11Z

Linked Issues/PRs

follows up chore(gas_price_service): refactor gas price service #2192 with some changes to clean up the da block costs source.

Description

Exposed a broadcast::Sender as the shared_state of the DaBlockCostsSource, which is to be subscribed to when a service needs this dependency.

Checklist

Breaking changes are clearly marked as such in the PR description and changelog
New behavior is reflected in tests
The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

I have reviewed the code myself
I have created follow-up issues caused by this PR and linked them here

After merging, notify other teams

[Add or remove entries as needed]

Rust SDK
Sway compiler
Platform documentation (for out-of-organization contributors, the person merging the PR will do this)
Someone else?

MitchTurner · 2024-10-02T15:14:49Z

crates/services/gas_price_service/src/v1/da_source_adapter/service.rs

+    type SharedData = Sender<DaBlockCosts>;

    type Task = Self;

    type TaskParams = ();

-    fn shared_data(&self) -> Self::SharedData {}
+    fn shared_data(&self) -> Self::SharedData {
+        self.shared_state.clone()
+    }


Wouldn't it make sense for

type SharedData = Receiver<DaBlockCosts>;

and

fn shared_data(&self) -> Self::SharedData { self.data_broadcaster.subscribe() }

Receiver<DaBlockCosts> doesn't implement Clone, whereas Sender is :)
we can generate a new subscription with a Sender of tokio::sync::broadcast

Unfortunate. This is yet another example of why the RunnableService trait is a bad abstraction.

I didn't get why it is bad, could you elaborate?=)

Just, semantically, a Sender isn't shared data. It isn't even a means of getting shared data. It's the object that you need to expose to generate the object (a Receiver) by which you receive shared data.

So not only are we in blatant violation of SRP with the SharedState (see God Object), but we are also requiring the consumer to understand our underlying implementation, i.e. to get this data you must know the fields and the interfaces of the fields of the SharedData

Instead of having a shared_data() getter, we should just have a get_da_block_costs_receiver() method (this is kinda an ISP thing as well). But we are relying on an overly-generalized RunnableService abstraction that hides our business domain and requires us to rely on implementation details (as seen above).

I don't see where it is a God object.

The shared state is your object that has the get_da_block_costs_receiver method.

We can rename SharedState into ServiceAccessPoint.

Instead of exposing the service and implementing the get_da_block_costs_receiver on the service, we have an access point to the service via the SharedState. No one has access to the service except the service itself. All interactions with the service are done via its access point.

we are also requiring the consumer to understand our underlying implementation, i.e. to get this data you must know the fields and the interfaces of the fields of the SharedData

we should just have a get_da_block_costs_receiver() method

I don't see why you need to know the fields of the shared state... It is just an object with the get_da_block_costs_receiver method.

I don't see where it is a God object.

Sorry. The God object point doesn't apply here. I was thinking about our txpool service SharedState I was looking at recently:

pub struct SharedState< PSProvider, ConsensusParamsProvider, GasPriceProvider, WasmChecker, MemoryPool, > { pool: TxPool<PSProvider>, current_height: Arc<RwLock<BlockHeight>>, consensus_parameters_provider: Arc<ConsensusParamsProvider>, gas_price_provider: Arc<GasPriceProvider>, wasm_checker: Arc<WasmChecker>, memory: Arc<MemoryPool>, heavy_async_processor: Arc<HeavyAsyncProcessor>, utxo_validation: bool, }

All interactions with the service are done via its access point.

This should be "access points". And those access points should be defined by the domain of the service, not alllll possilbe services. When I'm saying we should have a get_da_block_costs_receiver() method, I'm saying that has a Single Responsiblility: providing a receiver.

Instead, we have to call service.shared_data().get_da_block_costs_receiver() and we have an arbitrary constraint that SharedData must be Clone, which isn't relevant here--we don't need to clone anything.

xgreenx · 2024-10-02T21:58:53Z

crates/services/gas_price_service/src/v1/da_source_adapter/service.rs

+    type SharedData = Sender<DaBlockCosts>;

    type Task = Self;

    type TaskParams = ();

-    fn shared_data(&self) -> Self::SharedData {}
+    fn shared_data(&self) -> Self::SharedData {
+        self.shared_state.clone()
+    }


I didn't get why it is bad, could you elaborate?=)

xgreenx · 2024-10-02T21:59:43Z

crates/services/gas_price_service/src/v1/da_source_adapter/service.rs

@@ -69,13 +66,15 @@ where
 {
    const NAME: &'static str = "DaBlockCostsService";

-    type SharedData = ();
+    type SharedData = Sender<DaBlockCosts>;


Can we use a separate structure, please? This structure can provide a subscribe method that will return a receiver. In this case we don't expose sender to external world.

addressed in c0a47db

xgreenx

LGTM=)

xgreenx · 2024-10-03T09:12:18Z

crates/services/gas_price_service/src/v1/da_source_adapter/service.rs

 pub use anyhow::Result;

+#[derive(Clone)]
+pub struct DaBlockCostsSharedState(Sender<DaBlockCosts>);


I think "DaBlockCosts" for service and shared state is redundant since it is already live inside of the v1::da_source_adapter.

Also I think "adapter" is overloads the ports and adapters. We don't use this property in context of this file. I think it just can be da_source or da_soruce_service.

not sure I understand, are you asking me to rename something?

okay I re-read it ~ renaming it

addressed in 0ad2191

xgreenx · 2024-10-03T09:12:31Z

crates/services/gas_price_service/src/v1/da_source_adapter/service.rs

-        sender: Sender<DaBlockCosts>,
-        poll_interval: Option<Duration>,
-    ) -> Self {
+    pub fn new(source: Source, poll_interval: Option<Duration>) -> Self {


Why poll_interval can be Option?

we want to specify a sensible default to poll the block committer

…hannel

@rymnc

## Version v0.37.0 ### Added - [1609](#1609): Add DA compression support. Compressed blocks are stored in the offchain database when blocks are produced, and can be fetched using the GraphQL API. - [2290](#2290): Added a new CLI argument `--graphql-max-directives`. The default value is `10`. - [2195](#2195): Added enforcement of the limit on the size of the L2 transactions per block according to the `block_transaction_size_limit` parameter. - [2131](#2131): Add flow in TxPool in order to ask to newly connected peers to share their transaction pool - [2182](#2151): Limit number of transactions that can be fetched via TxSource::next - [2189](#2151): Select next DA height to never include more than u16::MAX -1 transactions from L1. - [2162](#2162): Pool structure with dependencies, etc.. for the next transaction pool module. Also adds insertion/verification process in PoolV2 and tests refactoring - [2265](#2265): Integrate Block Committer API for DA Block Costs. - [2280](#2280): Allow comma separated relayer addresses in cli - [2299](#2299): Support blobs in the predicates. - [2300](#2300): Added new function to `fuel-core-client` for checking whether a blob exists. ### Changed #### Breaking - [2299](#2299): Anyone who wants to participate in the transaction broadcasting via p2p must upgrade to support new predicates on the TxPool level. - [2299](#2299): Upgraded `fuel-vm` to `0.58.0`. More information in the [release](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.58.0). - [2276](#2276): Changed how complexity for blocks is calculated. The default complexity now is 80_000. All queries that somehow touch the block header now are more expensive. - [2290](#2290): Added a new GraphQL limit on number of `directives`. The default value is `10`. - [2206](#2206): Use timestamp of last block when dry running transactions. - [2153](#2153): Updated default gas costs for the local testnet configuration to match `fuel-core 0.35.0`. ## What's Changed * fix: use core-test.fuellabs.net for dnsaddr resolution by @rymnc in #2214 * Removed state transition bytecode from the local testnet by @xgreenx in #2215 * Send whole transaction pool upon subscription to gossip by @AurelienFT in #2131 * Update default gas costs based on 0.35.0 benchmarks by @xgreenx in #2153 * feat: Use timestamp of last block when dry running transactions by @netrome in #2206 * fix(dnsaddr_resolution): use fqdn separator to prevent suffixing by dns resolvers by @rymnc in #2222 * TransactionSource: specify maximum number of transactions to be fetched by @acerone85 in #2182 * Implement worst case scenario for price algorithm v1 by @rafal-ch in #2219 * chore(gas_price_service): define port for L2 data by @rymnc in #2224 * Block producer selects da height to never exceed u64::MAX - 1 transactions from L1 by @acerone85 in #2189 * Weekly `cargo update` by @github-actions in #2236 * Use fees to calculate DA reward and avoid issues with Gwei/Wei conversions by @MitchTurner in #2229 * Protect against passing `i128::MIN` to `abs()` which causes overflow by @rafal-ch in #2241 * Acquire `da_finalization_period` from the command line by @rafal-ch in #2240 * Executor: test Tx_count limit with incorrect tx source by @acerone85 in #2242 * Minor updates to docs + a few typos fixed by @rafal-ch in #2250 * chore(gas_price_service): move algorithm_updater to fuel-core-gas-price-service by @rymnc in #2246 * Use single heavy input in the `transaction_throughput.rs` benchmarks by @xgreenx in #2205 * Enforce the block size limit by @rafal-ch in #2195 * feat: build ARM and AMD in parallel by @mchristopher in #2130 * Weekly `cargo update` by @github-actions in #2268 * chore(gas_price_service): split into v0 and v1 and squash FuelGasPriceUpdater type into GasPriceService by @rymnc in #2256 * feat(gas_price_service): update block committer da source with established contract by @rymnc in #2265 * Use bytes from `unrecorded_blocks` rather from the block from DA by @MitchTurner in #2252 * TxPool v2 General architecture by @AurelienFT in #2162 * Add value delimiter and tests args by @AurelienFT in #2280 * fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel by @rymnc in #2278 * fix(combined_database): syncing auxiliary databases on startup with custom behaviour by @rymnc in #2272 * fix: Manually encode Authorization header for eventsource_client by @Br1ght0ne in #2284 * Address `async-graphql` vulnerability by @MitchTurner in #2290 * Update the WASM compatibility tests for `0.36` release by @rafal-ch in #2271 * DA compression by @Dentosal in #1609 * Use different port for every version compatibility test by @rafal-ch in #2301 * Fix block query complexity by @xgreenx in #2297 * Support blobs in predicates by @Voxelot in #2299 **Full Changelog**: v0.36.0...v0.37.0

rymnc self-assigned this Oct 2, 2024

rymnc added the no changelog Skip the CI check of the changelog modification label Oct 2, 2024

rymnc marked this pull request as ready for review October 2, 2024 10:43

rymnc requested review from xgreenx, Dentosal and MitchTurner as code owners October 2, 2024 10:43

rymnc requested a review from a team October 2, 2024 10:43

MitchTurner reviewed Oct 2, 2024

View reviewed changes

rymnc requested a review from a team October 2, 2024 15:58

xgreenx reviewed Oct 2, 2024

View reviewed changes

rymnc force-pushed the chore/rework-block-committer-shared-state branch from c0a47db to 204b8a2 Compare October 3, 2024 06:27

xgreenx reviewed Oct 3, 2024

View reviewed changes

rymnc requested review from MitchTurner and xgreenx October 3, 2024 09:35

xgreenx approved these changes Oct 3, 2024

View reviewed changes

rymnc added 6 commits October 3, 2024 16:45

fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose c…

a9d3d78

…hannel

fix: assert the error

55d1925

fix: hide Sender behind newtype

b10b40c

fix: remove cache as the source guarantees uniqueness

9fd592b

fix: interval

8b88599

chore: rename files

5c80de0

rymnc force-pushed the chore/rework-block-committer-shared-state branch from 0ad2191 to 5c80de0 Compare October 3, 2024 11:15

Merge branch 'master' into chore/rework-block-committer-shared-state

0435740

MitchTurner approved these changes Oct 3, 2024

View reviewed changes

rymnc merged commit 2d11df2 into master Oct 3, 2024
38 checks passed

rymnc deleted the chore/rework-block-committer-shared-state branch October 3, 2024 14:19

xgreenx mentioned this pull request Oct 5, 2024

Release v0.37.0 #2302

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel #2278

fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel #2278

rymnc commented Oct 2, 2024

MitchTurner Oct 2, 2024 •

edited

Loading

rymnc Oct 2, 2024 •

edited

Loading

MitchTurner Oct 2, 2024

xgreenx Oct 2, 2024

MitchTurner Oct 3, 2024 •

edited

Loading

xgreenx Oct 3, 2024

MitchTurner Oct 3, 2024

MitchTurner Oct 3, 2024

MitchTurner Oct 3, 2024

xgreenx Oct 2, 2024

xgreenx Oct 2, 2024

rymnc Oct 2, 2024

xgreenx left a comment

xgreenx Oct 3, 2024

rymnc Oct 3, 2024

rymnc Oct 3, 2024

rymnc Oct 3, 2024

xgreenx Oct 3, 2024

rymnc Oct 3, 2024

fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel #2278

fix(da_block_costs): remove Arc<Mutex<>> on shared_state and expose channel #2278

Conversation

rymnc commented Oct 2, 2024

Linked Issues/PRs

Description

Checklist

Before requesting review

After merging, notify other teams

MitchTurner Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

rymnc Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MitchTurner Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xgreenx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MitchTurner Oct 2, 2024 •

edited

Loading

rymnc Oct 2, 2024 •

edited

Loading

MitchTurner Oct 3, 2024 •

edited

Loading