Code

Mempool improvements#6640

Code

Merged

andreibancioiu merged 161 commits intorc/spica-patch-mempoolmultiversx/mx-chain-go:rc/spica-patch-mempoolfrom feat/mempoolmultiversx/mx-chain-go:feat/mempoolCopy head branch name to clipboardDec 5, 2024

+1,619-579

Commits

Commits on Jul 9, 2024

• Reference newer storage-go.

  andreibancioiu

  committed
  859fd8d

  Copy full SHA for 859fd8d

• Fix tests.

  andreibancioiu

  committed
  10c67f1

  Copy full SHA for 10c67f1

Commits on Jul 10, 2024

• Newer storage-go.

  andreibancioiu

  committed
  fd214d6

  Copy full SHA for fd214d6

• Adjust NumTxPerSenderBatchForFillingMiniblock, MaxGasBandwidthPerBatchPerSender.

  andreibancioiu

  committed
  8f92b0c

  Copy full SHA for 8f92b0c

• Update go.mod.

  andreibancioiu

  committed
  3ea4345

  Copy full SHA for 3ea4345

• Fix tests.

  andreibancioiu

  committed
  3b2f40e

  Copy full SHA for 3b2f40e

Commits on Jul 15, 2024

• Integrate new selection parameter.

  andreibancioiu

  committed
  1202873

  Copy full SHA for 1202873

• Adjust / skip test.

  andreibancioiu

  committed
  fea4455

  Copy full SHA for fea4455

• Some TODOs, update reference of storage-go.

  andreibancioiu

  committed
  61efec0

  Copy full SHA for 61efec0

Commits on Jul 16, 2024

• Optimize shardedTxPool.removeTxBulk.

  andreibancioiu

  committed
  81e742a

  Copy full SHA for 81e742a

Commits on Jul 17, 2024

• Testing workaround - notify account nonce when adding tx in pool.

  andreibancioiu

  committed
  6e7d92d

  Copy full SHA for 6e7d92d

• Update reference.

  andreibancioiu

  committed
  17aa534

  Copy full SHA for 17aa534

• Extra check on workaround for testing.

  andreibancioiu

  committed
  618f2f0

  Copy full SHA for 618f2f0

Commits on Jul 26, 2024

• Pass epoch notifier to sharded TX pool.

  andreibancioiu

  committed
  04a2764

  Copy full SHA for 04a2764

• calculate hash only on log trace level set.

  cristure

  committed
  6841919

  Copy full SHA for 6841919

Commits on Jul 29, 2024

• add non-memory thrasher test.

  cristure

  committed
  85d93ac

  Copy full SHA for 85d93ac

• Minor refactoring / cleanup.

  andreibancioiu

  committed
  3a2a25a

  Copy full SHA for 3a2a25a

Commits on Jul 30, 2024

• Define account nonce provider. Pass epoch notifier and account nonce provider to sharded tx pool.

  andreibancioiu

  committed
  20f174f

  Copy full SHA for 20f174f

• Fix some tests.

  andreibancioiu

  committed
  652d417

  Copy full SHA for 652d417

• In sharded tx pool, on tx add, notify cache about sender nonce.

  andreibancioiu

  committed
  9063904

  Copy full SHA for 9063904

Commits on Jul 31, 2024

• Fix additional tests.

  andreibancioiu

  committed
  c8f4d6d

  Copy full SHA for c8f4d6d

• Fix short test.

  andreibancioiu

  committed
  f126716

  Copy full SHA for f126716

• Fix linter.

  andreibancioiu

  committed
  8a4f344

  Copy full SHA for 8a4f344

• Fix long tests (return err in nonce provider).

  andreibancioiu

  committed
  416ad59

  Copy full SHA for 416ad59

Commits on Aug 6, 2024

• Adjust log, tests. Plus minor refactoring.

  andreibancioiu

  committed
  25cd44f

  Copy full SHA for 25cd44f

Commits on Aug 8, 2024

• Merge branch 'master' into master-into-mempool-2024-09-08

  andreibancioiu

  committed
  8bde909

  Copy full SHA for 8bde909

• Merge branch 'master' into mempool-10

  andreibancioiu

  committed
  ce579e3

  Copy full SHA for ce579e3

• Merge pull request #6381 from multiversx/master-into-mempool-2024-09-08

  Show description for c48f5d2
  andreibancioiu

  authored
  c48f5d2

  Copy full SHA for c48f5d2

Commits on Aug 20, 2024

• Add TODO.

  andreibancioiu

  committed
  665c53c

  Copy full SHA for 665c53c

• Merge branch 'master' into mempool-10

  andreibancioiu

  committed
  2d641f5

  Copy full SHA for 2d641f5

• Merge branch 'feat/mempool-plus-plus' into mempool-10

  andreibancioiu

  authored
  8125028

  Copy full SHA for 8125028

• Merge branch 'master' into master-into-feat-mempool-plus-plus-2024-08-20

  andreibancioiu

  committed
  53c679b

  Copy full SHA for 53c679b

• Merge pull request #6397 from multiversx/master-into-feat-mempool-plus-plus-2024-08-20

  Show description for 52058da
  andreibancioiu

  authored
  52058da

  Copy full SHA for 52058da

• Merge branch 'feat/mempool-plus-plus' into mempool-10

  andreibancioiu

  authored
  bf90aff

  Copy full SHA for bf90aff

Commits on Sep 3, 2024

• Merge branch 'rc/v1.7.next1' into benchmark_trie

  AdoAdoAdo

  authored
  79eb08c

  Copy full SHA for 79eb08c

Commits on Sep 6, 2024

• Reference mempool (update).

  andreibancioiu

  committed
  5f8d3d4

  Copy full SHA for 5f8d3d4

• Merge branch 'mempool-10' of ssh://github.com/multiversx/mx-chain-go into mempool-10

  andreibancioiu

  committed
  d4c35bc

  Copy full SHA for d4c35bc

• Merge branch 'rc/v1.7.next1' into benchmark_trie

  andreibancioiu

  authored
  1ec7c5c

  Copy full SHA for 1ec7c5c

• Convert artificial test into a benchmark.

  andreibancioiu

  committed
  af01f4a

  Copy full SHA for af01f4a

Commits on Sep 18, 2024

• Alter log level.

  andreibancioiu

  committed
  365f31b

  Copy full SHA for 365f31b

Commits on Nov 4, 2024

• Merge pull request #6579 from multiversx/master

  Show description for 050e9e1
  andreibancioiu

  authored
  050e9e1

  Copy full SHA for 050e9e1

• Merge branch 'feat/mempool-plus-plus' into mempool-10

  andreibancioiu

  committed
  4166323

  Copy full SHA for 4166323

• Integrate new mempool. Fix short tests.

  andreibancioiu

  committed
  72b7534

  Copy full SHA for 72b7534

• Fix some long tests.

  andreibancioiu

  committed
  cd5076b

  Copy full SHA for cd5076b

Commits on Nov 5, 2024

• Bad txs should be logged with "debug" level (low incidence).

  andreibancioiu

  committed
  77ae8af

  Copy full SHA for 77ae8af

• Integrate selection with txs limit.

  andreibancioiu

  committed
  dd3729d

  Copy full SHA for dd3729d

• Fix interface.

  andreibancioiu

  committed
  8349dcf

  Copy full SHA for 8349dcf

• Fix tests.

  andreibancioiu

  committed
  5c9f999

  Copy full SHA for 5c9f999

• Fix constant.

  andreibancioiu

  committed
  4ae8d38

  Copy full SHA for 4ae8d38

• shouldNotifyCacheAboutSenderNonce (or not).

  andreibancioiu

  committed
  84e41dc

  Copy full SHA for 84e41dc

Commits on Nov 6, 2024

• Reference newest storage-go.

  andreibancioiu

  committed
  9fc3b29

  Copy full SHA for 9fc3b29

• Reference newer storage-go.

  andreibancioiu

  committed
  ccad4bf

  Copy full SHA for ccad4bf

• Call notifyTransactionProviderIfNeeded() within createScheduledMiniBlocks().

  andreibancioiu

  committed
  18698c1

  Copy full SHA for 18698c1

• Remove code not needed (was thought to be useful for the initial mempool improvement proposal).

  andreibancioiu

  committed
  caaf1ef

  Copy full SHA for caaf1ef

• Fix concurrency issue.

  andreibancioiu

  committed
  72a8f5a

  Copy full SHA for 72a8f5a

• Do not alter log level anymore.

  andreibancioiu

  committed
  f00bbde

  Copy full SHA for f00bbde

Commits on Nov 7, 2024

• Optimize cleanup around scheduled transactions.

  andreibancioiu

  committed
  0b39162

  Copy full SHA for 0b39162

• Merge branch 'rc/v1.7.next1' into selection-by-ppu

  andreibancioiu

  committed
  3d9cb44

  Copy full SHA for 3d9cb44

• Fix some long tests (mempool).

  andreibancioiu

  committed
  1a5806e

  Copy full SHA for 1a5806e

• Adjust call of "notifyTransactionProviderIfNeeded".

  andreibancioiu

  committed
  3962e4d

  Copy full SHA for 3962e4d

Commits on Nov 8, 2024

• Undo changes related to scheduled txs cleanup.

  andreibancioiu

  committed
  fdb86cb

  Copy full SHA for fdb86cb

Commits on Nov 11, 2024

• Reference new storage-go. Send account notifications upon creating / executing scheduled miniblocks.

  andreibancioiu

  committed
  c8e978e

  Copy full SHA for c8e978e

• Commit changes to go.mod.

  andreibancioiu

  committed
  d4a53bf

  Copy full SHA for d4a53bf

• Fix go.sum.

  andreibancioiu

  committed
  23634d7

  Copy full SHA for 23634d7

Commits on Nov 12, 2024

• Reference new storage go and fix long tests.

  andreibancioiu

  committed
  248813f

  Copy full SHA for 248813f

• When reverting a block, ForgetAllAccountNoncesInMempool.

  andreibancioiu

  committed
  a585719

  Copy full SHA for a585719

• Fix mock.

  andreibancioiu

  committed
  22026c6

  Copy full SHA for 22026c6

• Fix after self-review.

  andreibancioiu

  committed
  5f8d03f

  Copy full SHA for 5f8d03f

• Fix after self review (refactor, comments, additional tests).

  andreibancioiu

  committed
  5db19b5

  Copy full SHA for 5db19b5

• Additional unit tests.

  andreibancioiu

  committed
  cc2eda5

  Copy full SHA for cc2eda5

• Fix import.

  andreibancioiu

  committed
  6ee6292

  Copy full SHA for 6ee6292

• Fix linter issues.

  andreibancioiu

  committed
  a498ef0

  Copy full SHA for a498ef0

Commits on Nov 13, 2024

• Fix tests.

  andreibancioiu

  committed
  878d7b6

  Copy full SHA for 878d7b6

• Fix after review (part 1).

  andreibancioiu

  committed
  90bbf04

  Copy full SHA for 90bbf04

• Fix after review (part 2).

  andreibancioiu

  committed
  e0590fe

  Copy full SHA for e0590fe

• fix spamming warnings from apiTransactionProcessor

  sstanculeanu

  committed
  ba08014

  Copy full SHA for ba08014

• No more ForgetAllAccountNonces().

  andreibancioiu

  committed
  febe899

  Copy full SHA for febe899

• Remove NotifyAccountNonce().

  andreibancioiu

  committed
  535d51b

  Copy full SHA for 535d51b

• Properly invoke NewTxCache().

  andreibancioiu

  committed
  bca90d9

  Copy full SHA for bca90d9

• Reference new storage-go.

  andreibancioiu

  committed
  278acf9

  Copy full SHA for 278acf9

Commits on Nov 14, 2024

• Fix long test.

  andreibancioiu

  committed
  f533a2b

  Copy full SHA for f533a2b

• Merge branch 'master' into selection-by-ppu

  andreibancioiu

  committed
  43ae39d

  Copy full SHA for 43ae39d

• Merge pull request #6604 from multiversx/fix_spam_logs

  Show description for 20dde29
  sstanculeanu

  authored
  20dde29

  Copy full SHA for 20dde29

• Newer storage (upon fix after review).

  andreibancioiu

  committed
  aae947b

  Copy full SHA for aae947b

• Merge branch 'selection-by-ppu' of ssh://github.com/multiversx/mx-chain-go into selection-by-ppu

  andreibancioiu

  committed
  b6d55cb

  Copy full SHA for b6d55cb

• Fix memory tests.

  andreibancioiu

  committed
  eda1a61

  Copy full SHA for eda1a61

• Newer storage-go.

  andreibancioiu

  committed
  4b8fb0a

  Copy full SHA for 4b8fb0a

• Merge branch 'selection-by-ppu' into MX-16107-no-more-notify

  andreibancioiu

  committed
  a98f0a4

  Copy full SHA for a98f0a4

• Sketch CS test: TestMempoolWithChainSimulator_Eviction.

  andreibancioiu

  committed
  0cb7187

  Copy full SHA for 0cb7187

• Merge branch 'rc/v1.7.next1' into benchmark_trie

  andreibancioiu

  authored
  bc70dcc

  Copy full SHA for bc70dcc

• Merge pull request #6338 from multiversx/benchmark_trie

  Show description for 6a80857
  andreibancioiu

  authored
  6a80857

  Copy full SHA for 6a80857

• Optimize GetExistingAccount(). Only encode address if log level is TRACE.

  andreibancioiu

  committed
  c4f2db5

  Copy full SHA for c4f2db5

• Merge branch 'MX-16107-no-more-notify' of ssh://github.com/multiversx/mx-chain-go into MX-16107-no-more-notify

  andreibancioiu

  committed
  1cb7594

  Copy full SHA for 1cb7594

Commits on Nov 15, 2024

• Merge branch 'rc/spica-patch-mempool' into selection-by-ppu

  andreibancioiu

  authored
  ea50a40

  Copy full SHA for ea50a40

• Use proper tag for storage-go.

  andreibancioiu

  committed
  3ddc19f

  Copy full SHA for 3ddc19f

• Pass account nonce provider (backed by the accounts adapter) to SelectTransactions().

  andreibancioiu

  committed
  26ddc02

  Copy full SHA for 26ddc02

• Optimized logging. Don't directly encode to hex before calling log.Trace(). Encoding will happen under the hood (if trace is active).

  andreibancioiu

  committed
  38aeb55

  Copy full SHA for 38aeb55

• Remove logic around "accountTxsShards" (was only needed for mempool notifications).

  andreibancioiu

  committed
  297e530

  Copy full SHA for 297e530

• Remove code not needed anymore.

  andreibancioiu

  committed
  e1d4287

  Copy full SHA for e1d4287

• Minor refactoring around nonce account provider.

  andreibancioiu

  committed
  f84a3c6

  Copy full SHA for f84a3c6

• Sketch additional CS tests (WIP).

  andreibancioiu

  committed
  9aec5b9

  Copy full SHA for 9aec5b9

Commits on Nov 18, 2024

• AccountNonceProvider becomes AccountStateProvider (more information from account state is necessary).

  andreibancioiu

  committed
  5b937db

  Copy full SHA for 5b937db

• Merge branch 'selection-by-ppu' into MX-16107-no-more-notify

  andreibancioiu

  committed
  7fa9a6c

  Copy full SHA for 7fa9a6c

• Fix tests, reference newer storage.

  andreibancioiu

  committed
  7741b4e

  Copy full SHA for 7741b4e

Commits on Nov 19, 2024

• Fix long test.

  andreibancioiu

  committed
  da36969

  Copy full SHA for da36969

• Optimizations. Fix after self-review.

  andreibancioiu

  committed
  377cca5

  Copy full SHA for 377cca5

• Fix old concurrency issue.

  andreibancioiu

  committed
  5bb1b40

  Copy full SHA for 5bb1b40

• Fix linter issues.

  andreibancioiu

  committed
  0d295b8

  Copy full SHA for 0d295b8

• Remove unused field.

  andreibancioiu

  committed
  f87f1c3

  Copy full SHA for f87f1c3

• Sketch a simulator for transactions selection - called in CreateBlockStarted() - works for observers, as well.

  andreibancioiu

  committed
  4834372

  Copy full SHA for 4834372

• Feed guardian checker into process/block/preprocess/transactions.

  andreibancioiu

  committed
  9ff7f5c

  Copy full SHA for 9ff7f5c

• Enhance accountStateProvider - make it know about guardians.

  andreibancioiu

  committed
  0b5c262

  Copy full SHA for 0b5c262

Commits on Nov 20, 2024

• Reference newer storage-go.

  andreibancioiu

  committed
  cf4f1a8

  Copy full SHA for cf4f1a8

• Fix tests.

  andreibancioiu

  committed
  1faa7f1

  Copy full SHA for 1faa7f1

• Newer storage go. Additional tracing.

  andreibancioiu

  committed
  5b00521

  Copy full SHA for 5b00521

• Merge pull request #6581 from multiversx/selection-by-ppu

  Show description for 4d7e013
  andreibancioiu

  authored
  4d7e013

  Copy full SHA for 4d7e013

• Merge branch 'feat/mempool' into MX-16107-no-more-notify

  andreibancioiu

  authored
  7953116

  Copy full SHA for 7953116

• Fix tests.

  andreibancioiu

  committed
  e1cb834

  Copy full SHA for e1cb834

Commits on Nov 21, 2024

• Reference new storage-go.

  andreibancioiu

  committed
  eca8029

  Copy full SHA for eca8029

• Fix tests, fix after self-review.

  andreibancioiu

  committed
  9005e76

  Copy full SHA for 9005e76

• Remove chain simulator test (since it's not polished). Will be added at a later time.

  andreibancioiu

  committed
  cfd1ea8

  Copy full SHA for cfd1ea8

Commits on Nov 22, 2024

• In "interceptedHeader", handle concurrent access.

  andreibancioiu

  committed
  e02d607

  Copy full SHA for e02d607

Commits on Nov 24, 2024

• Transaction selection - call "VerifyGuardian" within AccountStateProvider.

  andreibancioiu

  committed
  aacbedc

  Copy full SHA for aacbedc

• Handle non-executable transactions related to guardians in a separate PR.

  andreibancioiu

  committed
  2d52815

  Copy full SHA for 2d52815

Commits on Nov 25, 2024

• Fix linter issues.

  andreibancioiu

  committed
  2a160f3

  Copy full SHA for 2a160f3

• Merge branch 'MX-16107-no-more-notify' into MX-16179-guardians

  andreibancioiu

  committed
  d36c7c5

  Copy full SHA for d36c7c5

• Add some unit tests.

  andreibancioiu

  committed
  7d0b272

  Copy full SHA for 7d0b272

• Rename state provider to selection session.

  andreibancioiu

  committed
  7da7ef3

  Copy full SHA for 7da7ef3

• (final) fix after self-review.

  andreibancioiu

  committed
  47159a5

  Copy full SHA for 47159a5

• Merge branch 'MX-16107-no-more-notify' into MX-16179-guardians

  andreibancioiu

  committed
  da02598

  Copy full SHA for da02598

• Reference newer storage-go.

  andreibancioiu

  committed
  7376985

  Copy full SHA for 7376985

Commits on Nov 26, 2024

• Sketch GetTransferredValue(). Work in progress.

  andreibancioiu

  committed
  1cb65c2

  Copy full SHA for 1cb65c2

• Additional tests.

  andreibancioiu

  committed
  16e35e2

  Copy full SHA for 16e35e2

• Fix after review.

  andreibancioiu

  committed
  25b2594

  Copy full SHA for 25b2594

Commits on Nov 27, 2024

• Merge branch 'MX-16107-no-more-notify' into MX-16179-guardians

  andreibancioiu

  committed
  cdc543f

  Copy full SHA for cdc543f

• Merge branch 'MX-16179-guardians' into MX-16160-consumed-balance

  andreibancioiu

  committed
  c1695d3

  Copy full SHA for c1695d3

• Update reference to storage-go.

  andreibancioiu

  committed
  6090ae4

  Copy full SHA for 6090ae4

• Merge pull request #6603 from multiversx/MX-16107-no-more-notify

  Show description for e04679e
  andreibancioiu

  authored
  e04679e

  Copy full SHA for e04679e

• Merge branch 'feat/mempool' into MX-16179-guardians

  andreibancioiu

  authored
  90d05c0

  Copy full SHA for 90d05c0

Commits on Nov 28, 2024

• Fix after review.

  andreibancioiu

  committed
  f2f6acf

  Copy full SHA for f2f6acf

• Merge branch 'MX-16179-guardians' into MX-16160-consumed-balance

  andreibancioiu

  committed
  959affe

  Copy full SHA for 959affe

• Merge pull request #6628 from multiversx/MX-16179-guardians

  Show description for 960b81e
  andreibancioiu

  authored
  960b81e

  Copy full SHA for 960b81e

• Fix after review. Benchmark.

  andreibancioiu

  committed
  6e33cbf

  Copy full SHA for 6e33cbf

• Merge branch 'feat/mempool' into MX-16160-consumed-balance

  andreibancioiu

  authored
  59d7ec7

  Copy full SHA for 59d7ec7

• Fix long test.

  andreibancioiu

  committed
  7eed6f5

  Copy full SHA for 7eed6f5

• Merge pull request #6637 from multiversx/MX-16160-consumed-balance

  Show description for 03e21bf
  AdoAdoAdo

  authored
  03e21bf

  Copy full SHA for 03e21bf

Commits on Nov 29, 2024

• Sketch mempool host. Move "GetTransferredValue()".

  andreibancioiu

  committed
  619e579

  Copy full SHA for 619e579

• Add tests, reference storage-go.

  andreibancioiu

  committed
  33bf019

  Copy full SHA for 33bf019

• Fix after review.

  andreibancioiu

  committed
  9428366

  Copy full SHA for 9428366

• Implement an ephemeral accounts cache within a selection session.

  andreibancioiu

  committed
  8185866

  Copy full SHA for 8185866

• Merge pull request #6641 from multiversx/MX-16211-optimize-get-value

  Show description for d556eb8
  andreibancioiu

  authored
  d556eb8

  Copy full SHA for d556eb8

• Merge branch 'feat/mempool' into MX-16223-mempool-memoization

  andreibancioiu

  authored
  9b1afba

  Copy full SHA for 9b1afba

• Reference newer storage-go.

  andreibancioiu

  committed
  91c17ad

  Copy full SHA for 91c17ad

• Merge branch 'MX-16223-mempool-memoization' of ssh://github.com/multiversx/mx-chain-go into MX-16223-mempool-memoization

  andreibancioiu

  committed
  ffaeb85

  Copy full SHA for ffaeb85

• Add unit test, fix after review (references to storage-go).

  andreibancioiu

  committed
  389c78b

  Copy full SHA for 389c78b

• Merge pull request #6643 from multiversx/MX-16223-mempool-memoization

  Show description for a7c336e
  andreibancioiu

  authored
  a7c336e

  Copy full SHA for a7c336e

Commits on Dec 2, 2024

• Adjust TxDataPool.SizePerSender.

  andreibancioiu

  committed
  881d8f5

  Copy full SHA for 881d8f5

• Merge pull request #6644 from multiversx/MX-16202-sender-constraints

  Show description for 8bce7d1
  andreibancioiu

  authored
  8bce7d1

  Copy full SHA for 8bce7d1

• Optimization: lower "maxNum" when selecting transactions.

  andreibancioiu

  committed
  d75b3eb

  Copy full SHA for d75b3eb

• Reference storage-go.

  andreibancioiu

  committed
  801c199

  Copy full SHA for 801c199

• Merge pull request #6645 from multiversx/MX-16207-selection-max-num

  Show description for 11fc7c6
  andreibancioiu

  authored
  11fc7c6

  Copy full SHA for 11fc7c6