feat: async block writers

[ata-nas]

Story:

AS A Block Node Operator
I WANT the Block Node to be as performant as possible
SO THAT I can minimize operating costs.

This task aims to improve the architecture and the performance of the local persistence handler.

DEPENDS ON: #309

Tech Notes:

Currently, the local block writer abstraction is neither async, nor stateless. To make it async, we need to detach the writer from having any other logic than simply writing a batch of files to the appropriate place. The writer must not be bothered with checking any state, making calculations or decisions "should or shouldn't I write some files to some place". The block writer must simply be supplied some data to persist and once it is flagged that it has received everything, it should then do it's work and be gone once it is done.

One important note is that we are constantly receiving the block stream, guaranteed that we have a batch of block items. Batches can start with a block header and can end with block proof. It absolutely cannot start with a block proof, nor can end with a block header! Also, it absolutely cannot have a block header or proof somewhere in the middle of the received batch. This allows us to very easily denote and distinguish where a block starts (we check that a received batch starts with a block header) and where it ends (we check that a received batch ends with a block proof), all batches of items received in between such batches (if any) are always of items for the given denoted block.

All that beings said, In order to achieve async writing of the block files, we need to do the following:

• simplify the block writer:
  • the block writer only knows where to write data that it is being supplied & how to write it
  • the block writer makes no decisions as to wether it must/must not write some data
  • each distinct block (collection of block items) must be supplied to a separate block writer, which then knows how to handle this data in an async fashion
• persistence handler is now the decision maker
  • block writers are now managed by the block persistence
  • shift the responsibility of deciding wether to persist a block or continue to accept items from the block writer (current implementation) to the persistence handler, which is anyhow receiving the stream, so it can make such decisions and is able to supply a block writer with the correct collection of items
  • each block writer has a transfer queue, where we will be pushing items from the persistence handler as they are received until we denote that a block has ended
  • after a block has ended, we flag the writer that it has received everything and it can continue asynchronously to proceed with writing all the items to the appropriate place
  • in the meantime, we are now pushing the following items for the next block in a new writer, managed by the persistence handler