Documentation

How often do we check the queue

We estimate the size of the event with the number of the transactions in the block. By doing this we accumulate some number of blocks but with less than queueSize number of transactions. This approach helps to process blocks with a constant memory usage.

However, once we are in sync with the node, we want to process every block instead of batches of blocks so that we can update the database as frequently as possible.

Just accumulating queueSize blocks doesn't work as a block can have any number of transactions. It works fine at the beginning of the chain but later blocks grow in their size and the memory usage grows tremendously.

processEventsQueue reads events from TBQueue , collects enough RollForward s to append blocks at once.