Nodes & Shard Syncing Behavior

OR EVERYTHING YOU DIDN’T WANT TO KNOW ABOUT SYNCING, NOR WERE EVER GOING TO ASK :rofl:

I’m currently trying to resolve a shard stall on Shard 0 for one of my Nodes. I’ve deleted the synced shard data hoping a fresh sync will fix the issue.

Jury’s still out on that, but in the meantime …
Morgan Freeman: Jury’s back and … no, did not fix the issue.

If you’ve deleted your shard data and then followed the progress via the Node Monitor, you will notice your Node syncs beacon blocks for a while then starts randomly syncing shard blocks some time later.

My Node synced just beacon blocks until the beacon height reached ~140,000. Then random other shards started syncing too.

But is that really the case? Does the node start syncing the shards in a (pseudo-)random manner?

TL;DR No, not random at all. It’s deterministic based on your Node’s past committee participation.

Let’s explore that further.

To confirm my theory, I needed to consult the records I keep about my each of nodes. I record the date of every PRV reward. And I also record the epoch, shard, index slot of the Node and current PRV price, for each reward. The latter datapoints I added to my recordkeeping a few months after my first Node went online. Never know if or when that info might be useful.

This particular Node has been online since January 2020. It wasn’t until Epoch 906 that I started recording those other datapoints.

image

I recorded a reward for Epoch 906 on 3/25/2020 when this node was in Shard 6’s committee.

Each epoch is 350 blocks. Therefore 906 epochs * 350 blocks per epoch = 317,100 blocks. Once the synced beacon height of this node reaches ~317,100, I would be able to start confirming my hypothesis if any of the shards are yet to start syncing.

image

By that block height, only shard 0 has yet to start syncing. Scrolling forward in my records, I see that Node won’t join shard 0 until Epoch 1424.

image

A node selected for committee will be assigned a Pending role and enter the Preparation Phase. Upon entering the Preparation Phase a node will sync the assigned shard’s blockchain. Until very recently this would occur about 1.5 epochs ahead of an earning epoch. 350 blocks per epoch * 1.5 epochs = 525 blocks. Thus if Epoch 1424 is (350 blocks per epoch * 1424 epochs = 498,400 blocks), then the preparation phase should start 525 blocks earlier (498,400 - 525 = 497,875)

Therefore I expect – between beacon height ~497,875 - 498,400 – the Node Monitor tool should show shard 0 finally begins syncing, just as it would have done on 6/18/2020.

Approaching ~498,000 …

image

… and just after lower end of my predicted range …

image
 
Syncing, glorious syncing!

image
 
image
 
… this node earned from Epoch 1424 to Epoch 1426. (1426 * 350 = 499,100)

image

Huzzah! As I postulated, my node did not begin syncing shard 0 until the synced beacon height reaches the first epoch it was in committee for shard 0 and stops syncing that shard when the block height indicates it left committee.

Of course this sync behavior will be true for every Node, not just mine. This node was just an illustrative guinea pig.

Incidentally, this explains why shards continue to appear to randomly sync in short bursts while the beacon blockchain continuously syncs from zero. Each shard is able to sync blocks when the beacon height reaches an epoch(s) in which that node was in a given shard’s committee. Because the beacon sync advances the beacon block height faster than real-time, each shard may only sync a few thousand blocks before the synced beacon height reaches a block where the node is no longer in committee or has joined another shard’s committee.

Pretty neat, right?

13 Likes

Thank you for this explanation. I learned a lot.

1 Like