missing node monitor statistics

The Node Monitor server encountered issues and needed to be rebooted. Please check your earnings in app. Ignore things that look odd on the node monitor.

Earnings in the app are down for shard3, also no stats for epoch 5850 in the app.

app as in monitor.incognito.org or the phone app?

In the app more > power > node > monitor detail. Not sure where is the information coming from there but isn’t it supposed to be the same as the browser page ?

We now advise node operators to only reference the monitor.incognito.org page as it is more detailed and shows a more accurate status.

The recommended page shows decreased rewards for shard 3?

Thank you for the screenshot @radonm. I have pinged the devs, let’s wait for their input.

1 Like

Oh, yeah, you’re right, shard 3 looks like a bad shard to validate. :slightly_smiling_face:

Screen Shot 2022-06-09 at 11.39.17

Shard 5 also has some lower rewards, all other shards are > 7 PRV
Screen Shot 2022-06-09 at 11.41.14

Funny thing btw. I have no node in shard7 according to the node monitor. Is it special or is it just the nature of randomness? =)

Shard assignment is by demand I think, I have no validator in shard 1.

Things are back to normal in epoch 5856, 5857 still in progress…

1 Like

Shard 3 back to normal rate.

1 Like

Shards 4 and 5 are earning less that others:
image

Shard 1 is giving 7.2PRV at the moment while shard 6 is giving 6.6PRV. If I understand correctly, this is due to shards not producing all the blocks they should? If this is what we’re supposed to expect or if the team is facing issues with the block proposers right now, I have no idea. Maybe @Support can shed a light on how high rewards we should expect and what (if anything) is going wrong.

2 Likes

No stats for epoch 5887
image

Thank you for the report @radonm, while the vote count % shows (-) this should not impact earnings. I’ve pinged the devs for them to look into this.

You’re definitely having problems with your servers lately. Do you need help with hosting from the community? :wink:

Epoch 5891 stats missing for shard 5, reward was only 6.6059
Epoch 5892 stats missing for shard 6, got full reward 7.2706

Is all the incognito code on github? Was trying to find the incognito-node-monitor backend, that serves /pubkeystat/ but that seems to be closed source, or did I just miss it?

1 Like

For reward, the monitor just show what it see when beacon chain distribute reward to a shard. Shard5 recently is unstable, the number of created block is less than other shard, so the reward is also less than the others.
The vote stat has certain problem, but if you are not getting slashed or get 0 reward, everything is fine. We are looking at vote stat problem.

It seems Shard6 is getting unstable as well
image

To make things more interesting I have 2 validators in shard 6 getting different rewards for epoch 5902
image
image
@Support what is going on ?

1 Like

@Support here we go again, 5967 stats missing on my nodes in shards 3, 4, 5, and 6. Got two nodes in shard 2 that is showing epoch 5967 though.

Is it possible, for us whiny bitches, to run the node-monitor locally? I only found the front-end code on github, Is the back-end code closed source?