Best way to verify node functionality

I am trying to determine the best way to verify my pNode is “working”.

Previously I’ve had the usual issues where it’s really “working” but the app doesn’t show it properly, and so I got used to ignoring the app when the app wasn’t showing any gains, as when I would unplug and replug it in they would suddenly show.

Now I withdrew my earnings over a month ago (beginning of February)- and haven’t earned since. The node status constantly says “working” (which is normally a good sign). However I know it couldn’t have been working because due to an IP Change with my service provider, and subsequent firewall changes, I discovered the nodes IP address had changed - so the app was saying “working” but none of the ports could reach it. I have since fixed this and reset it up, with the appropriate ports forwarding to the new IP.

However it hasn’t earned in over a month, so I was wondering - what is the best way to verify a pNode is actually on the network and able to be seen and chosen appropriately (since the app said it was “working” that whole time, but couldn’t have been).

(Also of note - I’m not sure how long the app or mainnet checks these things, but my pNode was unplugged and offline for an hour or two while I sorted out some firewall stuff - but the whole time the node appeared “working” - but eventually when replugged it in it DID show with its new IP, so I know it was seen.)

The pNode is plugged in, and wired directly to a switch, hardwired.

Thanks for any help.

2 Likes

Welcome to the forum.

There is always the search function. It will give you some results with info.
And @Chucky put this post together to list some community built tools.

2 Likes

@Jamie Thank you so much.

I definitely have been checking out Nito bot and it’s great. I believe the second link there has been down as of late (watch.incognito.market) as I have been trying that but the site isn’t responding. I appreciate the links and will check them out.

I searched through the forums a bit and tried some different things (opening up more ports, etc.).

The one issue I’m trying to pin down is why the pNode shows as “working” if it can’t be reached. (Which was the case for a bit.) The internal IP had changed - so the port forwarding was no longer valid. So whether 9334 or 9433, it would have been routed to an IP that was either dead, or not a node if it was reassigned at some point. But the whole time the app said “working”. I was just wondering if there was a way or tool to verify that type of situation.

As for Nito bot - I found it last week, and it’s been helpful. Just the right tool for the job. I really appreciate the work @Josh_Hamon has put in. The vast majority of the info I am looking for is now one message away.

Thanks again.

3 Likes

:+1: :+1: :+1:

Some users have stated they still earn even when they completely take their node offline. This is a problem with the fact that if a node has been registered with the validator key, it apparently can still be chosen even when not reachable. It will still earn, therefore, for being part of the committee even when it doesn’t contribute any validation. I could be wrong about this explanation, but that’s how I understand it. I haven’t seen any other explanation for why offline pNodes and vNodes still earn forever.

@Jamie @Chucky does my explanation seem right? I think this is a good chance to clear this up, since I imagine a lot of people would be confused when their offline nodes still earn despite everyone telling them you have to be online to be chosen as a validator, and only validators can earn.

1 Like

This is a known issue and will be fixed in the future versions. This is one of the reasons why the fixed nodes belonging to the team still exist. @int1

1 Like

The apps status is unfortunately unreliable. Often because the app can’t reach the node, while the Incognito network might have no problem reaching the node.

In most cases however the app would then show “offline”, while for the network the node is still online. Are you using the latest version of the app?

Yes. What you are describing is not what I am discussing. Many of us have shown that you can literally remove a node from the network (disconnect it from the internet, unplug it, etc.), and it will still earn. This is not an issue of an app showing the wrong status or being unreachable from one but not the other. This is about when a node is definitively unreachable because the pNode is unplugged and disconnected from internet or a vNode is totally shut down. Maybe what abduraman said is correct, though. Would like confirmation on what exactly the root of the known issue is.

This is a known side effect of the extended “temporary” current committee configuration.

WHY ARE THERE FIXED VALIDATORS?

There are 32 validators per each of the 8 shards in each epoch’s committee. 22 of those validators in each of the 8 shards are maintained by the core dev team. 22 is 2/3 of the current size of a shard’s committee. 2/3 of each shard’s committee are required to achieve consensus on the validity of each block mined by the shard. This insures that each shard can achieve consensus, even if all 10 of the community validators in each shard completely fail to participate in committee consensus.

The original roadmap intended this to a be a temporary situation. The “fixed validators” were to be released within 3 months of the mainnet genesis. This would give the network enough time to assure the stability of the community validators was fully demonstrated. During this period the mechanism for penalizing and removing non-participating validators – slashing – was not implemented. The non-participation of community validators was not detrimental to the network: the number of fixed validators insured that consensus would always be achieved during this short transition period.

Work began on scaling the shard committee sizes and it became apparent that the fixed validators could not be released until dynamic committee size was implemented. Slashing also could not be released until this mechanism was fully developed. As the development timeline for dynamic sharding shifted, the timeline for releasing the fixed validators was pushed out a few more months to 6 months of mainnet genesis, then a full year to April 2021, then to Q2 2021 and currently is planned for early Q3 2021.

This leaves the network in what was intended to be a temporary situation – non-participating validators are not slashed from the network and thus earn block rewards for committee participation.

BUT WHY ARE BLOCK REWARDS DISTRIBUTED TO THE NON-PARTICIPATING VALIDATORS?

Because there is no active slashing mechanism, a non-participating validator is seen as “participating” in committee when selected, despite producing no blocks. “Participating” here is effectively the same as “not slashed.” The block rewards are then distributed to all 32 validators in each shard’s committee because all 32 validators fall into the “participating”/“not slashed” definition and thus are eligible to receive an equal portion of a committee’s block rewards.

OK. SO WHY ARE PHYSICALLY OFFLINE VALIDATORS SELECTED TO COMMITTEE?

The eligibility for committee selection is “not slashed” and a properly staked 1750 PRV. Once a validator has properly staked 1750 PRV, the validator is known to the network by the BLS mining key stored on-chain in the pool of validators. There is no IP check, heartbeat or other mechanism to confirm online status of the validator’s node. The scope for this kind of validator node management will likely fall under/rely on the slashing mechanism. However as there is no active slashing, the mining key is never removed from the known pool of properly staked validators and will remain eligible for committee selection until unstaked (or slashing is activated).

9 Likes

And as a corollary – though the “fixed validators” are enough to acheive consensus alone, the community validators do actively participate in validating proposed blocks. The proof is literally on-chain.

This is the validation data for Shard 1’s block 1083807.

image

Here we see that community validators in index position 24, 29 and 31 validated this block.

[source] [alt source]

And four blocks later, Shard 1’s block 1083811:

image

A slightly different group of validators but still including some community validators.

[source] [alt source]

For any given committee, validator slots #23 through #32 (zero indexed 22 through 31) are the community validators. Anyone can literally follow community validators selected to Preparation Phase (aka role “Pending”) move into the bottom four slots of each committee at the beginning of each epoch. The previous community validators in slots #23-#28 are shifted up 4 slots. This pushes validators in slots #25 and #26 out of committee: this is why sometimes a validator earns for 3 consecutive epochs and sometimes for 2. It all depends on which slot a validator enters committee: slots #31 and #32 will earn for 3 epochs while slots #29 and #30 will earn for just 2. Selection to committee is random. Number of earning epochs while in committee is not random.

9 Likes

Damn Damn Damn it…thank you @Mike_Wagner!!! for finally giving us all an explanation that is clear and logical and makes sense…and that has nothing to do with FUD or conspiracy theories that have been but a bane upon the community of recently…thank you once again to Mike for his thorough explanation… :grinning: :sunglasses:

@Mike_Wagner Thank you so much. This may even explain some of the previous behavior I’ve seen over the past year(s)? (Trying to figure out exactly how long this pNode has been at it for tax purposes now - fun fun fun) (Also I see that will be made much easier in the next week!) Very enlightening.

Also @jamie, yes on the latest app version and node firmware as well I believe (1.0.5).