New Feature Announcement: Slashing in Sharding with Committee Lifecycle V2

Improvement

  • Faster candidates’ assignments into one shard chain.
  • Balance probability for candidates to be selected into a shard chain.
  • Improve Incognito chain security and stability of block-creation.
  • Enhance synchronization between the beacon chain and shard chains.
  • Shard chains automatically recover.

New Feature

  1. Slashing.
  2. Upgrade committee life cycle (Committee Life Cycle V2).
  3. Upgrade shard block-creation (Block Producing V2).
  4. Upgrade committee change method.
  5. Add Unstake feature.

How new features achieve improve Incognito chain

  1. Slashing

Improve security, toward a decentralized system. Bad validators, which don’t sign enough blocks in one epoch, will be slashed.

Slashed nodes will be removed from any committee lists. In the meanwhile, nodes’ stakes are returned to their account. In other words, users’ PRV would remain intact but their nodes will no longer play any roles in the system.

  1. Committee Life Cycle V2

Using Cuckoo Rules to balance the probability for candidates to be selected into a shard chain and faster candidates assignment into one shard chain.

Improve shard chain security by the ability to expand and shrink validators list in a more flexible approach.

  1. Block Producing V2

To create a new shard block, a shard chain must process a new beacon block with a higher height.

Enhance synchronization between shard chains and the beacon chain.

Shard chain is not able to ignore beacon instructions, which keep the system running correctly and stable.

  1. Upgrade committee change method

Regardless of the working progress of nodes in shard chains, new committee lists will be forced to use at the beginning of every epoch. All committee lists will be managed by the beacon chain, which is a more secure and stable chain.

Note: the combination of features 3 and 4 makes shard chains unable to disobey the beacon chain.

  1. Unstake feature

A convenience feature for users while still maintaining the security and stability of block-creation.

Nodes in the candidate role are allowed to use this feature. Nodes in other roles just try to stop the auto-stake mode.

Affected User

  • Users who stake, node investors
  • Blockchain developer
  • Anyone who is interested in sharding in Blockchain

Roadmap:

  1. From 2/5/2021, the new features source code is available on our official GitHub

  2. Committee Life Cycle V2, Unstake Feature: Week 2, May 2021

  3. Slashing: Week 2, June 2021

8 Likes

If this will be effective as of that time, it is so early since many unanswered questions and complaints are waiting to be answered in the many other posts.

3 Likes

Has slashing also been setup to remove bad nodes. Example, nodes that attempt to report false information or inaccurate amounts?

2 Likes

Nice idea @Jared, we are working on that problem. The team are considering some solutions and will publish them soon.

3 Likes

Slashing only affect nodes that have zero impact on the network. I see no reason why we have to delay this.

1 Like

In fact, the problem is already this. You may see the problems in the topics related to Monitor tool:


4 Likes

Does the team expect to resolve all outstanding node issues in just ONE month? I mean seriously? I have two pNode issues that have been outstanding with support since last fall. That’s months. Plural. Those pNodes still have exactly the same issue they had months ago. Yet my support case has been silent for months. And now – what – my issues will be resolved magically in less than 1 month otherwise those nodes will be forcibly unstaked on first committee substitution following the release of slashing?

This is beyond tone-deaf, I’m afraid. The community has been signaling for months that tools, issues and support thereof needed to be addressed well in advance of slashing. But the team has and continues to demure on this point.

Every community stake, whether through a pNode or a vNode, has been earning regardless of actual committee participation since the genesis block. Until the Node Monitor tool was just recently released, Node operators – pNode operators especially – have literally had no insight to the actual effectiveness of the Nodes they operated.

The only validation – if you can call it that – was via block rewards. Block rewards are currently distributed whether a community Node participated in voting or not, while in committee. This engendered a set-it-and-forget-it mindset for operators. But all this really validated was that a Node had been successfully staked and its BLS mining key indexed by the blockchain. That Node could then go offline forever, and still earn block rewards all the same.

Currently we see that over 1300 of the current ~2700+ Nodes indexed by the Incognito network aren’t producing enough (or any) votes for committee participation and will be subject to slashing in less than a month. This leaves just 1400 properly working validators. The Incognito team currently operates ~600 of those via Provide. Add in the 176 fixed Nodes and the Incognito team will still be operating half of the Incognito network after slashing.

The team cannot continue to kick this can down the road. Shard & beacon sync stalls, node stalls, offline, broken Docker updates and general issues need to be addressed for those 1300 validators. Documentation for resolving each type of identified issue needs to be made available immediately.

A process for pNode operators needs to be developed. pNode operators do not have terminal access to “fix” their Nodes. At best, a pNode operator could disassemble one side panel, permanently dislodging the wifi antenna, to access HDMI and USB ports. Many pNodes operators may not be technically inclined. Indeed that was a selling point of the pNode - no technical skills required. It’s presumptuous to now expect these operators to crack open a pNode, connect a monitor, keyboard and mouse and re-run run.sh, delete then pull fresh Docker tags, and otherwise resolve potentially complex software issues. And even then, none of that is even documented, so at best they would be cobbling together info spread across the forum, Telegram or however else they could.

Nevertheless, in one month, these operators will begin to have their pNodes forcibly unstaked and their only recourse will be to privately restake their pNode because the team has prematurely ended support for restoring funded staking.

This doesn’t even address the core issue surrounding that slashing event, which should be the main outcome of a slashing event. Was the Docker up-to-date? Maybe the blockchain storage database became corrupted. Or logs filled up the container blocking further writes?

As it is currently being proposed, I strongly believe continuing forward without addressing Node operator issues will have a significant negative impact on the number of validators on the network. The resulting turmoil is certain to continue eroding – if not accelerating the decline of – community confidence in the project.

11 Likes

The team totally understands this problem. Monitor tools inform users of their nodes’ performance. We will be based on the community nodes’ performance to decide when Slashing is enabled. That said, the Slashing release date is not 100% finalized.

3 Likes

Hi @Mike_Wagner, many thanks for your feedback.

The team will based on community nodes’ performance to decide the exact Slashing release date.

According to the new protocol, within 1 month, your nodes will be assigned into shard chains. Then we will collect their stats, performances, and how many blocks they sign in 1 epoch. I believe with this data, you can adjust your nodes’ configuration to avoid being slashed.
Of course, it’s our responsibility to help our users config their nodes and contribute to the chain.

It’s true, and this is what we are trying to solve. Slashing won’t 100% solve this problem. If a node is assigned in one shard chain and later becomes validators. But if it failed to sign enough blocks in that epoch, it’s will be slashed at the end of that epoch. In this case, users STILL RECEIVE block rewards in that epoch along with their stake back.
This Slashing is just light punitive approach. We assume the community is trying to act honestly, and they want their nodes to be helpful for the system. Force a bad nodes to unstake is a kind of notification: “Your nodes are not helpful, please fix your nodes”. Of course, the team must help the users to get their nodes perform well.

That said, the main outcome of slashing event is notify users their nodes need to fix something. The team will work with the community to figure out how to fix.

Thanks for your feedback again. An important work before slashing events is to help users fix the nodes problem.

4 Likes

PNode question will be handled by mr @khanhj for throughout care.

1 Like

I do think the “light punitive approach” is the best approach versus an economic slash used by other PoS protocols.

I don’t think anyone is disagreeing with the “why” of slashing. The community wants participating Nodes to be fully voting. The community doesn’t want non-voting Nodes or Nodes that validate bad blocks.

However what concerns me is that any insight into voting participation has only recently been made available. For over a year operators have been standing up Nodes believing they are working properly. It is only very recently that we are understanding that half of the Nodes aren’t voting properly – for whatever reason. I’d wager that Nodes with 0 votes are likely not doing so out of malicious intent. These are likely failing to vote due to poor configuration, stuck updates, improper monitoring or maybe improper resource allocation for VMs and containers.

However the experience will be significantly different between vNode and pNode operators. For vNode operators these are issues that can and should eventually be documented and resolved. pNode operators currently have little to no path for resolving any issue with the pNode.

For a vNode operator – unstaking the 1750 PRV stake returns that amount to the vNode operators wallet. It is a private stake they provided. While inconvenient and a signal that vNode has issues that need to be resolved, a resolution can be implemented quickly and the vNode restaked. This makes sense.

For a funded pNode operator – unstaking the 1750 PRV stake returns that amount to the Incognito team’s funding wallet. The stake is funded. With the official end to support for restoring staked funds, there is no way to restore the funded stake. For recent pNode operators it is almost certain they do not have the amount needed to privately stake a Node (otherwise they would have already done so). Even for someone operating a pNode since the genesis block, that pNode will not have generated enough PRV for a full stake.

Slashing – for funded pNodes – will effectively brick those pNodes. For a privately funded pNode, slashing would be a similar experience to that of a vNode operator. But those operating funded pNodes stand to be forcibly kicked out of the network for issues they can’t currently fix themselves and with no recourse to rejoin the network with the same funded stake.

7 Likes

First, I totally understand pNodes problem, we will think about the correct solution later. The team won’t let our users’ concern go silent.

We are still collecting nodes’ stats. We will analyze the data before conduct any action.

Nice feedback. Will this a good approach? I asked myself the same question as well. The point here, it’s hard to get an ultimate solution at first or impossible to do so. We try to deliver a good light solution at first. It’s easy for the user to adapt, for the team to develop, and for the system to deploy and maintain.
FYI, we are developing a more advance solution to make the network security. We will publish later when the designed is completed.

3 Likes

Can you elaborate on this? I’m not quite sure I follow. How is this different than the current setup for unstaking?

1 Like

Thanks @Mike_Wagner for your comments as a validator. Im in the exact same boat. The node monitor feature recently created has been great but how to fix pnode or vnode issues is not clear at all.

I understand the developers are thinly spread also dealing with support so I’d strongly emphasize that a separate post on how to address these solutions should be clearly communicated.

3 Likes

The new Unstake feature will allow nodes in Candidate role to unstake immediately.

3 Likes

Per

User Guide

It’s unclear when the Candidate role happens, could you clarify?

I believe Candidate role is when a node is staked & online but not in preparation or committee, aka waiting.

image

1 Like