Dynamic committee size and dynamic sharding: implementation phase

Objective:

  1. Improve the robustness of the chain: avoid the chain blocked by the unavailability of nodes, network disconnected, inadequate node power.
  2. Adapts well with the variety of the number of validators
  3. Strengthen the security: trustless to shard committee

Estimate End Date: June 2021

July 2020

  • Beacon sync, insert and store all shard blocks
    • Beacon node must sync and insert shard from beginning
  • Refactor Committee State in Blockchain package for later implementation
  • Technical Specification for New Staking Flow

August 2020

  • Implement new staking flow

    • Staking to become candidate
    • Candidate Life cycle (not included monitoring and report)
    • Unstake flow
    • Refine Stop auto re-staking flow
  • Implement new swap rule

    • Not included increasing/decrease (case 3,4) shard
  • Technical Specification of Slashing Misbehaved Validator

September 2020

  • Integrate new Staking flow and swapping rule with with Smart Contract
  • Implement Slashing Misbehaved Validator

October 2020

  • New Staking flow and swapping rule are ready for local test (TestNet in happy case)
  • Implement, Integrate and testing Slashing Misbehaved Validator with Smart Contract and other Feature

November 2020

  • Ship New Staking flow and swapping rule at the end of Nov or early Dec
  • Ship Slashing Misbehaved Validator at the end of Nov or early Dec

March 2021

  • Reduce committee size dynamically

June 2021

  • Dynamic grow/shrink number of Shards

Key results:

  • Committee size is dynamically changed with the number of validators
  • Dynamic sharding
  • Beacon is able to verify the correctness of every single shard. As a result, a shard could verify cross-shard transactions through the beacon chain.

Note:

20 Likes

This task will begin in 1st Jun:

  1. Design working flow @hy @hungngo @0xkumi @dungtran
  2. Implementation @hy @hungngo
  3. Unittest, local testing @hy @hungngo
  4. Integration testing by QC
6 Likes

Update 2020 Jun 8:

Github link: https://github.com/incognitochain/incognito-chain/tree/dev/dynamic-committee-size
Result:

  • Finish implement Beacon sync, insert, store all shard blocks

Next:

  • Testing
6 Likes

Update 2020 Jun 12:

Result:

  • In testing phase and fixing bug

Next week:

  • Testing
  • Move to next task while waiting for QC
4 Likes

On June 24th, 2020

  • Update the random shard assignment scheme to keep the balance of the shards’ validators.
  • Update the solution for problem 2,3

Detail design: https://docs.google.com/document/d/19KChrg0B2LmfT_t-K8vovCG-mXUkAYG88j5FQsKPXT4/edit#

6 Likes

Update 26 June 2020

Finish:

  • Agreement on refactoring code approach
  • Refactor Instruction related to committee (stake, assign, swap, stop-auto stake)

Next:

  • Refactor Committee State, remain the current assign and swap rule
  • Test after finish refactoring code

This week, we do not spend much time on the main task as the code based of incognito-chain is very huge now. After thoughtfully consideration, we decide it need to be refactor into smaller component, make it easier to maintain in the future. It might take some time now, but new staking flow can be well-implemented and well-tested with high quality code.

8 Likes

Update 3 July 2020

Finish:

  • QC testing on Beacon Sync as Fullnode
  • Implement Beacon Committee State

Next:

  • Begin implementing Shard Committee State
  • Begin implementing New Staking Flow
  • Finish implementing + unit-test Beacon Committee State
6 Likes

Yesterday, we talked about dynamic committee size - new staking flow, which supposes to be released on mainnet this September. This introduces new earing lifecycle of a validator, reduce time to sync shard’s data significantly, and also help user to unstake immediately.

image

Again, checkout the technical specs here

8 Likes

Update 10 July 2020

Finish:

  • Refactor:
    • Local testing for Beacon Committee State (transaction, stake, stop auto stake)
    • Instruction unit test
    • 80% implementation Shard Committee State
  • New staking flow, swap rule
    • Agreement for new swap rule
    • 60% design for new staking flow and swap rule

Next

  • Refactor
    • Finish Implementing Shard Committee State
    • Begin local test for both Shard and Beacon Committee State
  • New Staking flow and Swap rule
  • Finish first design for implementation
  • Begin implementation
4 Likes

Update 17 July 2020

Finish:

  • Refactor:
    • Implementation Shard Refactoring
    • Merge code from dev/master-consensus-optimized to dev/dynamic-committee-size
  • New Staking flow:
    • Implementation Spec

Next

  • Refactor
    • Shard Committee local testing and fix bug
  • Review New Staking Flow implementation spec
  • Implement new staking flow
6 Likes

Update 27 July 2020

Finish:

  • Refactor:
    • Implementation Both Beacon and Shard Refactoring. Ready for QC testing
  • New Staking Flow:
    • Low Level Technical Spec (sent out later in this week)
    • Finish 20% implementation

Next

  • Refactor:
    • Testing with QC team
  • New Staking Flow:
    • Continue implementing
1 Like

Update 3 Aug 2020

In progress:

  • Refactor:
    • Testing with QC team
  • New Staking flow: 40%
  • Unstake flow: 20% (estimated end date in this week)
2 Likes