Shard 2 Stalling at 433077 on Multiple vNodes

[abduraman]

I have no node syncing shard 0 but 2 and 6 are ok.

[fredlee]

Yeah, I run multiple nodes on the same infura. I recently changed to a node.js script managing my validators, but I still have an old shell script that runs two nodes on the same machine. It looks a lot like your script, but yours is cleaner, because I didn’t think of looping through array keys instead of values.

  i=0
  for validator_key in "${validators[@]}"; do
    rpc_port=$(($first_rpc_port + i))
    node_port=$(($first_node_port + i))

    i=$((i+1))
    data_dir=${DATA_DIR}/node_$i
    echo "Starting inc_validator_$i container on $node_port (RPC $rpc_port)"

    set -x
    docker run --restart=always --net inc_net -p $node_port:$node_port -p $rpc_port:$rpc_port \
        -e NODE_PORT=$node_port -e RPC_PORT=$rpc_port -e BOOTNODE_IP=$bootnode \
        -e GETH_NAME=$geth_name -e GETH_PROTOCOL= -e GETH_PORT= -e FULLNODE= \
        -e MININGKEY=${validator_key} -e TESTNET=false -e LIMIT_FEE=1 \
        -v ${data_dir}:/data -d --name inc_validator_$i incognitochain/incognito-mainnet:${latest_tag}
    set +x
  done
}

Don’t forget to set the empty -e GETH_PROTOCOL= -e GETH_PORT= because it will append the default values if it’s not set at all and end up with http://https://mainnet.infura.io/v3/a1b2c3...:8545. It’s quite an ugly piece of code with no checks.

https://we.incognito.org/t/solved-vnode-sync-errors-since-version-20210313-3/12390/11?u=fredlee

(What´s your -itd for?)

[Josh_Hamon]

@fredlee

(What´s your -itd for?)

I forked this from @mesquka, so I’m not 100% but I think it might have been intended as -it -d? Per a quick search:

"

docker run -it -d --name container_name image_name bash

The above command will create a new container with the specified name from the specified docker image. The container name is optional.

• The -i option means that it will be interactive mode (you can enter commands to it)
• The -t option gives you a terminal (so that you can use it as if you used ssh to enter the container).
• The -d option (daemon mode) keeps the container running in the background.
• bash is the command it runs.
  "

Though think combining all three options into one flag isn’t an issue here.

---

I don’t have -e GETH_PROTOCOL= -e GETH_PORT= but will give that a try.

---

UPDATE: Still seeing if shard syncing will make it past the roadblock but with @fredlee’s change I’m already seeing calls to infura, so I’m hopeful.

---

UPDATE: On Shard0 I’m past the roadblock by adding in the code suggested above. This is using the forked script I mentioned above and the image from 06/26. Later today I’ll work on trying it with other nodes.

[fredlee]

Nah -itd is probably no issue at all. I just wondered what it was for.

[zes333]

I found the problem of getting stuck in shard 63902 according to your script) If you are interested, knock on the PM)

[Josh_Hamon]

Now I’m stalling in different places:

• shard0 - 63902
• shard2 - 293816

But not every time

[fredlee]

Updating myself here, do not run the latest version with -e GETH_PROTOCOL= and full url in GETH_NAME

https://we.incognito.org/t/shard-stalls-are-back-in-20210630-1/12803/6?u=fredlee