Shard 2 Stalling at 433077 on Multiple vNodes

I have no node syncing shard 0 but 2 and 6 are ok.

Yeah, I run multiple nodes on the same infura. I recently changed to a node.js script managing my validators, but I still have an old shell script that runs two nodes on the same machine. It looks a lot like your script, but yours is cleaner, because I didnā€™t think of looping through array keys instead of values. :blush:

  i=0
  for validator_key in "${validators[@]}"; do
    rpc_port=$(($first_rpc_port + i))
    node_port=$(($first_node_port + i))

    i=$((i+1))
    data_dir=${DATA_DIR}/node_$i
    echo "Starting inc_validator_$i container on $node_port (RPC $rpc_port)"

    set -x
    docker run --restart=always --net inc_net -p $node_port:$node_port -p $rpc_port:$rpc_port \
        -e NODE_PORT=$node_port -e RPC_PORT=$rpc_port -e BOOTNODE_IP=$bootnode \
        -e GETH_NAME=$geth_name -e GETH_PROTOCOL= -e GETH_PORT= -e FULLNODE= \
        -e MININGKEY=${validator_key} -e TESTNET=false -e LIMIT_FEE=1 \
        -v ${data_dir}:/data -d --name inc_validator_$i incognitochain/incognito-mainnet:${latest_tag}
    set +x
  done
}

Donā€™t forget to set the empty -e GETH_PROTOCOL= -e GETH_PORT= because it will append the default values if itā€™s not set at all and end up with http://https://mainnet.infura.io/v3/a1b2c3...:8545. Itā€™s quite an ugly piece of code with no checks. :face_with_hand_over_mouth:

https://we.incognito.org/t/solved-vnode-sync-errors-since-version-20210313-3/12390/11?u=fredlee

(WhatĀ“s your -itd for?)

I forked this from @mesquka, so Iā€™m not 100% but I think it might have been intended as -it -d? Per a quick search:

"

docker run -it -d --name container_name image_name bash

The above command will create a new container with the specified name from the specified docker image. The container name is optional.

  • The -i option means that it will be interactive mode (you can enter commands to it)
  • The -t option gives you a terminal (so that you can use it as if you used ssh to enter the container).
  • The -d option (daemon mode) keeps the container running in the background.
  • bash is the command it runs.
    "

Though think combining all three options into one flag isnā€™t an issue here.


I donā€™t have -e GETH_PROTOCOL= -e GETH_PORT= but will give that a try.


UPDATE: Still seeing if shard syncing will make it past the roadblock but with @fredleeā€™s change Iā€™m already seeing calls to infura, so Iā€™m hopeful.


UPDATE: On Shard0 Iā€™m past the roadblock by adding in the code suggested above. This is using the forked script I mentioned above and the image from 06/26. Later today Iā€™ll work on trying it with other nodes.

1 Like

Nah -itd is probably no issue at all. I just wondered what it was for. :relaxed:

:space_invader:
I found the problem of getting stuck in shard 63902 according to your script) If you are interested, knock on the PM)

Now Iā€™m stalling in different places:

  • shard0 - 63902
  • shard2 - 293816

But not every time

Updating myself here, do not run the latest version with -e GETH_PROTOCOL= and full url in GETH_NAME

https://we.incognito.org/t/shard-stalls-are-back-in-20210630-1/12803/6?u=fredlee

1 Like