The bootstrapping script pulls data comes from a running validator node and this is done once a week on Mondays. That is why you are showing 30K blocks behind.
We have it on the list of things to change and will most likely have the data refreshed every 2 days compared to the current weekly.
6/10/2022 Update - Files are now updated daily.
Jared, docker is not starting after bootstrap data is downloaded. What could be wrong?
it seems loop of restarting. Help!
Thanks
I would recommend creating hard links instead of copying the files, that way you could save some space and time.
# Stop containers.
docker container stop inc_mainnet_0
docker container stop inc_mainnet_1
# Set the variables accordingly.
fromNode=/home/incognito/node_data_0
toNode=/home/incognito/node_data_1
shardName=shard5
# Create the hard links.
rm -rf $toNode/mainnet/block/$shardName
cp -al $fromNode/mainnet/block/$shardName $toNode/mainnet/block/$shardName
# There are some files that must be copied, not linked.
cd $toNode/mainnet/block/$shardName
rm C* L* M*
cp $fromNode/mainnet/block/$shardName/C* .
cp $fromNode/mainnet/block/$shardName/L* .
cp $fromNode/mainnet/block/$shardName/M* .
# Make sure all files are editable to the incognito user.
chown incognito:incognito /home/incognito -R
# Start containers again.
docker container start inc_mainnet_0
docker container start inc_mainnet_1
Here is a node script I made to automate the creation of hard links. You could run it periodically without any problem to save a lot of space.
@Jared Question about permissions. I have noticed when I use the ./bootstrap.sh without sudo, all those blocks that I get come in owned by user:group incognito:incognito, but then after I restart inc_mainnet_0 container and it eventually gets the remaining blocks to get me up to latest, then those blocks are owned by root:root . I assume because we usually start in the installer script with sudo ./inc_node_installer.sh . My question is: some of my block *.ldb files are owned by incognito:incognito and some are owned by root:root, is this going to cause a problem at some point? Who (what user and group) should the proper owner of all *.ldb files be?
It should not have an impact on performance since the docker container is ran with root privileges so it can access from any user. I will update the post to put sudo before ./bootstrap.sh so all files are owned by the same.
Is the data source updated automatically? When I use the script and download the shard data I noticed some shards are very far behind and requires hours and hours of syncing. Shard 7 is 500k blocks behind for example.
The bootstrap files offered by Incognito are updated every week on Monday. In the future we may increase the interval that these files are updated.
Weird. If I go through all downloads 00,0-7 and then start the node I get the following heights:
beacon 1936451 (19.2G download)
shard0 1556202 (36.5G download)
shard1 1566596 (13.2G download)
shard2 1930751 (10.4G download)
shard3 1629497 (15.0G download)
shard4 1929529 (14.6G download)
shard5 1511193 (9.2G download)
shard6 1659406 (12.1G download)
shard7 1427293 (11.2G download)
Only 3 of those are up to date, the rest are way behind. (started on fresh node with empty folder, on a new machine, so it’s not my old data).
Is this for a fullnode?
http://51.222.82.132/bootstrap/
There are the amounts you should have after running bootstrap. Keep in mind those files are compressed.
Yeah, its for a fullnode. Looking at the bootstrap.sh output, that’s the same url it downloaded from and the sizes looks correct. It unpacked all of them without showing any errors.
@Jared @Support Any reason why this isn’t working?
“wget http://51.222.82.132/bootstrap/bootstrap.sh && sudo chmod +x bootstrap.sh && sudo bash bootstrap.sh”
It just hangs…
We migrated some servers last week. We will send you new link for bootstrap later.
May I, too, have the new URL?
The guide will be upgraded to reflect any changes made and we will comment here as well.
The original post was edited to update the bootstrap script to Version 1.5.
Version 1.5 - 6/9/2022 -
Keep watch for Version 2.0, as this version will support bootstrapping the new (currently beta) validator database modes.
Hey @Jared. What’s the ETA for 2.0? Getting very close to needing a new server with more storage space.