Hello @Jared,
My vNodes are being slashed when they get to committee. I have been bootstrapping each node after activating. This is becoming more often now.
Any known issue or update I should be working on? I am using ver 22.10 of “Oct-2022-Incg-ScriptNodes” script. Any help is appreciated.
Who is your hosting provider?
Contabo VPS XL
Contabo throttles the Disk I/O and I can no longer recommend them. You can call and complain (it’s a Germany number) but the problem will resurface later on.
How many nodes do you have on that server?
I have 10 nodes in a server. I am observing this problem in many servers. I raised a ticket, but did not get convincing help.
Could you suggest any other hosting provider? Any specific configuration?
They will just give you the generic “this is a shared system so speeds are not guaranteed”. It’s a tactic to attempt to get you to pay for VDS or Dedicated Servers.
You can try going on the dashboard and naming the server to something like Amy’s Server or Jessica’s Server, sometimes you get someone on the phone who will see that and remove the throttle.
I just purchased a beefy server and will see if this works out. This method involves a hefty upfront cost and possibly recurring monthly costs.
How to check status of “throttles the Disk I/O”? and what should I ask Contabo to do during this situation?
First, stop all docker containers with:
sudo docker container stop $(sudo docker container ls -q --filter name=inc_mainnet_*)
Next run the following which will check the disk speed:
dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync
The best bet is to call during their open hours and tell them you’re losing money because of the poor I/O impact. Keep in mind that depending on your phone carrier the cost could be high, if that is the case use Google Voice or another alternative as it will be a lot cheaper.
This will be a temporary fix, they will end up throttling it again later on.
root@vmi1038965:/# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 43.3814 s, 24.8 MB/s
Here is the output of the script ran in the troubled server.
What data shows the I/O impact? and what numbers represent bad I/O?
That’s the metric you want to refer to.
SSDs are recommended for Incognito nodes.
From Google,
“A standard HDD will read and write at typically 80MB/s to 160MB/s, but an SSD reads and writes at between **200MB/s to 550MB/s”
I purchased many servers with the below configurations, especially with 1.6TB SSD. I am paying $40.49 per month for each server. But I am facing this issue with all servers. It looks like my servers are not configured to be SSD based.
I had 4 XL VPS from Contabo. They will tell you that they are on a shared system and that is why there is poor disk I/O.
