Slashed nodes

[VINWOSNI]

@Jared,
Ok, thank you. Is there a timeframe of when I should try again or any other steps I can take to reverse the slash?

[Jared]

We currently do not have a timeframe. I would just let it resync for now and I’ll let you know when the bootstrap is ready.

[VINWOSNI]

@Jared,
The pnode shows offline now, i’m assuming because of the deletion process at the start of the bootstrapping but not able to copy the files back? honestly, i don’t know.
Am we able to chat on telegram to get it online and syncing? I don’t know how to make it start up again.
(when i closed out the bootstrapping, i used command 88 as instructed, but as i noted before, the process didn’t finish, including the shard part. should i run it again for the shard?)

[Jared]

For now run the following:

sudo RM -rf /home/nuc/aos/inco-data

[VINWOSNI]

I’m doing something wrong, I’m getting “command not found” when I try to run it.
Am i trying to run the command from the wrong prompt?

[pickle]

@JG20, this keeps happening to me too. about the same time as your first post in this thread. contabo as well. i wrote to them about this sudden dip in performance, and they told me to get a VDS

i’ve reduced the number of nodes active on my VPS XL at any one time to just 4, and there are still issues. tearing my hair out!

[Jared]

sorry I was on mobile. rm not RM.

[Jared]

Yeah I have quite a few VPS with them. They are pushing me to drop several grand and self host a server locally.

[JG20]

All my vnodes and pnodes went down an hour or so ago. The vnodes cycled back fairly quickly. The 3 pnodes that were earning got slashed. They all came back online when the votes hit 48, 46, 45. Fully synced, showing latest, etc. No internet issues whatsoever. Was there an update pushed?

[Jared]

No updates from Incognito. Do you have automatic updates turned on for docker?

[JG20]

Yes it is.

[Tempestblack]

Just wanted to say that it seems that many Pnode and Vnode owners seemed to have had this issue and still do…well thank you @Jared for helping with the issue and hopefully, everyone will get their nodes back in order and with some time and research the dev team will figure it out…for now all of us just need to keep a mindful eye on our nodes and when a slash occurs create a trouble ticket so that it can be addressed/restaked…

[brico84]

Just got slashed last night. was there a code update? My nodes are all struggling to get back to “latest” status. Hate when this happens. I regularly check the node monitor, but unless I happen to be looking during the exact window that one of my nodes in committee is having an issue, there is no way to know I need to triage the situation until I get a notification from NITO that a node was slashed.

Very frustrating.

[Jared]

Please follow these steps to pull and send us the logs:

@Jared

Step 3 - Pull Logs & Send to Support

If in the event none of the above steps fix your pNode you will need to pull the logs and send them to support. Please follow the following steps in order to do this.

1. Click the Browser button at the top of the dashboard.
2. Navigate to the most recent log files ().
3. Click the Download button to save them to your computer.
4. Upload to Google, Dropbox, or another cloud provider.
5. Send a message to @support with a link to said file. Please also include what issue(s) you were experiencing.

[brico84]

@Jared my pNodes are fine, this was a vNode

[SPAddict25]

Adding to that…I have also found that node monitor can lag a bit with the status too. E.g. Node Monitor says your vnode is offline for 30 - 45 minutes when in reality its running and syncing data.

I’ve definitely noticed much more slashing in the past couple of weeks than before. Was much more stable a few weeks ago.

[brico84]

Monitor is showing updated vote stats continuing to drop (specifically on my now pNode that is also showing offline). I unplugged and plugged it back in (had my spouse do it). I am not home and can’t further look in to it, but i can see its been offline for an hour, and vote stats dropping well below 50 threshold. This is super frustrating.

[Jared]

@brico84

I unplugged and plugged it back in (had my spouse do it).

pNodes should never he unplugged unless properly first shut down via the SSH command sudo shutdown -h now . Pulling the plug randomly without a safe shutdown is almost always guaranteed to cause a stalled/corrupted beacon/shard.

@brico84

@Jared my pNodes are fine, this was a vNode

Please submit the logs to us in support ticket so we can review them. Also include your host provider, server type and how many nodes run on your server.

[VINWOSNI]

Another pNode just got slashed, plus it (and 1 other) shows Not Latest Version, but my 3rd pNode is online and looks fine.
I just checked all 3 nodes on node monitor 2 days ago and everything was online and fine. Is there something I need to do or just wait like before for a fix to correct them?

[Jared]

Hello @VINWOSNI,

If you need us to take a look at them please open a support ticket or PM me.