sorry I was on mobile. rm not RM.
Yeah I have quite a few VPS with them. They are pushing me to drop several grand and self host a server locally.
All my vnodes and pnodes went down an hour or so ago. The vnodes cycled back fairly quickly. The 3 pnodes that were earning got slashed. They all came back online when the votes hit 48, 46, 45. Fully synced, showing latest, etc. No internet issues whatsoever. Was there an update pushed?
No updates from Incognito. Do you have automatic updates turned on for docker?
Yes it is.
Just wanted to say that it seems that many Pnode and Vnode owners seemed to have had this issue and still do…well thank you @Jared for helping with the issue and hopefully, everyone will get their nodes back in order and with some time and research the dev team will figure it out…for now all of us just need to keep a mindful eye on our nodes and when a slash occurs create a trouble ticket so that it can be addressed/restaked…
Just got slashed last night. was there a code update? My nodes are all struggling to get back to “latest” status. Hate when this happens. I regularly check the node monitor, but unless I happen to be looking during the exact window that one of my nodes in committee is having an issue, there is no way to know I need to triage the situation until I get a notification from NITO that a node was slashed.
Please follow these steps to pull and send us the logs:
@Jared my pNodes are fine, this was a vNode
Adding to that…I have also found that node monitor can lag a bit with the status too. E.g. Node Monitor says your vnode is offline for 30 - 45 minutes when in reality its running and syncing data.
I’ve definitely noticed much more slashing in the past couple of weeks than before. Was much more stable a few weeks ago.
Monitor is showing updated vote stats continuing to drop (specifically on my now pNode that is also showing offline). I unplugged and plugged it back in (had my spouse do it). I am not home and can’t further look in to it, but i can see its been offline for an hour, and vote stats dropping well below 50 threshold. This is super frustrating.
pNodes should never he unplugged unless properly first shut down via the SSH command
sudo shutdown -h now . Pulling the plug randomly without a safe shutdown is almost always guaranteed to cause a stalled/corrupted beacon/shard.
Please submit the logs to us in support ticket so we can review them. Also include your host provider, server type and how many nodes run on your server.
Another pNode just got slashed, plus it (and 1 other) shows Not Latest Version, but my 3rd pNode is online and looks fine.
I just checked all 3 nodes on node monitor 2 days ago and everything was online and fine. Is there something I need to do or just wait like before for a fix to correct them?
If you need us to take a look at them please open a support ticket or PM me.