Hi,
It looks like my pNode was slashed, and I’m unsure why:
I followed some of the troubleshooting steps (found at Official pNode Troubleshooting Guide), including restarting the Docker, restarting my pNode, and updating the firmware.
Please let me know what more I should do, and how to get my pNode staked again!
Cheers,
Andy
It appears your pNode is showing as stalling. If it is still stalling follow this guide to bootstrap it:
After successfully bootstrapping confirm that the sync status is Latest and then send me your QR code (found on the bottom of your pNode) so I can add it to the re-stake queue.
Hi Jared. Thanks for the guide. Bootstrapped the Beacon Data and now it looks like the sync status is back to Latest:
Will send you a PM with my QR code!
Thanks for the help!
All of my pNodes just went offline (1 at first, now the rest), all have power and internet. one is in committee… wth is going on here?
Anyone else seeing something like this at the moment?
Same just happened to me, again. Have checked the node monitor multiple times today and everything looked fine. Just checked NITO and saw some pnodes were slashed. Two were earning, got slashed, votes stopped at 47 & 45 (last week the votes were at 48, 46, 45). Everything is online and latest when I open the node monitor. This has happened multiple times over the last few weeks.
Too much coincidence for these to be isolated incidents. I’m spending more time with these nodes than with my own kids.
Ouch 
Please PM me so we can do a remote session and I can review the logs.
Agreed, something weird is going on… no way we are all having this problem at the same time.
Same thing happened to mine as it was going into committee. Everything was in good order.
Just had this happen to my pnodes again…all the pnodes went down, one was earning, slashed at 46 votes, all immediately back online. Posting this just to document while we troubleshoot.
@Jared, will ping you later for a remote session.
Thank you everyone for the reports and submitting your node logs.
We are actively looking into what could be causing this issue. It appears to be related to the docker container auto update feature.
Myself, or another member of the core team, will reply to this thread when we have more information.
If your pNode with funded stake by Incognito was slashed please send your QR code to @Support. Also include your node’s most recent log file according to this guide (step 3): Official pNode Troubleshooting Guide
@AllegedlyAndy, @JG20, @fonz, @brico84 and others;
Update:
It appears Docker Hub is having issues and causing node docker containers to think there is a new docker container version when there is not. The devs are working on fixing this issue and making sure it doesn’t happen again but in the meantime, we have a workaround.
The following command works for vNodes, please connect to your node via SSH and issue the following command string:
> wget https://raw.githubusercontent.com/incognitochain/incognito-chain/blink_patch/bin/blink_patch1.sh && sudo bash blink_patch1.sh
We are working on a fix for pNodes and will post when we have one.
We now have a fix for pNodes. Please connect to your pNode via SSH and run the following command:
> wget https://raw.githubusercontent.com/incognitochain/incognito-chain/blink_patch/bin/pnode_updater_patch1.sh && sudo bash pnode_updater_patch1.sh
If anyone has any questions, comments, etc feel free to post them below.
Can you inform me on how to get my pnodes connected to this command… Thank you
Via Windows OS, click the Windows button > type PowerShell and click on it > type ssh nuc@pNode_IP_Address (change pNode_IP_Address to the actual IP address). When the password field comes up type “Binh!2345” without the quotes.
Now copy and paste the 1-line command string and hit enter. That is all, you may now close out of the window.
If on Mac, navigate to your Utilities folder via Finder and click on Terminal.
Thanks for the fix. Got it updated.
The pNode fix reverts it’s self. I’m not sure what the time frame is but it does.
I believe I’ve fixed your start.sh script. Should I just create a branch and merge request on github?
Yes, please @fiend138 you can create PR then the team will review.
May be the script in aos/ability/incognito/main.py pull a new start.sh file but i dont know in what condition it triggered.
I’ll review the main.py file and see how it’s generating the start.sh file. I think it’s using the start_incognito.sh file as a template, but I haven’t looked into it very deeply yet. I don’t suppose there’s a PRV reward for submitting fixes/contributions?
Please review the following pull request.
I’m not sure if there’s a better repository for this, but it was the only one I could find when searching for “pNode”.
Thanks a lot for your kindness in helping us fix the issue. But the since the pNode was no longer in production, the repo above as obsoleted. Even if your code is merged, there’s no way for the pNode to apply the fix automatically.
On other hand, it basically has the same result as running the pnode_updater_patch1.sh which @Jared mention above. So the only thing we should do now is to use that patch.
