Raspberry Pi 400 or Pi 4 Vnode Setup (for experienced linux users)

Disclaimer: Running an Incognito Vnode on a pi 4 or pi 400 will tax the quad-core cpu and shorten its lifespan. Given that there are better single board processors to use than a raspberry pi, this is not the most efficient way to run an incognito vnode. Nevertheless, this setup demonstrates the versatility of a raspberry pi and the incognito network. Go Incognito!

These Raspberry PI Vnode setup instructions are meant to be used in conjunction with Rocky’s important forum link for setting up Incognito nodes. Thank you @Rocky!

How to setup your own node…

For Raspberry Pi 400 or Pi 4 vnode setup, (a) extra preparation and (b) an extra edit to inc_node_installer.sh are added to Rocky’s setup steps listed in “METHOD 2: Non-Interactive Setup” listed at Rocky’s link above. This is for experienced linux and pi users. Building on Rocky’s instructions, let’s call this:

METHOD 3: NON-INTERACTIVE SETUP ONLY FOR PI 400 AND PI 4 !

Preparations (the hardest part)
  1. Don’t forget to open your chosen router ports for incognito RPC(defaults in script are 8334 and 9334) and open ssh port if used from outside LAN.

  2. On Windows computer, use PI Imager [https://www.raspberrypi.com/software/] and choose “Ubuntu Server 21.10 64 bit OS for arm64” (or “Ubuntu Desktop 21.10 64bit OS” which is more memory intensive) to create a 32gig boot-up sdcard for the raspberry pi.

  3. Boot up the Pi with Ubuntu Server sdcard and setup your ssd external drive connection with:

     df -h               (usb ssd drive if mounted usually shows up as "sda1", not always)
     sudo blkid           (copy UUID number of ssd device "sda1")
     sudo nano /etc/fstab  (add this line to fstab to automount ssd on reboot)
     UUID=add UUID# here            /media/username      ext4     defaults    0    0
    
Notes on fstab

Add the copied UUID# of ssd device “sda1” and change the mountpoint “/media/username” to your username and folder location. All incognito data will be stored here and you need to remember this folder name to add to inc_node_installer.sh script later.

  1. Set up infura and save/remember the infura url link/code to also add later to inc_node_installer.sh script. Instructions at Rocky’s “setup your own node” link listed above.

  2. UBUNTU 21.10 Server 64bit OS setup — After Ubuntu boots up and external ssd drive connection is tested (by shutting down and rebooting to see that fstab settings are correct), login/open terminal or ssh into terminal and enter the following commands:

     sudo apt update && sudo apt upgrade (this takes awhile, be patient.)
     sudo reboot
     sudo apt upgrade  (install all upgrades, especially "raspi-"modules.)
     sudo apt install curl wget  (install curl & wget which are needed to download scripts)	
     sudo apt install docker.io
     sudo apt-get install qemu binfmt-support qemu-user-static (Special Magic!! see note below)
    
Optional Test of qemu

Note: The last command above installs the “qemu” emulator and the binfmt “abstraction layers” that will automatically run linux/amd64 binaries through the “qemu-x86_64” emulator. Docker runs the incognito container/image made for linux/amd64 platform (i.e. not made for the raspberry pi ARM architecture) through qemu emulating a linux/amd64 platform. Below are links to the sources for you to research the magical qemu emulator further:
https://www.qemu.org/
https://wiki.debian.org/QemuUserEmulation
https://wiki.debian.org/RaspberryPi/qemu-user-static

Optional TEST of qemu:
sudo update-binfmts --display

In the output, look for these lines indicating the x86_64 (amd64) emulator is enabled:

    qemu-x86_64 (enabled):
    package = qemu-user-static
    type = magic
    offset = 0
    magic = \x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
    mask = \xff\xff\xff\xff\xff\xfe\xfe\xfc\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
    interpreter = /usr/libexec/qemu-binfmt/x86_64-binfmt-P
    detector = 
Optional Test of Docker
sudo docker version     (if issues, first try "sudo reboot" and try again)
sudo docker run hello-world   (better TEST of docker network connections) 
sudo docker system prune -a    (this removes hello-world test container and all other containers)

PREPARATION COMPLETE! YOU NOW HAVE A RASPBERRY PI THAT CAN MAGICALLY EMULATE OTHER PLATFORMS AND CPU ARCHITECTURES (EVEN AFTER REBOOTS).

Step 1 : Download Installer Script

This is the same as Rocky’s Step 1.

curl -O https://raw.githubusercontent.com/incognitochain/incognito-chain/production/bin/inc_node_installer.sh
Step 2: Edit Installer Script

This is the same as Rocky’s Step 2.

sudo nano inc_node_installer.sh

Change the appropriate script lines as already explained in Rocky’s node setup link above:

A. add your validator key(s)
B. make adjustments to port numbers if not using defaults
C. add the incognito home data directory location saved in Preparations: step 3.
D. add infura link saved in Preparations: step 4.

lines in script to edit:

VALIDATOR_K="validator key1,key2,key3"
GETH_NAME="https://mainnet.infura.io/v3/xxxyyy"    (see Rocky's setup url) 
PORT_RPC=""						    (remember to open ports on router)
PORT_NODE=""					    (remember to open ports on router)

INC_HOME="/media/username/$USER_NAME"    

Note: Setting INC_HOME folder is important. Add your ssd device “sda1” mountpoint location folder “/media/username” in example earlier in front of $USER_NAME variable. This folder might be different for you depending on how external ssd was setup. This sets incognito home folder which contains all the data.

Don’t save and exit nano text editor yet until completing next EXTRA step 3 for RASPBERRY PI 400 and 4 only.

Extra Step 3: For Raspberry Pi 400 and Pi 4 ONLY!

While still editing the inc_node_installer.sh script, you are going to add this parameter " --platform linux/amd64" to a specific line in the script file.

Use Nano text editor’s find/search function and find this line in the script:

docker run --restart=always --net inc_net \

[This was line number 309 in my inc_node_installer.sh script downloaded June 2022 and this line is located in the section titled: “Start the incognito mainnet docker container”. The line number may change as script improvements are developed, so use also the section title to confirm line location.]

CHANGE THE LINE TO THIS:

docker run --platform linux/amd64 --restart=always --net inc_net \

You are simply adding “–platform linux/amd64” immediately after the “docker run” command and before the other command parameters and switches.

SAVE the changes to inc_node_installer.sh and EXIT the text editor.

Note: Adding this extra switch/parameter “–platform linux/amd64” to that line above tells docker to run the “linux/amd64” incognito docker container/image
even though docker detects the raspberry pi “linux/arm64” host platform/architecture. The qemu emulator does the rest (i.e. emulates the amd64 platform).

Final Step 4: Make Script Executable and Execute it

This is the same as Rocky’s Step 3.

sudo chmod +x inc_node_installer.sh       
sudo ./inc_node_installer.sh -y    

After many minutes (it will take more than a blink of the eye for a pi), check to see if the amd64 docker incognito container is up and running on a PI400 or PI 4 arm64 host with:

sudo docker ps

If all doesn’t go well, check “journalctl | grep Inc” to see what errors or warnings show up. Try also, “sudo docker stop inc_mainnet_0” and then “sudo docker start inc_mainnet_0”. Many issues are fixable if you look and search online for similar error messages.

Bootstrapping

After you confirm inc_mainnet_0 is up and running in docker for 30 minutes or more,
one should learn how to bootstrap the beacon chain by downloading this script posted by Jared in the Incognito forum:

wget https://bootstrap.incognito.org/bootstrap.sh && sudo chmod +x bootstrap.sh && sudo bash bootstrap.sh

Note: you will need to remember your ssd device mountpoint “/media/username” (in the example above) to point the script to the incognito home data folder.

Instructions are at link below. Thank you @Jared for sharing the bootstrap.sh script. More info on it here:

https://we.incognito.org/t/node-operator-bootstrapping-introduction-guide/14239

My Raspberry Pi400 Observations

Running docker + the qemu emulator + incognito node will push raspberry pi’s cpu usage high while syncing and in committee, but cpu temp remains under 50 degrees Celsius for me at least. Active cooling of raspberry pi cpu should be used and a bare minimum of apps should be run on the pi while running incognito node. To improve pi 400 cooling, see this: https://www.instructables.com/Improving-the-Cooling-on-the-Raspberry-Pi-400/ I have used the cooling improvement above, but I do not have my pi 400 overclocked. I used Jared’s bootstrap.sh script to save beacon and shard syncing times.

Recorded data regarding my raspberry pi400 for reference:

Vote Stats % for last seven committee epochs with staked raspberry pi 400: (5975)98%, (5976)99%, (5977)98% ,(5978)99% ,(5979)98% ,(5980)79% ,(5981)100%

CPU Usage while “in committee”: lowest: 4.3%, highest 57.7%, average 21.7%
CPU Celsius Temps while “in committee”: 38.9, 38.5, 39.1, 39.4, 37

CPU Usage while “syncing shard”: lowest 38.4%, highest 96.6%, average 68.8%
CPU Celsius Temps while “syncing shard”: 43.8, 44.8, 45.3, 45.2

CPU Usage while “pending” with “latest” sync: lowest 2.8%, highest 24.9%, avg: 6%
CPU Celsius Temps while “pending” with “latest” sync: 37.0, 36.0, 38.0, 38.1, 38.5

All cpu temps and cpu percentages were random samplings taken at different times and the cpu usage is a percentage usage of all 4 cores as indicated by “top -i”(irix mode off = shift-i) and “htop”. This is intended as a rough estimate of pi 400 performance not a systematically computed benchmark.

CONCLUSION:

GIVEN THAT RUNNING AN INCOGNITO NODE THIS WAY WILL TAX THE RASPBERRY PI 400/4’S CPU WHEN IN COMMITTEE, THIS IS NOT THE BEST WAY TO RUN AN INCOGNITO VNODE. THERE ARE BETTER SINGLE BOARD PROCESSORS TO USE FOR A VNODE THAN A RASPBERRY PI.

NEVERTHELESS, THE PI COULD BE USED AS A TEMPORARY EMERGENCY BACKUP VNODE WHEN THINGS GO WRONG WITH ONE OF YOUR OTHER INCOGNITO VNODES.

Go Incognito!

10 Likes

Awesome write up. Very detailed and well outlined. Thank you for taking the time to put this together. :clap: :partying_face:

4 Likes

Omg what a great topic! I got a Pi 3 model, not sure if it has sufficient resource to run a node but definitely will follow your instruction to give it a try on the weekend.

Thank you!

5 Likes

What does that mean?

Cpu usage will run high at times during syncing and committee.

1 Like

Time for an update regarding running a vnode on a Raspberry Pi. Ever since the Incognito Network improvements, in particular the reduction of blocktime by half, the raspberry pi running a single vnode has been straining to keep up. Where I noted earlier that CPU usage was between a 33% and 45% depending on its role (from “pending” to “in committee”), now with the blocktime reduction (or faster blocks), rapsberry pi CPU usage “in committee” is between 88% and 97%. I think it safe to say that the faster and more efficient Incognito network gets, the less viable raspberry pi’s become, except for such experimental usage. My raspberry pi has been slashed a couple of times now particularly when other vnodes on my network are in committee at the same time. The pi cannot keep up with other quad-core cpu’s when competing for limited though available bandwidth.

The raspberry pi experiment will soon be at an end the next time it is slashed. On the bright side, the raspberry pi’s limitations running a vnode indicate the developers many improvements to the network. Congrats, devs, you made the network so efficient and speedy, my raspberry pi will very soon no longer be able to keep up.
Go Incognito!

2 Likes

I’d like to share my experience with doing this on two raspberry pi 4B models with 8GB of RAM: they’re working!

My first Raspberry Pi 4B has successfully finished several committee cycles, with the last one getting a vote count between 67% and 84%. I have a feeling that the 8GB of RAM is highly advised if not required. Unfortunately the raspberry pi 5 does not support ubuntu server and I’ve read they don’t plan on it any time soon. It also uses a different CPU and I wasn’t able to get qemu to run.

As for the process laid out by Socrates: thank you.

For anyone else looking to follow this, here are a few things that tripped me up.

1.) boot up the pi (I plug it in with an ethernet and ssh into it) and immediately go through the “df -h” steps to mount the UUID. Reboot. Then start updating.

EDIT: As I was setting up my second Pi node I had a lot of issues where bootstrapping files would not be recognized and the docker container would constantly restart. After looking at my first one that works I found that when mounting the UUID I didn’t update the /media/username. I’m not sure why, but when I did the file structure got a little messy. So for the the line:
UUID=add UUID# here /media/username ext4 defaults 0 0"

I only inserted my UUID code (with the quotes) and it worked.

2.) for the step to download the installer script it is now “sudo curl -O https://raw.githubusercontent.com/incognitochain/incognito-chain/production/bin/blink.sh

inc_node_installer.sh was replaced with blink.sh

While editing the blink.sh file I only add my validator key, infura link, the alteration as mentioned by Socrates (use ctl W to use the search and find) and I add my main user profile into INC_HOME. For example changing “/home/$USER_NAME” to “/home/my profile/$USER_NAME”

2.5) It doesn’t hurt to head into the incognito directory after running blink.sh to run “run_node.sh” (sudo ./run_node.sh) - I’ve noticed that doesn’t happen automatically sometimes.

3.) Remember when using the bootstrap.sh file to do the same thing on startup. It will prompt to the directory /home/incognito. Insert your user name so it is /home/your user name/incognito.

4.) If you get to a specific point where something simply will not work - it’s probably best to just reformat the SD card and restart. After doing the process several times I’m able to get things done no problem - the issues I’ve ran into were clearly due to an error I made in a step.

EDIT: I still mess things up. This is a process where a very small detail can cause processes to not work so be patient and know that once it’s done you’re golden.

Some weird issues I had included qemu not installing, the bootstrap script not working, issues with files not recognizing where they were… you’ll get the hang of it. Again, big shout out to @Socrates for figuring all this out to begin with.

3 Likes

Interesting, I’ve been looking into something similar to this for a while now.

What are you using for node data storage?

My first raspberry pi has a 500 gb micro SD card - for the latest one I’m going with a 256GB. I’d recommend getting the fastest SD card possible to make bootstrapping a little less painful (I’ll update this if I notice an increased vote %). I got this for the one that is bootstrapping as I type this at a few MB/s faster than the other one.

How is the wear on the drive? SD cards are not ideal for constant read and write. If you upgrade to an external SSD you’ll most likely see your vote count increase.

I just used fio on my first pi4B (it’s been running for months) and it estimates around 96% capability. I’ll also update this if I run into issues down the road. I imagine a good micro sd card makes a huge difference but I’ll have to look into external SSDs as well.

3 Likes

As a quick update, I have seen a marked increase in vote percentages using a SanDisk Extreme(A2) instead of a SanDisk Ultra(A1) - (the extreme has a 190 MB/s read/write speed vs. the ultra 140 MB/s read/write speed).

With the A1 I saw 67% - 84% vote count % and with the A2 I am seeing 71% - 88% vote count %.

Getting the more expensive A2 version of the disk does have some real benefits if slashing is a concern. It also bootstraps a bit faster.

1 Like