Raspberry PI 101 (how to feel less like an idiot)

Goals of Rpi
1. My hardware setup
2. Installation
3. useful tools and links
- 3.1. on using docker

Goals of Rpi

I had a lot of things going on on my iMac workstation that I wanted to move away to the Pi. The reasons behind this:

Leaving it on costs more power (for sake of mathematics, lets say 100W on average for the iMac, and 10W for the Pi)
I’m doing a lot of development and ‘fun’ on the iMac, which smakes it prone to crash and/or having to reinstall. Having more people in the house suddenly not being able to turn lights on or off, I found out that is a terrible experience.
It needs to (keep) work(ing). Think about time machine backups, streaming content from an external drive, that shouldn’t be bothering your other tasks. Even though a mac is unix underneath, a headless machine has proven to be more stable.
(shameful but) I haven’t used docker before!
Running this thing headless (allows it to hide in a cabinet or a spot thats not visible) is a nice challenge that I regretted a few times, when dealing with partitioning for instance.
Being independant from sonos, Apple or others to be able to stream audio throughout the house (and reducing cost) – I’m in love with Roon, but Roon core needs a heavier machine than a Pi. There must be something that can replace that experience around!

How could I forget how terrible I am at these things when I started…

I started over at least 5 times (flashing the image, rebooting the system, forgetting the WLAN config etc.) because I’m the stubborn one that wants to run this thing headless only…

I’ve learned, I’m sharing. There are a million websites and documents that share this process, but I’d like to do this with you, hoping it got you thinking in the right direction. If yes, you’re welcome to leave a comment!

1. My hardware setup

my Apple Pi 🥧 accessed from the iPad

Rasp pi 4, 2 GB (wasn’t any other in stock, happy camper, with ~~8-9~~15 containers running it sits quiet at about 50% memory )
64 GB SD card (pretty standard sandisk thingy, didn’t bother to run it on the external drive – performance is quite similar unless you run an SSD). I’ve ordered a USB to NVM2 converter to see the performance. That should be quite dramatic.
10 TB seagate drive as external storage
Cabled to hook up to the AppleTV for homebridge, but they’re switched to a Google wifi router for access to the rest of the house.
Mac OSX 11 for flashing using Etcher and SSH using iTerm2/ohmyZSH.
Audioquest dragonfly red for streaming audio (works fine in raspbian, no drivers needed)

2. Installation

2.1. Flashing your SD card with raspbian buster lite

Picked lite because I have 2 GB of memory to work with, but makes this thing harder to do – we like the challenge don’t we!

Download buster lite (or desktop if you prefer) from the standard place
Download Etcher
Flash your drive using Etcher
Eject the disk
Wonder why the bloody thing isn’t doing anything.

2.1.1. Setup SSH and wifi

Ah yes! Don’t forgot this!

Create a new empty file called SSH. (no extensions or anything, plain ‘SSH’. No content, this allows SSH access). copy it to the SD.
Create a new file called wpa_supplicant.conf on the root of the SSD.

Contents of wpa_supplicant.conf:

country=de
update_config=1
ctrl_interface=/var/run/wpa_supplicant

network={
 ssid="<your SSID>"
 psk="<Password for your WiFi>"
}

Change your country, SSD and PSK fitting your setup
Copy both files to the root of your SD card (called BOOT now since flashing)
Eject the drive
Put it in the pi
Cross your fingers again.
Hope it works! (at this time I connected a display to be able to see what was happening, useful!)

Note: don’t panic if you lose the SSH/wpa_supplicant files once your Pi has booted, that is expected behavior I found out. (by trial and error)

Go do the raspi-config thing now! sudo raspi-config

Change at least:

Your password
Your timezone
Hostname
Your password (not kidding!)

And don’t forget to update your Pi regularly. Debian has an active community, your Pi uses a package system called “apt-get” that helps you keep your system updated.

That means that as soon as you’ve booted your pi again do the following:

sudo apt-get update
sudo apt-get upgrade

This will take time. Be happy! (and I got used to the wait, since I screwed up a lot and had to begin from start again)

2.1.2. Allowing login through SSH (without using a password)

I hate entering passwords, especially if my device already asks for one. I enjoy the ease of 1password, and in case of ‘security’ of my Pies, SSH works well. It takes time to get your head around public/private keys and what they do, but you’ll manage.

For the purpose of logging into your Pi using SSH; do follow what Rpi already has written down on their site.

In short;

Generate a key on your device (where you want to log in from); ssh-keygen (in my case my mac)
Copy the generated key to your RPi – `ssh-copy-id @’ (e.g. pi@192.168.1.1)
try and log in, ssh pi@192.168.1.1 – you might need to ‘trust’ it once, but then you’re done!

Optional but useful
4. Add your passphrase to the mac keychain, saves you typing a PW each time; ssh-add -K ~/.ssh/id_rsa

2.1.2.1. Troubleshooting the Mac and SSH

Make sure you setup the rights correctly on your pi’s ~/.ssh folder; (I changed the rights on my home folder because of reasons, but with that killed an easy login)

chmod 700 /home/your_user
chmod 700 /home/your_user/.ssh
chmod 600 /home/your_user/.ssh/authorized_keys

Apparantly the SSH Agent isn’t always running correctly on the Mac. Make sure you add the following to your ~/.ssh/config file;

Host *
    UseKeychain yes
    AddKeysToAgent  yes

2.2. Mmanaging USB drives

I formatted my 10 TB drive into a few pieces;

a 8 TiB partition for an Apple time machine
a 1 TiB partition for docker images / containers
The rest for SMB storage among other things.

All are EXT4 for now, the following command worked to format the drives:

sudo mkfs.ext4 /dev/sda1

Where SDA1 is the specific drive to format.

I’m skipping a few details here, since my scenario is specific. But make sure to read the following, since that gets it all together:

How to format and mount a drive

I’ve considered changing the size a few times afterwards (my partner had a bigger need than I expected for disk space) – I decided to connect my drive to my mac, boot a virtual linux setup with a gui and use Gparted to do so.

There is a commandline alternative where it’s based on called parted, and in combination with fdisk can do destructive stuff. I didn’t touch that after one attempt.

2.3. OMV5 (openmediavault) on debian – (FAILED!)

We all fail. And this thing failed on my end. I thought I found a ‘one way ticket’ into a good solution (OMV offers it’s own docker management for instance), but this worked me into more problems than I thought. I felt this whole thing was much more ‘bloatware’ than what my purpose was here (lean and mean Pi machine!)

OMV allows you to work the other way around and that it will be your new “OS” to work with; it allows you to:

Manage your filesystem
Manage your drives/mounts
Setup docker/docker images inside OMV, you have it all done inside a GUI (using Portainer if I’m not mistaken)

Status 25/3/20:
It seems things are better now, but as said – YMMV…

Beta OMV5 installation instructions here

2.4. AFP (netatalk / apple file protocol) on Pi (without docker) – (WORKS!)

This was my first attempt on getting AFP (apple’s file protocol) going on the Pi. I had to do a full clean install to get rid of OMV5 here ^^ but hell, it worked!

AFP is a very interesting powerful protocol that’s much faster in comparison to SMB, but eventually I let go of AFP – the docker images are not that great, I couldn’t get them going the way I wanted (and AFP outside docker was pretty heavy on resources for some reason)

Read up if you want to go this route though (and sponsor me a 8 GB version too!)

2.5. SMB (Samba) on Pi (without docker) – (WORKS!!)

This worked pretty quickly and easily! Read below!

The easiest one to get going

note: Check the comments for buster commands

2.6. Docker for Raspbian – (WORKS!)

Here the fun started. I started using docker on the Pi. My previous assumptions (command line virtual machines like vagrant / virtualbox) have always been a tough thing to get get my head around, and impossible to get rid of – I was hesitant to start.

But now I’m a happy docker camper; it runs smoothly – even though some containers are heavier on resources than others. Be aware of that!

My docker containers running smoothly together on the pi

Install docker

apt-get update
apt-get upgrade
curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh
sudo apt-get -y install docker-compose

Make sure your user can execute docker;
```
sudo usermod -aG docker pi
```
and see if it works now!
```
docker run hello-world
```

Good! Ready to get docker-compose going (trust me on this one, it makes your typing life much easier)

sudo apt-get install libffi-dev
sudo apt-get install -y python python-pip
sudo pip install docker-compose

2.7. optional, moving the storage of your docker containers to external storage

Moving the storage for docker files around wasn’t that hard – I followed the following by changing the daemon settings for Docker:
```
sudo nano /etc/docker/daemon.json
```
Adding this content to that file (make sure this is the volume you want your containers on)
```
{
    "graph": "/mnt/docker"
}
```
Reboot your Pi. (there are other ways, but I like sudo reboot, the power of the terminal!)
Check if docker is running from the right drive;
```
docker info|grep "Docker Root Dir"
```
It should tell you where it is storing it’s stuff now.
Then you can safely remove the original docker setup (if any) on your SD card;
```
rm -rf /var/lib/docker
```

2.8. Samba, using docker (WORKS!)

Super simple and quick to setup – It’s far from secure, but that isn’t the purpose here (doing SMB on a Pi in a corporate environment is a bad idea, the performance isn’t that great, but I’m at home)

Rpi Samba

Docker command to run (still need to wrap into docker-compose):

docker run -d --restart=always \
-p 192.168.86.224:445:445 \
-v /home/pi/smb/conf:/etc/samba \
-v /home/pi:/share/pi_home \
-v /mnt/share:/share/share \
--name smb trnape/rpi-samba \
-u "maxmustermann:password" \
-s "Share:/share/share:rw:maxmustermann"

Rename your stuff accordingly, this is just an example.

2.9. AFP (time machine share), using docker (WORKS!)

This one worked to get a TM volume going, but didn’t do that much else. YMMV!

Simple docker command to get a machine started

docker-compose yaml:

version: "3"
services:
  timemachine:
    image: timemachine-rpi
    ports:
      - "548:548"
      - "636:636"
    deploy:
      restart_policy:
        condition: any
        max_attempts: 2
    volumes:
      - /mnt/timemachine:/timemachine

2.10. Homebridge on docker with docker-compose (WORKS!)

By the time I was ready to move over my homebridge to the Pi I hit the 10 new installs mark for the Pi. I could dream the contents of the files, and started writing this .MD doc to document my progress.

It was time to put this thing to work! Let’s go with homebridge!

I found the ‘magic’ of docker-compose, where I was able to get to know YAML and it’s ‘spacious’ peculiarities.

mkdir /home/pi/homebridge
cd /home/pi/homebridge
sudo nano docker-compose.yml

Here’s the contents of my docker-compose file;

version: '2'
services:
  homebridge:
    image: oznu/homebridge:latest
    restart: always
    network_mode: host
    volumes:
      - ./config:/homebridge
    environment:
      - PGID=1000
      - PUID=1000
      - HOMEBRIDGE_CONFIG_UI=1
      - HOMEBRIDGE_CONFIG_UI_PORT=8080
      - TZ=Europe/Berlin
      - HOMEBRIDGE_DEBUG=0

Save (CTRL+O) and exit (CTRL+X, both of these will be your friends for life) the file, type docker-compose up and once more cross your fingers!

If you’ve installed docker, this should work now! Once the machine is running, it should be approachable on ‘http://yourhostname:8080’.

Very useful:

github article by oznu

2.11. Sonoshttpapi on docker with docker-compose (WORKS!)

I’m using docker-compose to get this thing going, feel free to grab the script (or fork me on Github)

make sure you setup your filestructure first, make a new directory where you’d like to setup all this.

mkdir clips
mkdir settings
mkdir cache
mkdir presets
curl https://raw.githubusercontent.com/jishi/node-sonos-http-api/master/presets/example.json > presets/example.json

now open your ‘docker-compose.yaml’ file

sudo touch docker-compose.yaml
sudo nano docker-compose.yaml

and add to it these contents;

version: '2'
services:
  sonoshttpapi:
    image: jonmaddox/rpi-docker-node-sonos-http-api
    restart: always
    network_mode: host
    volumes:
      - ./settings:/app/settings
      - ./clips:/app/clips
      - ./cache:/app/cache
      - ./presets:/app/presets

And one more container is done! Get her going again with docker-compose up

Very useful:

github repo by @maddox (thanks Man, you got this thing going for me)
THE sonos http api

2.12. Music streaming server with (Docker) DAAPD

Things like DAAPD are an amazing example of how powerful, streamlined and slick open source projects have become over the years.

I’ve considered multiple options to stream audio that I won’t go in to detail here, but DAAPD has been thus far the one that is free, easy to setup, stable and with great performance.

My docker-compose file for you:

version: "2.1"
services:
  daapd:
    #image: ghcr.io/linuxserver/daapd
    image: keesromkes/daapd:latest
    container_name: daapd
    network_mode: host
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./config:/config
      - ./music:/music
      - /mnt/share/daapd_music:/mnt/share/daapd_music
      - ./backuplogs:/var/log
    restart: unless-stopped
    ports:
      - "3688:3688"

I know, it’s an old docker-compose version. Haven’t had the time to update yet.

DAAPD running on my applepie

Mind you, I’ve built my own DAAPD docker file since I wanted to get rid of a few additional features, and reduce the footprint. You can always find a good one on docker hub, as commented in the docker-compose above.

A nice little video from Darko.audio on the spoti-pi:

https://youtube.com/watch?v=-CfnXOlYiz

//todo: pihole, deconz, nodered (2x), nrgok, portainer, nginx, mqtt, raspotify, mopidy

3. useful tools and links

Rcconf –> to check what services are running on your pi;
```
sudo apt-get install rcconf
```
Byobu –> Great bit of terminal kit to be able to keep on running tasks after closing your SSH session and manage multiple windows, sessions and splits.

Byobu showing it’s splitscreen prowess

Set your timezone, hostname etc. –> sudo rasp-config mostly, but I came to this because of this article.
Great read through on making your PI a bit tougher

3.1. on using docker

3.1.1. Running a docker container ‘detached’

A luxurious name to be able to run your container without ‘seeing it’; normally docker-compose would show you the output of the container until you ctrl+c out of it, and then stop it. Detaching means it will run without you seeing it.
```
docker-compose up -d
```
Will make sure that it now runs on it’s own. you’ll see that in your terminal, and can check if it runs by:
```
docker stats
```

It neatly shows you the running containers!

Now if you’d like to see the logs, use the following command:

docker logs homebridge

Note: ctrl+c out of the logs will not stop the container!

3.1.2. Shutting down your container

docker-compose down

(easy huh…)

3.1.3. Log in to the docker container

Use docker ps or docker stats to get the name of the existing container.
Use the command docker exec -it <container name> /bin/bash to get a bash shell in the container. (assuming the container has bash in it’s image). exit will get you out.
Or directly use docker exec -it <container name> <command> to execute the command you specify inside the container. This can be useful for updating NPM packages for instance on a node.js based app container.

3.1.4. Update all your docker images

docker images |grep -v REPOSITORY|awk '{print $1}'|xargs -L1 docker pull

I’m not using this one often since it takes time, and it will update the images for all containers, not always knowing what changed.

3.1.5. Connect a finder folder to SSH / SFTP (connect from your mac to the Pi)

Your pi will show up as ‘bonjour enabled SFTP’ drive already in apps like transmit, but if you want to enable access through SSH regularly, here’s how to do it.

Here’s a thread

Installing two homebrew packages (on Mac)

brew cask install osxfuse
brew install sshfs

Adding a volume that is connected through SSH:

sshfs pi@applepie.local:/home/pi/ /Volumes/Applepie -ovolname=HOME

Removing it (forceful):

sudo diskutil unmount force /Volumes/Applepie