My Pi home server

Motivation

More and more people in my surrounding have been getting into the home server community. From hosting Plex to running Minecraft servers out of their bedroom. While I played around with some low-level electronics in the past, I was eager to to apply some knowledge I picked up at work.

Between my partner moving from iOS to Android and thereby losing Airdrop and Bo Burnham temporarily removing some of his music from Spotify I already had two good reasons to start thinking about hosting a NAS.

Not long after I stumbled across Pihole and Castblock and the deal was sealed at that point. I got out my Raspberry Pi, an Ethernet cable and an old SSD and I got to work.

Setup

If you want to copy this setup, or simply look further into the setup take a look at the Github Repo please by all means be my guest! All you need to do is install Docker and Docker Compose on the target machine, set up some secrets in the Github repository and the docker-compose file and you should be good to go!

Github secrets are great, because you wouldn't want to include API keys or database credentials in the repository, or storing them permanently on the deployment machine. Including them as part of the repository stores them safely and will include them in the Docker image at deployment - which means you will be able to debug related issues locally in exactly the same way.

github.com

Docker compose

One of the main reasons why I deploy applications as Docker containers is because I can use a single file to deploy every application. When I inevitably break my setup by transferring to a different boot drive or just generally messing about with it, I will have a simple recovery process.

It also means there is a lot less margin for error as I'm no longer manually setting up everything. Docker compose is a way of writing Infrastructure as Code, and this ability to reproduce the exact same setup 1:1 on any machine is a bliss. Especially as it means you can track this setup in git.

Applications

So what is in that Docker compose file?

Pihole

An absolute must-have. Pihole is an easy-to-use method of setting up ad-blocking on your network. Effectively it acts as a DNS server, but it has a blocklist of known ad providers. So when any device that's connected to it makes a DNS request to an ad-server, Pihole will block that request making the client think that the service is unavailable. Oftentimes, this will simply result in no ads being shown. You know those games that get advertised on social media that serve you an ad every 30 seconds? Well, you can actually play those now without having to turn off your WiFi/Data each time. It won't work for everything though, Youtube for example serves their ads from the same service as the actual videos. But it works around 80% of the time.

Ideally, you would set your router to use Pihole as a DNS server. If you have a router that was not provided by your ISP you should be able to do this - but in the UK at least DNS is typically how ISPs block certain websites (which they have to by law) so they don't allow you to do this. But for most common devices like phones, tablets and laptops it is relatively straightforward to change your DNS server.

The Pi handles this great. I have mine connected to my router via Ethernet to reduce additional latency. New DNS requests sometimes have a notable stutter, I would estimate around 200ms more than usual. But successful responses are usually cached on the client anyway, so it isn't that noticeable after the first week of use.

docker-compose.yaml
services:
pihole:
	container_name: pihole
	image: pihole/pihole:latest
	hostname: pihole
	ports:
		- "53:53/tcp"
		- "53:53/udp"
		- "67:67/udp"
		- "81:80/tcp"
		- "443:443/tcp"
	environment:
		TZ: "Europe/London"
	volumes:
		- "pihole-data:/etc/pihole"
		- "pihole-dns:/etc/dnsmasq.d"
	dns:
		- 127.0.0.1
		- 1.1.1.1
	cap_add:
		- NET_ADMIN
	restart: unless-stopped

volumes:
- pihole-data
- pihole-dns

Plex

Plex allows you to store media on your NAS and watch it anywhere on your network, including Chromecasts. Personally, I use it more to keep copies of my favourite Youtube videos in case they get taken down in the future. But it's also amazing for storing a personal movie collection as it finds metadata, cover art etc for you and displays it in a nice app or web-interface for you.

Sadly, with all those other bits running, the Pi does get a bit overwhelmed with certain content. Compression is strongly favoured, and I keep my files at 720p. I'm hoping that when Pi 4s are available again, I'll be able to upgrade and increase this to 1080p.

docker-compose.yaml
services:
plex:
	container_name: plex
	image: lscr.io/linuxserver/plex:latest
	network_mode: host
	restart: unless-stopped
	environment:
		- PUID=100
		- PGID=1000
		- VERSION=docker
	volumes:
		- /home/lilly/filebrowser/files/plex/config:/config
		- /home/lilly/filebrowser/files/plex/tvshows:/tv
		- /home/lilly/filebrowser/files/plex/movies:/movies
		- /home/lilly/filebrowser/files/plex/music:/music
		- /home/lilly/filebrowser/files/plex/videos:/videos
		- plexappstate:/var/lib/plexmediaserver/Library/Application Support/Plex Media Server

volumes:
- plexappstate

Filebrowser

There are certainly better NAS solutions out there. But Filebrowser is easy to set up and has a simple interface. Personally, I only really use it to store things like ROMs, user manuals etc - things that I find myself downloading about once every 6 months. It's more for my partner as he's recently moved from iOS to Android and needs to move files between his devices quite frequently and he's feeling the absence of Airdrop. This is a simple solution that doesn't compress anything, doesn't send anything to the cloud, and doesn't try to push another monthly subscription onto you. This means that we are in charge of backups, maintenance and increasing storage capacity, but it's exactly the straightforward solution we were looking for.

docker-compose.yaml
services:
filebrowser:
	container_name: filebrowser
	image: hurlenko/filebrowser
	restart: unless-stopped
	ports:
		- "5000:5000"
	volumes:
		- /home/lilly/filebrowser/files:/data
		- /home/lilly/filebrowser/branding:/branding
		- /home/lilly/filebrowser/config:/config
	environment:
		- FB_BASEURL: "/files"

Pi VPN

Of course, I can have all the apps running on my home server within my network, but as soon as I leave the house I lose access to all of it. So hosting a VPN is the obvious solution to this problem - especially for Pihole as having a proper adblocker while I'm out and about is fantastic.
A lot of performance is left on the table though. Connections are slow, and I would not want to watch a Youtube video through this connection.

PiVPN is incredibly easy to set up. I had a few attempts at setting up OpenVPN but to no avail. You set it running as a Docker container and don't really need to touch it again other than setting up your connection profiles. If you know your stuf with VPNs then this is probably not for you, but it's simple enough for me and gets the basic job done.

Castblock

If you're familiar with what SponsorBlock is in Youtube Vanced (RIP) then you get the general idea of Castblock. It effectively scans your 'cast-able' devices on the network and implements sponsorblock. The community reports where the sponsor reads are in a video and it will skip over the video for you.

docker-compose.yaml
services:
castblock:
	container_name: castblock
	image: erdnaxeli/castblock
	restart: unless-stopped
	network_mode: host
	environment:
		- MUTE_ADS: true

What it can't do unfortunately is block the actual ads that Youtube places in front of a video. Even if there was a way that the developers could, I suspect that Google would be pretty quick in sending a Cease&Desist. But it does have the ability to mute ads that are playing. The volume adjustment is not always quite perfect, but the volume is always back on for when the video starts/resumes so I can't see any harm in having it enabled.

Postgres

Having a database always accessible in your network is very handy. When prototyping for a new project I can rely on having an external database available in a safe environment, meaning I don't have to worry about security just yet. It also provides the flexibility of not having to install a local Postgres instance on every machine I want to develop on. Whether that's my personal laptop, my work laptop after work or my desktop PC on the weekends - I can just clone the git repo, install dependencies and set off exactly where I left off.

There's no specific reason for Postgres out of all the SQL flavours. It's just a kind that I have previous experience in and have come to like. And the setup to run it inside of a Docker container is much more simplified than installing it on bare metal while retaining the ability to take backups of the data folder.

I don't use it often, but I have never felt like it was slowing me down, so I would say that the Pi is a good machine to run a basic database on.

Dashboard

Of course, it's nice to have all those apps installed and available, but as my Human Computer Interactions professor said "if you don't see it, it doesn't exist". Especially if I want my partner to make good use of this and get excited about it too, I need to create a way for them to access it all from his phone etc.

So rather than using some pre-existing solution that checks the status of services unreliably and includes a fancy background I thought I may as well create this myself. A static HTML page would have been enough as effectively, all that it needs to do, is redirect to certain ports. But instead, I decided to build an Express app so that I could write applications for it myself.

The first application I added myself is a timesheet app. When I was on my first graduate placement as a developer I realised how bad my time-keeping was and I was aware that I was working overtime without counting it properly. So instead I decided to build this in my free time so that I can hold myself accountable and not work for free anymore.

Then later in the year when Pokemon Brilliant Diamond was released, I started to get into Pokemon again, and online resources around it have exploded since the Black/White 2 days. Trying to work out something as simple as "at what level does this Pokemon evolve" takes a few minutes of switching between sites, scrolling past trivia, finding the right generation entry etc. This process has become incredibly frustrating, so I set out to find an API and build my own front-end for it. If you're looking to build something similar I strongly recommend PokeAPI. So basic information like item lookups, evolutions, move sets etc are now available at my fingertips. Having this flexibility also allowed me to display content in German and English. So it's equally as useful when I replay childhood games in German on my DS, as it is when I play new games in English on the Switch.

docker-compose.yaml
services:
dashboard:
	container_name: dashboard
	image: helblinglilly/piserver:TIMESTAMP
	restart: unless-stopped
	ports:
		- "80:8080"
	environment:
		- POSTGRES_PASSWORD: "PASSWORD"

Limitations

So I am running a considerable amount of applications on this poor Raspberry Pi 3B+. I am currently looking at getting Sonarr set up for Plex, but I just don't have the performance spare for it, and it somehow refuses to work on my 0w2. So my ability to expand is starting to fade, which is why I'm desperately looking at getting a Pi 4 now. Getting a second, powerful Pi would then enable me to start tinkering with distributed computing at a lower scale.

Currently, my Pi just sits behind my ISP's router on a little table, which means that physical space is limited. I wouldn't be comfortable deploying a 3.5" HDD in this current location. In a dream scenario, this would all just be an old desktop sitting on the floor, but this comes with its noise and power efficiency implications. But honestly, this is a silent, low-power solution that can sit in my living room without disturbing us or driving up our energy bill. Hopefully ARM will continue to grow more into mainstream computing. Modern tech, especially GPUs, keep getting more and more powerful but their power consumption is equally sky-rocketing. Situations like this call for power-efficient systems outside of laptops.../../../components/SocialPreview.jsx