I am working on setting up a home server but I want it to be reproducible if I need to make large changes, switch out hardware, or restore from a failure. What do you use to handle this?
How do you manage your home server configuration
Poorly, which is to say that I just let borgmatic back up all my compose files and hope for the best
Git controlled docker-compose files and backed up docker data volumes.pretty easy to go back to a point in time.
reproducible
You tried writing bash scripts that set things up for you, haven’t you? It’s NixOS for you.
NixOS
Out of curiosity: Are you running nix-ops with nix-secrets or how did you cover orchestration & credentials?
I use flakes and all hosts are configured from a single flake, where each host has its own configuration. I have some custom modules and even custom package in the same flake. I also use home manager. I have 4 hosts managed in total: home server, laptop, gaming PC, and a cloud server. All hosts were provisioned using nixos-anywhere + disko, except for the first one which was installed manually. For secrets I use sops-nix, encrypted secrets are stored in the same flake/repo.
Not op, but using colmena for that
NixOS for configuration and restic for data
systemd unit files, because its all podman containers.
Proxmox on the metal, then every service as a docker container inside an LXC or VM. Proxmox does nice snapshots (to my NAS) making it a breeze to move them from machine to machine or blow away the Proxmox install and reimport them. All the docker compose files are in git, and the things I apply to every LXC/VM (my monitoring endpoint, apt cache setup etc) are all applied with ansible playbooks also in git. All the LXC’s are cloned from a golden image that has my keys, tailscale setup etc.
This is pretty much my setup as well. Proxmox on bare metal, then everything I do are in Ubuntu LXC containers, which have docker installed inside each of them running whatever docker stack.
I just installed Portainer and got the standalone agents installed on each LXC container, so it’s helped massively with managing each docker setup.
Of course you can do whatever base image you want for the LXC container, I just prefer Ubuntu for my homelab.
I do need to setup a golden image though to make stand-ups easier…one thing at a time though!
Incus and ansible
I got a bunch of docker compose files and the envs documented so its easy to spin things up again or rollback changes. It works well enough if I’m good about keeping everything all up to date and not making changes without noting it down for myself later.
Terraform and ansible. Script service configuration and use source control. Containerize services where possible to make them system agnostic.
How do you decide what’s for Terraform and what’s for Ansible?
They’re good at different things.
Terraform is better at “here is a configuration file - make my infrastructure look like it” and Ansible is better at “do these things on these servers”.
In my case I use Terraform to create proxmox VMs and then Ansible provisions and configures software on those VMs.
Carefully
I’m just using Unraid for the server, after many iterations (PhotonOS, VMware, baremetal Windows Server, …). After many OSes, partial and complete hardware replacements, and general problems, I gave up trying to manage the base server too much. Backups are generally good enough if hardware fails or I break something.
The other side of this is that I’ve moved to having very, very little config on the server itself. Virtually everything of value is in a docker container with a single (admittedly way too large) docker compose file that describes all the services.
I think this is the ideal way for how I use a home server. Your mileage might vary, but I’ve learned the hard way that it’s really hard to maintain a server over the very long term and not also marry yourself to the specific hardware and OS configuration.
I went the nuclear option and am using Talos with Flux to manage my homelab.
My source of truth is the git repo with all my cluster and application configs. With this setup, I can tear everything down and within 30 min have a working cluster with everything installed automatically.
The place I work at uses chef so I use cinc to manage my vms and proxmox host.
I’m the only user of my setup, but I configure docker compose stacks, use configs as bind mounts, and track everything in a git repo synchronized every now and then.
