Justin

I would probably remove python 2 support, it was end of life when the project was started.

I dont think Immich supports turning a normal account into an sso account, though it may be possible with manual database editing.

I believe only the controller needs cooling, not the dies.

Kubernetes is great for single nodes! It definitely is more advanced than docker compose, but it’s actually not hard at all if you read through the documentation. It definitely makes running containers easier in the long run.

Here is my git repo for my big Kubernetes cluster at home: https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications

It started out as just a NFS server and a Kubernetes server running on Proxmox in 2021.

It’s not going to make a meaningful difference in your threat model and it will cause a lot of hassle for extra configuration and broken docker images, so I wouldn’t bother.

There is some nice tooling for transparent user name spaces coming down the pipeline in Kubernetes which will be a nice 0-effort security upgrade, but if you don’t have the tooling, I would say it’s not worth it.

https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/

Hetzner Storage box is $20/month for 10tb.

Probably not that hard to build a simple flask frontend around it.

Automatically processing files in an S3/WebDAV directory would also be useful.

https://docs.k3s.io/installation/uninstall

There is also a k3s option for Nixos, which removes the security and side-affect risks of running a random bash script installer.

Very true. Each brick you lay upgrades your setup and your skillset. There are very few mistakes in Kubernetes as long as you make sure your state is backed up.

For question 1: You can have multiple resource objects in a single file, each resource object just needs to be separated by ---. The small resource definitions help keep things organized when you’re working with dozens of precisely configured services. It’s a lot more readable than the other solutions out there.

For question 2, unfortunately Docker Compose is much more common than Kubernetes. There are definitely some apps that provide kubernetes documentation, especially Kubernetes operators and enterprise stuff, but Docker-Compose definitely has bigger market share for self-hosted apps. You’ll have to get experienced with turning a docker compose example into deployment+service+pvc.

Kubernetes does take a lot of the headaches out of managing self-hosted clusters though. The self-healing, smart networking, and batteries-included operators for reverse-proxy/database/ACME all save so much hassle and maintenance. Definitely Install ingress-nginx, cert-manager, ArgoCD, and CNPG (in order of difficulty).

Try to write yaml resources yourself instead of fiddling with Helm values.yaml. Usually the developer experience is MUCH nicer.

Feel free to take inspiration/copy from my 500+ container cluster: https://codeberg.org/jlh/h5b/src/branch/main/argo

In my repo, custom_applications are directories with hand-written/copy-pasted yaml files auto-synced via ArgoCD Operator, while external_applications are helm installations, managed via ArgoCD Operator Applications.

helm charts are awful, i didn’t really like cdk8s either tbh. I think the future “package format” might be operators or Crossplane Composite Resources

Userspace is allowed to break themselves

all home routers have NAT which functions as a firewall, but VPSes don’t cone with any firewall by default, so you’d have to set one up. Also VPS ranges seem to hotter for scanning.

Your stuff is more likely to get scanned sitting in a VPS with no firewall than behind a firewall on a home network

Yeah Stalwart seems to have a lot of momentum, I’ll probably be setting up a server with my kubernetes+ceph cluster this month.

Check out NixOS. It can build qcow images from scratch for you to import into proxmox

https://github.com/nix-community/nixos-generators

I have 8 bare-metal servers and I do everything automated with NixOS, I rarely ever access the servers directly.

Here are the nixos configs for my DHCP server and kubernetes servers that you can use as a base.

https://codeberg.org/jlh/h5b/src/branch/main/porygonz

https://codeberg.org/jlh/h5b/src/branch/main/nodes

For what it’s worth, Ive been using Ansible off and on at work for 8 years, and I think it’s pretty outdated and clunky these days, there are much smarter ways to manage workloads such as kubernetes, cloud-init, terraform, and NixOS. If you don’t want to get into Kubernetes then definitely learn NixOS.

not to mention there are 48 and 64gb dimms out now too that work with basically all alder lake atoms

Yeah, what you’re talking about is called GitOps. Using git as the single source of truth for your infrastructure. I have this set up for my home servers.

https://codeberg.org/jlh/h5b

nodes has NixOS configuration for my 5 kubernetes servers and a script that builds a flash drive for each of them to use as a boot drive (same setup for porygonz, but that’s my dedicated DHCP/DNS/NTP mini server)

mikrotik has a dump of my Mikrotik router config and a script that deploys the config from the git repo.

applications has all my kubernetes config: containers, proxies, load balancers, config files, certificate renewal, databases, clustered raid, etc. It’s all super automated. A pretty typical “operator” container to run in Kubernetes is ArgoCD, which watches a git repo and automatically deploys any changes or desyncs back to the Kubernetes API so it’s always in sync with git. I don’t use any GUI or console commands to deploy or update a container, I just edit git and commit.

The kubernetes cluster runs about 400 containers, most of them just automatic replicas of services for high-availability. Of course there’s always some manual setup steps outside of git, like partitioning drives, joining the nodes to the cluster, writing hardware-specific config, and bootstrapping Argocd to watch git. But overall, my house could burn down tomorrow and I would have everything I need to redeploy using this git repo, the secrets git repo, and my backups of my databases and container /data dirs.

I think Portainer supports doing GitOps on Docker compose? Never used it.

https://docs.portainer.io/user/docker/stacks/add

Argocd is really the gold standard for GitOps though. I highly recommend trying out k3s on a server and running ArgoCD on it, it’s super easy to use.

https://argo-cd.readthedocs.io/en/stable/getting_started/

Kubernetes is definitely different than Docker Compose, and tutorials are usually written for Docker compose.yml, not Kubernetes Deployments, but It’s super powerful and automated. Very hard to crash once you have it running. I don’t think it’s as scary as a lot of people think, and you definitely don’t need more than one server to run it.

nah you’re probably not going to get any benefits from it. The best way to make your setup more maintainable is to start putting your compose/kubernetes configuration in git, if you’re not already.

85C could be throttling