Specifically, use home.arpa, if you must use a private domain.

I run k3s in my homelab as a single node cluster. I’m very familiar with kubernetes in general, so it’s just easier for me to reason with a control plane.

Some of the benefits I find useful:

• ArgoCD set to fire and forget will automatically update software versions as they happen. I use nix to lower the burden of maintaining my chart forks. Sometimes they break, but
• VictoriaMetrics easily collects all the metrics from everything in the cluster with very little manual tinkering, so I am notified when things break, and
• zfs-localpv provides in-cluster management for data snapshots, so when things do break I can easily roll back to a known good state.

k3s is, of course, a memory hog, I’d estimate it and cilium (my CNS of choice) eat up about 2Gb ram and a bit under one core. It’s something you can tune to some extent, though. But then, I can easily do pod routing via VPN and create services that will automatically get a public IP from my endless IPv6 pool and get that address assigned a DNS name in like 10 lines of Yaml.

Is there anything interesting at all reported in /proc/spl/kstat/zfs/dbgmsg?

I did ran out of pcie, yeah :-( the network peaks at about 26gbit/s, which is the most you can squeeze out of pcie 3.0 x4. I could move the nvmes off the pcie 4.0 x16 (I have two m2 slots on the motherboard itself), but I planned to expand the nvme storage to 4x SSDs and I’m out of the pci lanes on the other end of the fiber either way (that box has all x16 going to the gpu)

I run 3900X with a 40Gbit fiber, packed with HDDs and nvmes. The box fluctuates around 90-110W use.

when you said that Nextcloud might not meet your needs, was your concern specifically the server-side data format?

I’d prefer them as plain files. Technically it doesn’t matter much to me if it’s a database, if I have to spin up an S3-compatible API, or if I need to slice up a zvol for it, but I just prefer the files because then I can do zfs snapshots (in which I trust) and backup with restic (in which I trust)

Try VictoriaMetrics. Basically the same feature set as Prometheus, but so much more resource friendly for homelab scale. I store some metrics for 12 months now, because it’s easy.

oh, that’s actually a fair point! You’re correct.

DHCPv6-PD is still effectively broken, though.

I tried opn/ pfsense, VyOS (the rolling one. Stable is paid only), and a couple commercial options. Surprisingly not a single free/foss option can do IPv6 properly (I was looking specifically for prefix delegation for downstream routers). Cashed out for a single RouterOS CHR license and never bothered since.

But otherwise I tend to like VyOS. the rolling releases as the only free option make it somewhat questionable for something more serious though.

I’d be curious to see comparison with Logseq. As it’s rightly mentioned, there are thousands of note taking apps and I’m not quite sure I see the selling point of SB. I really love the idea of notes as a database, but the query langauage seems subpar, more akin to obsidian’s dataview than the overwhelming power of tiddlywiki’s filters or Logseq’s queries.

I went from evernote to tiddlywiki to Obsidian to Logseq and somewhat stuck here now because I got the powerful queries in a very neat UI. With the market oversaturated as it is, I’d be nice to see what Silverbullet brings to the game that others don’t, what are the distinguishing features.

I disabled DHCP and IPv6

Why, though?

I went for a much simpler approach lately as I downscaled my hardware for efficiency.

I run NixOS on the bare metal. It gives the system management a declarative approach, just like kubernetes would. On top of that, I run libvirt as a hypervisor. In other scenarios I’d use tinyvmm and cloud-hypervisor, but I found qemu way better for the variety of homelab workloads and libvirt is pretty straightforward.

Some vms have pci passthrough, e.g. my routeros vm gets a bunch of NICs directly, some have various funny network topology. Libvirt used to be a pain in that regard, but it’s actually fine with NixOS because you manage both sides of the networking stack in declarative configuration.

I run NixOS on the vms too (now for the sake of easy upgrades), and I have a bit of a split between running services natively (systemd is very good about “containerizing” things nowadays) and using docker (mostly because of laziness, e.g. Elastiflow was easier to deploy this way). Finally, I have a single dokerized Ubuntu that’s more like a VM (as in, I never had a dockerfile for it, it’s fully stateful) running the matter home automaton bits because I gave up on properly containing the matter python stack and went for an easy way out.

Now, a word about alternatives.

I used to run Ubuntu. No more. Upgrading the OS is always a huge pain even if everything is in docker. I want my OS to be managed in a config file and be able to easily roll back to the previous state. I used to run k3s, but even though it is much thinner than k8s, it is still very much ram hungry and I just don’t want to pay for that. Besides, complex networking is often non-trivial due to how its networking works, and multus is a world of pain. I used to run different hypervisors for the VMs (kubevirt, tinyvmm, a bunch others). I went way back to libvirt mostly because it’s straightforward in tuning very specific qemu bits I cared for in the homelab. I have some cpu overprovisioning, so I want to make my quotas set up extremely precisely, sacrificing the right workloads.

I’ll make a note here that a firewall is useful for internal traffic, too. Those IoT devices can get pretty annoying, so you’d want to e.g. drop your cheap webcams into a VLAN and disallow them from talking to enjoying but their cloud, and especially the other VLANs, or isolate Alexa capable device so it won’t try to figure what else you got there in your house over mDNS (it will).

A managed switch would do nicely. Having isolated ports on the switch (and the wifi AP) is also great if you want to make sure the specific device will only talk to the gateway and not its peers.

Regarding firewall stuff, disable it on your machine and you are fine.

How do you know OP doesn’t have a bunch of unsecured services sticking out into their LAN ready to be a target for the next cryptolocking scam?

Slightly sarcastic, but yeah, OP, do not just turn your firewall without understanding pros and cons of doing such. At the very least, see what your server exposes to the network (ss -tunlp will give you a good starting point), and see if there’s nothing unexpected in there that might be abused.

I went looking into how that works, and, apparently, tailscale adds individual node routes (in table 52). So yeah, you have very low chances of getting into trouble even if you have an interface with 100.64/10.

Yeah, you’re absolutely correct. I misread that thinking OP would have the CG NAT endpoint and taikscsle on the same physical device, which, I still think, would be a problem: you’d have two interfaces for 100.64.0.0/10. But if CG NAT terminates on the modem and you run taikscale on devices connected to it them there’s surely no issue at all.

Sorry, I meant the OPs modem.

I’m actually not sure you can easily get tailscale up and running om such as a setup as it uses the same cgnat ip range.

I second this. Loki for logs, VictoriaMetrics for metrics―it’s significantly more lightweight than ELK logging (and any lags are irrelevant for a homelab), and VM is similarly much more careful with RAM than Prometheus.

Look at them like this: VLANs are like running several cables between two spots that you can configure independently. In the very end it comes down to this: what virtual LAN number you have on the cable.

Your backbone devices (router and switches) can be configured to accept tagged traffic―your switch will send a packet prefixed with a VLAN index and your router will trust that the packet actually came from that VLAN on the switch port, or to tag traffic―like when you have some port on your switch where your PC is plugged in and the switch will tag those packets with some VLAN when it forwards them (to the router).

Once you grasp that, everything else pretty much boils down to managing several isolated networkd and how they cross-talk. You run a dhcp server over each network, its own set of other services and whatnot.

Oftentimes the “home” hardware will expect a single network and use some means of packets broadcast to reach each other. That’s how your phone can find all google homes on the network and apple homekit knows where your smart lights are. For that traffic to cross VLANs you’ll have to use some special software like mdns repeaters, but you can still isolate them.

Wrapping up, VLANs basically allow you the physical level isolation over a single cable. Mind that there are, of course, some bugs, e.g. I once found an issue with Unifi access points that allowed a well crafted packet to escape into VLAN 1 no matter what it was supposed to be tagged with. So don’t treat them as physically separate links.