Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@
and amends the first consequence of
[ADR-0039](0039-decline-proxmox-clustering-while-ifrit-is-the-range.md)

> [!NOTE]
> **One premise was wrong, 2026-09-20.** The Context below says that today
> nothing on VLAN 30 is admitted to `8006` on `Saruman`, and the Decision
> widens ADR-0014's rule by one line. Building `phoenix` found the Proxmox
> firewall on `Saruman` disabled, with neither `cluster.fw` nor `host.fw`
> present: ADR-0014's rule was decided, and the runbook that applies it —
> `build-the-playground.md` §4 — is gated on #101 and has not run. So every
> address on the segment reaches `8006`, and the line this ADR adds has no
> wall to go in. The decision stands as written — the line is what to write
> when the wall goes up, and the runbook's CAUTION not to raise it as a side
> effect of a guest build held. Enabling the firewall with all four rules is
> [#566](https://github.com/Gerrrt/HomeLab/issues/566). The text below is
> left as written, per ADR-0001.

## Context

[#436](https://github.com/Gerrrt/HomeLab/issues/436) is the prerequisite for
Expand Down
4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,10 +185,10 @@ hole from the monitoring VLAN into the monitored one.
| Host | VLAN | Stack | Contents |
| --- | --- | --- | --- |
| `prometheus` (10.0.99.20) | 🔴 99 | [`stacks/observability`](../stacks/observability) | Prometheus, Alertmanager, Loki, Grafana, snmp-exporter, blackbox-exporter, docker-socket-proxy, Alloy |
| `Saruman` (10.0.30.110) | 🟢 30 | *(none — and none intended)* | Proxmox VE 9, running one guest — `alexander`, built 2026-09-05 ([#262](https://github.com/Gerrrt/HomeLab/issues/262)) — and eight more planned below: `odin` for the security tooling (ADR-0030), `phoenix` for the deployment toolchain (ADR-0043) and ADR-0029's six-machine domain; Alloy agent (native package). It runs no compose stack by decision, not by omission: Docker would rewrite the iptables its own firewall relies on ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md)), which is why the agent here is the native package and why `stacks/lab` runs in a guest |
| `Saruman` (10.0.30.110) | 🟢 30 | *(none — and none intended)* | Proxmox VE 9, running two guests — `alexander`, built 2026-09-05 ([#262](https://github.com/Gerrrt/HomeLab/issues/262)), and `phoenix`, built 2026-09-20 ([#436](https://github.com/Gerrrt/HomeLab/issues/436)) — and seven more planned below: `odin` for the security tooling (ADR-0030) and ADR-0029's six-machine domain; Alloy agent (native package). It runs no compose stack by decision, not by omission: Docker would rewrite the iptables its own firewall relies on ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md)), which is why the agent here is the native package and why `stacks/lab` runs in a guest |
| `alexander` (10.0.30.40) | 🟢 30 | [`stacks/lab`](../stacks/lab) | Prometheus, Loki, Grafana, Alloy — the lab's own observability, which never remote-writes to VLAN 99 ([ADR-0007](adr/0007-defensive-estate-and-offensive-range.md), [ADR-0020](adr/0020-run-the-lab-stack-in-a-guest-with-its-own-prometheus.md)). A guest on `Saruman`, not the hypervisor; Alloy agent (Docker) |
| `odin` (10.0.30.60) | 🟢 30 | [`stacks/soc`](../stacks/soc) | **Not built yet** — Wazuh (indexer, manager, dashboard), Velociraptor and Alloy: the security half of ADR-0007, placed by [ADR-0030](adr/0030-give-the-security-tooling-its-own-guest-and-its-own-stack.md) on a second guest on `Saruman` because `alexander`'s 8 GiB cannot hold both. The stack is authored and CI-validated ahead of the guest, the way `stacks/lab` was ahead of `alexander`; the build is [`build-the-soc-guest.md`](runbooks/build-the-soc-guest.md), behind [#414](https://github.com/Gerrrt/HomeLab/issues/414). The six domain machines report to it as agents; its Alloy pushes to `alexander`, never to VLAN 99, and the indexer's health is the one series that crosses into the lab's Prometheus. Alloy agent (Docker) |
| `phoenix` (10.0.30.70) | 🟢 30 | *(none — a toolchain host, no Docker)* | **Not built yet** — the deployment host: a Proxmox API token, an SSH key and a checkout, so that the Packer, OpenTofu and Ansible work after [#436](https://github.com/Gerrrt/HomeLab/issues/436) has somewhere to run from. A guest on `Saruman`, placed by [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md), which also decides that the estate's CA key stays on `prometheus` and does not follow the toolchain here. It holds no age key and converges nothing; it is the WireGuard endpoint of [ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md) once that is built; the one thing it reaches that no other guest does is `8006` on `Saruman`, by a single host-firewall rule. Its Alloy pushes to `alexander`, never to VLAN 99; the build is [`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). Alloy agent (native package, `scripts/deploy-agent.sh`) |
| `phoenix` (10.0.30.70) | 🟢 30 | *(none — a toolchain host, no Docker)* | The deployment host, built 2026-09-20: a Proxmox API token, an SSH key and a checkout, so that the Packer, OpenTofu and Ansible work after [#436](https://github.com/Gerrrt/HomeLab/issues/436) has somewhere to run from. A guest on `Saruman`, placed by [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md), which also decides that the estate's CA key stays on `prometheus` and does not follow the toolchain here. It holds no age key and converges nothing; it is the WireGuard endpoint of [ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md) once that is built; the one thing it is meant to reach that no other guest does is `8006` on `Saruman`, by a single host-firewall rule — decided, and unwritten while [#566](https://github.com/Gerrrt/HomeLab/issues/566) is open, because the build found that firewall disabled. Its Alloy pushes to `alexander`, never to VLAN 99; the build is [`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). Alloy agent (native package, `scripts/deploy-agent.sh`) |
| `oracle` (10.0.99.30) | 🔴 99 | *(none — hand-run containers)* | The Lemmiwinks wiki and its Postgres, since 2025-11-12 ([ADR-0011](adr/0011-keep-the-wiki-internal.md)); Alloy agent (Docker, `scripts/deploy-agent.sh`); the off-host copies of the firewall export (`make backup-firewall`), of the weekly volume sets (`make backup`, [#535](https://github.com/Gerrrt/HomeLab/issues/535)) and of Jellyfin's state pulled off `smaug` (`make backup-nas`, [ADR-0045](adr/0045-pull-jellyfins-state-from-a-snapshot-over-ssh.md)). The estate's host for small off-host jobs — [ADR-0015](adr/0015-give-oracle-the-off-host-jobs.md) |
| `trinity` (10.0.99.40) | 🔴 99 | [`stacks/sensitive`](../stacks/sensitive) | **Not built yet** — ADR-0008's sensitive tier on the ProDesk 600 G4 of [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md), after the firewall restore is rehearsed on it ([#404](https://github.com/Gerrrt/HomeLab/issues/404)). The foundation is authored: Caddy as the published HTTPS port and step-ca issuing beneath the tier's own root rather than the estate's, which is left untouched ([#129](https://github.com/Gerrrt/HomeLab/issues/129), [#130](https://github.com/Gerrrt/HomeLab/issues/130), [ADR-0037](adr/0037-give-the-sensitive-tier-its-own-root-and-issue-beneath-it-over-acme.md)), with AdGuard Home behind Caddy and publishing 53 to the firewall's forwarder alone ([#135](https://github.com/Gerrrt/HomeLab/issues/135), [ADR-0010](adr/0010-keep-the-resolver-on-the-gateway.md)); Home Assistant ([#134](https://github.com/Gerrrt/HomeLab/issues/134)), Immich — four containers behind Caddy with a memory limit on each ([#132](https://github.com/Gerrrt/HomeLab/issues/132)) — Paperless-ngx with a Postgres and a Valkey of its own ([#133](https://github.com/Gerrrt/HomeLab/issues/133)) and Vaultwarden ([#131](https://github.com/Gerrrt/HomeLab/issues/131)) are authored as well; the rest follow. Alloy agent (Docker, `scripts/deploy-agent.sh`), pushing to `prometheus` like `oracle`'s |
| `smaug` (10.0.40.30) | 🟡 40 | [`stacks/media`](../stacks/media) | [ADR-0008](adr/0008-place-services-by-data-trust.md)'s media tier on the ThinkServer TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), placed and addressed by [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) and running TrueNAS rather than Ubuntu Server by [ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md). **Built, pooled and deployed**: TrueNAS on its boot SSD, the static above since 2026-09-16, the four inbound rules verified in position, the mirror `erebor` since 2026-09-18 and this stack running on it since 2026-09-19 — from a copy of the compose file on the pool, brought up with `docker compose` under TrueNAS's own Docker ([`build-the-nas.md`](runbooks/build-the-nas.md) §6). Jellyfin alone, publishing 8096 to the segment because the televisions reach it natively and no firewall rule is involved at all ([#138](https://github.com/Gerrrt/HomeLab/issues/138)). Scraped by `prometheus` on `9100`; it pushes nothing, and runs no Alloy — the estate's first scraped host, and the reason [#256](https://github.com/Gerrrt/HomeLab/issues/256) was more than a line of YAML. That issue settled the fork TrueNAS opened in it: `node_exporter`, as a digest-pinned container in this stack rather than TrueNAS's own endpoint, so the existing `99 → 40:9100` pass, the `host-overview` dashboard and seven rules in `host.rules.yaml` all keep working unchanged. The `node` job and `prometheus/targets/node.yaml` are live, and the target with them since 2026-09-19, once the exporter answered from the pool |
Expand Down
2 changes: 1 addition & 1 deletion docs/hardware.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ quietly swapped.
| `smaug` | Lenovo ThinkServer TS150 | Xeon E3-1225 v6 (4 cores) | 8 GB ECC | 240 GB SATA SSD (boot) + 2× 18 TB ZFS mirror `erebor` | TrueNAS 25.10 |

The observability stack runs on a thirteen-year-old MacBook. It handles four
SNMP devices at a 60-second interval, four Alloy agents, and 30 days of metric
SNMP devices at a 60-second interval, five Alloy agents, and 30 days of metric
retention without complaint — which is a useful thing to know before spending
money on a monitoring host. Its RAM is soldered at 8 GB and it has no built-in
Ethernet, so it reaches the network over a USB NIC.
Expand Down
33 changes: 20 additions & 13 deletions docs/network.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,13 +91,14 @@ truth for what a box actually does.
address ([`security.md`](security.md#what-this-repository-deliberately-does-not-publish)).
Before it, `morpheus` carried no `rdr` and no inbound WAN pass beyond DHCP
client replies — the state ADR-0011 measured in 2026-08. **Not built:** the
jumpbox ([#436](https://github.com/Gerrrt/HomeLab/issues/436)) does not exist,
so this describes a decision rather than a rule on the box. The endpoint
jumpbox exists — `phoenix`, built 2026-09-20
([#436](https://github.com/Gerrrt/HomeLab/issues/436)) — and the endpoint
question is answered —
[ADR-0044](adr/0044-answer-the-endpoint-with-dynamic-dns-from-morpheus.md):
a dynamic DNS record kept current by `morpheus`'s own client, on a WAN
address measured to be public rather than carrier-grade NAT — and the client
is not yet configured either.
address measured to be public rather than carrier-grade NAT — but no `rdr`
has been written and the client is not yet configured, so this still
describes a decision rather than a rule on the box.

[^modem]: [Xfinity Gateway (XB7)](https://www.xfinity.com/support/articles/broadband-gateways-userguides)
[^ProDesk]: [HP ProDesk 600 G4 Mini](https://www.microcenter.com/product/692358/)
Expand Down Expand Up @@ -360,6 +361,7 @@ Where things get broken on purpose.
| shiva | `10.0.30.10` | `94:57:a5:xx:xx:xx` | HPE iLO 4 (DL360 Gen9 BMC)[^Shiva] | iLO 2.82 | Rack U3 | Out-of-band management |
| Saruman | `10.0.30.110` | `14:02:ec:xx:xx:xx` | HPE ProLiant DL360 Gen9[^Shiva] | Proxmox VE 9 | Rack U3 | Hypervisor |
| alexander | `10.0.30.40` | `bc:24:11:xx:xx:xx` | KVM guest on `Saruman` | Ubuntu 26.04 LTS | Rack U3 | Lab observability |
| phoenix | `10.0.30.70` | `bc:24:11:xx:xx:xx` | KVM guest on `Saruman` | Ubuntu 26.04 LTS | Rack U3 | Deployment host |

### Notes

Expand All @@ -368,8 +370,10 @@ Where things get broken on purpose.
dedicated port, `Saruman` is the Proxmox install. They are separate addresses
and separate names, and conflating them is a mistake this document previously
made.
- `Saruman` runs one guest, `alexander`, built 2026-09-05
([#262](https://github.com/Gerrrt/HomeLab/issues/262)). It runs
- `Saruman` runs two guests: `alexander`, built 2026-09-05
([#262](https://github.com/Gerrrt/HomeLab/issues/262)), and `phoenix`, built
2026-09-20 ([#436](https://github.com/Gerrrt/HomeLab/issues/436)), described
below. `alexander` runs
[`stacks/lab`](../stacks/lab) — the lab's own Prometheus, Loki, Grafana and
Alloy. **It is a guest and not the hypervisor for a reason**: a compose stack
is Docker, and Docker would rewrite the iptables of the box whose own
Expand All @@ -390,13 +394,13 @@ Where things get broken on purpose.
pushes to `alexander` and not to Winterfell — "guests get no such rule"
covers it — and ADR-0029's six machines report to it as agents. Every path
it needs is intra-segment, so it adds no firewall rule.
- A third guest, `phoenix`, is planned at `10.0.30.70` — the next decade — as
- A third guest, `phoenix`, is at `10.0.30.70` — the next decade — as
the deployment host: the Proxmox API token, the SSH key and the checkout
that the Packer, OpenTofu and Ansible work after
[#436](https://github.com/Gerrrt/HomeLab/issues/436) runs from, placed by
[ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md).
**Not built**; it enters the table above when it is
([`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md)). It runs no stack
[ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md)
and built 2026-09-20 by
[`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). It runs no stack
and holds no key that signs anything — the estate's CA stays on
`prometheus`, and that ADR says why. It is also where
[ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md)'s
Expand All @@ -406,10 +410,13 @@ Where things get broken on purpose.
not to Winterfell, and it gets no pass out of this segment: a rule from it
into 99 would make it the bastion ADR-0002 and ADR-0012 declined. It adds
nothing on `morpheus`. What it adds is on `Saruman`: one line in the
hypervisor's own firewall admitting `10.0.30.70` to `8006`, the first
address on this segment with a path to the Proxmox API, which
hypervisor's own firewall admitting `10.0.30.70` to `8006`, which
`firewall-claims.yaml` cannot see because it lives in `/etc/pve` and not in
pf.
pf. That line is decided and not written: on 2026-09-20 the build found
`Saruman`'s firewall disabled and ADR-0014's rules never applied, so today
every address on this segment reaches `8006`, and
[#566](https://github.com/Gerrrt/HomeLab/issues/566) carries closing it
with all four rules.
- `Saruman` runs an Alloy agent and is the one host on this segment with a path
into Winterfell: a single pass, `10.0.30.110 → 10.0.99.20` on 9090 and 3100
TCP, unlogged and above the ADR-0014 tripwire. The hypervisor's own telemetry
Expand Down
7 changes: 6 additions & 1 deletion docs/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,12 @@ ADR-0007's decision — lab telemetry stays in the lab, so that deliberately
hostile data never lands in the store the estate is actually run from — and
[ADR-0020](adr/0020-run-the-lab-stack-in-a-guest-with-its-own-prometheus.md)
settles its shape. It runs on `alexander`, the guest
[#262](https://github.com/Gerrrt/HomeLab/issues/262) built on 2026-09-05.
[#262](https://github.com/Gerrrt/HomeLab/issues/262) built on 2026-09-05. Its
first client from off that guest is `phoenix`, the deployment host
([ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md),
built 2026-09-20): its native Alloy pushes over the two ports the lab stack
published for it ([#436](https://github.com/Gerrrt/HomeLab/issues/436)), and
it too reaches nothing here.

There is a third, and it reports to the second. [`stacks/soc`](../stacks/soc)
is Wazuh and Velociraptor on `odin`, another guest on `Saruman`
Expand Down
2 changes: 1 addition & 1 deletion docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -1179,7 +1179,7 @@ what left this one unfireable for months.
- **[#436](https://github.com/Gerrrt/HomeLab/issues/436) Build a deployment
jumpbox on ImaginationLAN, and decide where the CA lives.** Decided by
[ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md);
the guest is not built. The host is `phoenix`, `10.0.30.70`, a guest on
the guest was built 2026-09-20. The host is `phoenix`, `10.0.30.70`, a guest on
`Saruman` that holds the estate's first Proxmox API credential and the SSH
key the toolchain will inject into what it builds — the prerequisite for
the Packer, OpenTofu and Ansible issues, which today have nowhere to run
Expand Down
Loading
Loading