diff --git a/docs/adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md b/docs/adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md index ce31f757..d3260875 100644 --- a/docs/adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md +++ b/docs/adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md @@ -5,6 +5,20 @@ and amends the first consequence of [ADR-0039](0039-decline-proxmox-clustering-while-ifrit-is-the-range.md) +> [!NOTE] +> **One premise was wrong, 2026-09-20.** The Context below says that today +> nothing on VLAN 30 is admitted to `8006` on `Saruman`, and the Decision +> widens ADR-0014's rule by one line. Building `phoenix` found the Proxmox +> firewall on `Saruman` disabled, with neither `cluster.fw` nor `host.fw` +> present: ADR-0014's rule was decided, and the runbook that applies it โ€” +> `build-the-playground.md` ยง4 โ€” is gated on #101 and has not run. So every +> address on the segment reaches `8006`, and the line this ADR adds has no +> wall to go in. The decision stands as written โ€” the line is what to write +> when the wall goes up, and the runbook's CAUTION not to raise it as a side +> effect of a guest build held. Enabling the firewall with all four rules is +> [#566](https://github.com/Gerrrt/HomeLab/issues/566). The text below is +> left as written, per ADR-0001. + ## Context [#436](https://github.com/Gerrrt/HomeLab/issues/436) is the prerequisite for diff --git a/docs/architecture.md b/docs/architecture.md index 8855fe98..4b73670a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -185,10 +185,10 @@ hole from the monitoring VLAN into the monitored one. | Host | VLAN | Stack | Contents | | --- | --- | --- | --- | | `prometheus` (10.0.99.20) | ๐Ÿ”ด 99 | [`stacks/observability`](../stacks/observability) | Prometheus, Alertmanager, Loki, Grafana, snmp-exporter, blackbox-exporter, docker-socket-proxy, Alloy | -| `Saruman` (10.0.30.110) | ๐ŸŸข 30 | *(none โ€” and none intended)* | Proxmox VE 9, running one guest โ€” `alexander`, built 2026-09-05 ([#262](https://github.com/Gerrrt/HomeLab/issues/262)) โ€” and eight more planned below: `odin` for the security tooling (ADR-0030), `phoenix` for the deployment toolchain (ADR-0043) and ADR-0029's six-machine domain; Alloy agent (native package). It runs no compose stack by decision, not by omission: Docker would rewrite the iptables its own firewall relies on ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md)), which is why the agent here is the native package and why `stacks/lab` runs in a guest | +| `Saruman` (10.0.30.110) | ๐ŸŸข 30 | *(none โ€” and none intended)* | Proxmox VE 9, running two guests โ€” `alexander`, built 2026-09-05 ([#262](https://github.com/Gerrrt/HomeLab/issues/262)), and `phoenix`, built 2026-09-20 ([#436](https://github.com/Gerrrt/HomeLab/issues/436)) โ€” and seven more planned below: `odin` for the security tooling (ADR-0030) and ADR-0029's six-machine domain; Alloy agent (native package). It runs no compose stack by decision, not by omission: Docker would rewrite the iptables its own firewall relies on ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md)), which is why the agent here is the native package and why `stacks/lab` runs in a guest | | `alexander` (10.0.30.40) | ๐ŸŸข 30 | [`stacks/lab`](../stacks/lab) | Prometheus, Loki, Grafana, Alloy โ€” the lab's own observability, which never remote-writes to VLAN 99 ([ADR-0007](adr/0007-defensive-estate-and-offensive-range.md), [ADR-0020](adr/0020-run-the-lab-stack-in-a-guest-with-its-own-prometheus.md)). A guest on `Saruman`, not the hypervisor; Alloy agent (Docker) | | `odin` (10.0.30.60) | ๐ŸŸข 30 | [`stacks/soc`](../stacks/soc) | **Not built yet** โ€” Wazuh (indexer, manager, dashboard), Velociraptor and Alloy: the security half of ADR-0007, placed by [ADR-0030](adr/0030-give-the-security-tooling-its-own-guest-and-its-own-stack.md) on a second guest on `Saruman` because `alexander`'s 8 GiB cannot hold both. The stack is authored and CI-validated ahead of the guest, the way `stacks/lab` was ahead of `alexander`; the build is [`build-the-soc-guest.md`](runbooks/build-the-soc-guest.md), behind [#414](https://github.com/Gerrrt/HomeLab/issues/414). The six domain machines report to it as agents; its Alloy pushes to `alexander`, never to VLAN 99, and the indexer's health is the one series that crosses into the lab's Prometheus. Alloy agent (Docker) | -| `phoenix` (10.0.30.70) | ๐ŸŸข 30 | *(none โ€” a toolchain host, no Docker)* | **Not built yet** โ€” the deployment host: a Proxmox API token, an SSH key and a checkout, so that the Packer, OpenTofu and Ansible work after [#436](https://github.com/Gerrrt/HomeLab/issues/436) has somewhere to run from. A guest on `Saruman`, placed by [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md), which also decides that the estate's CA key stays on `prometheus` and does not follow the toolchain here. It holds no age key and converges nothing; it is the WireGuard endpoint of [ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md) once that is built; the one thing it reaches that no other guest does is `8006` on `Saruman`, by a single host-firewall rule. Its Alloy pushes to `alexander`, never to VLAN 99; the build is [`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). Alloy agent (native package, `scripts/deploy-agent.sh`) | +| `phoenix` (10.0.30.70) | ๐ŸŸข 30 | *(none โ€” a toolchain host, no Docker)* | The deployment host, built 2026-09-20: a Proxmox API token, an SSH key and a checkout, so that the Packer, OpenTofu and Ansible work after [#436](https://github.com/Gerrrt/HomeLab/issues/436) has somewhere to run from. A guest on `Saruman`, placed by [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md), which also decides that the estate's CA key stays on `prometheus` and does not follow the toolchain here. It holds no age key and converges nothing; it is the WireGuard endpoint of [ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md) once that is built; the one thing it is meant to reach that no other guest does is `8006` on `Saruman`, by a single host-firewall rule โ€” decided, and unwritten while [#566](https://github.com/Gerrrt/HomeLab/issues/566) is open, because the build found that firewall disabled. Its Alloy pushes to `alexander`, never to VLAN 99; the build is [`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). Alloy agent (native package, `scripts/deploy-agent.sh`) | | `oracle` (10.0.99.30) | ๐Ÿ”ด 99 | *(none โ€” hand-run containers)* | The Lemmiwinks wiki and its Postgres, since 2025-11-12 ([ADR-0011](adr/0011-keep-the-wiki-internal.md)); Alloy agent (Docker, `scripts/deploy-agent.sh`); the off-host copies of the firewall export (`make backup-firewall`), of the weekly volume sets (`make backup`, [#535](https://github.com/Gerrrt/HomeLab/issues/535)) and of Jellyfin's state pulled off `smaug` (`make backup-nas`, [ADR-0045](adr/0045-pull-jellyfins-state-from-a-snapshot-over-ssh.md)). The estate's host for small off-host jobs โ€” [ADR-0015](adr/0015-give-oracle-the-off-host-jobs.md) | | `trinity` (10.0.99.40) | ๐Ÿ”ด 99 | [`stacks/sensitive`](../stacks/sensitive) | **Not built yet** โ€” ADR-0008's sensitive tier on the ProDesk 600 G4 of [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md), after the firewall restore is rehearsed on it ([#404](https://github.com/Gerrrt/HomeLab/issues/404)). The foundation is authored: Caddy as the published HTTPS port and step-ca issuing beneath the tier's own root rather than the estate's, which is left untouched ([#129](https://github.com/Gerrrt/HomeLab/issues/129), [#130](https://github.com/Gerrrt/HomeLab/issues/130), [ADR-0037](adr/0037-give-the-sensitive-tier-its-own-root-and-issue-beneath-it-over-acme.md)), with AdGuard Home behind Caddy and publishing 53 to the firewall's forwarder alone ([#135](https://github.com/Gerrrt/HomeLab/issues/135), [ADR-0010](adr/0010-keep-the-resolver-on-the-gateway.md)); Home Assistant ([#134](https://github.com/Gerrrt/HomeLab/issues/134)), Immich โ€” four containers behind Caddy with a memory limit on each ([#132](https://github.com/Gerrrt/HomeLab/issues/132)) โ€” Paperless-ngx with a Postgres and a Valkey of its own ([#133](https://github.com/Gerrrt/HomeLab/issues/133)) and Vaultwarden ([#131](https://github.com/Gerrrt/HomeLab/issues/131)) are authored as well; the rest follow. Alloy agent (Docker, `scripts/deploy-agent.sh`), pushing to `prometheus` like `oracle`'s | | `smaug` (10.0.40.30) | ๐ŸŸก 40 | [`stacks/media`](../stacks/media) | [ADR-0008](adr/0008-place-services-by-data-trust.md)'s media tier on the ThinkServer TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), placed and addressed by [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) and running TrueNAS rather than Ubuntu Server by [ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md). **Built, pooled and deployed**: TrueNAS on its boot SSD, the static above since 2026-09-16, the four inbound rules verified in position, the mirror `erebor` since 2026-09-18 and this stack running on it since 2026-09-19 โ€” from a copy of the compose file on the pool, brought up with `docker compose` under TrueNAS's own Docker ([`build-the-nas.md`](runbooks/build-the-nas.md) ยง6). Jellyfin alone, publishing 8096 to the segment because the televisions reach it natively and no firewall rule is involved at all ([#138](https://github.com/Gerrrt/HomeLab/issues/138)). Scraped by `prometheus` on `9100`; it pushes nothing, and runs no Alloy โ€” the estate's first scraped host, and the reason [#256](https://github.com/Gerrrt/HomeLab/issues/256) was more than a line of YAML. That issue settled the fork TrueNAS opened in it: `node_exporter`, as a digest-pinned container in this stack rather than TrueNAS's own endpoint, so the existing `99 โ†’ 40:9100` pass, the `host-overview` dashboard and seven rules in `host.rules.yaml` all keep working unchanged. The `node` job and `prometheus/targets/node.yaml` are live, and the target with them since 2026-09-19, once the exporter answered from the pool | diff --git a/docs/hardware.md b/docs/hardware.md index c75a27ad..168c3439 100644 --- a/docs/hardware.md +++ b/docs/hardware.md @@ -55,7 +55,7 @@ quietly swapped. | `smaug` | Lenovo ThinkServer TS150 | Xeon E3-1225 v6 (4 cores) | 8 GB ECC | 240 GB SATA SSD (boot) + 2ร— 18 TB ZFS mirror `erebor` | TrueNAS 25.10 | The observability stack runs on a thirteen-year-old MacBook. It handles four -SNMP devices at a 60-second interval, four Alloy agents, and 30 days of metric +SNMP devices at a 60-second interval, five Alloy agents, and 30 days of metric retention without complaint โ€” which is a useful thing to know before spending money on a monitoring host. Its RAM is soldered at 8 GB and it has no built-in Ethernet, so it reaches the network over a USB NIC. diff --git a/docs/network.md b/docs/network.md index 962d1f2f..925f5cb0 100644 --- a/docs/network.md +++ b/docs/network.md @@ -91,13 +91,14 @@ truth for what a box actually does. address ([`security.md`](security.md#what-this-repository-deliberately-does-not-publish)). Before it, `morpheus` carried no `rdr` and no inbound WAN pass beyond DHCP client replies โ€” the state ADR-0011 measured in 2026-08. **Not built:** the - jumpbox ([#436](https://github.com/Gerrrt/HomeLab/issues/436)) does not exist, - so this describes a decision rather than a rule on the box. The endpoint + jumpbox exists โ€” `phoenix`, built 2026-09-20 + ([#436](https://github.com/Gerrrt/HomeLab/issues/436)) โ€” and the endpoint question is answered โ€” [ADR-0044](adr/0044-answer-the-endpoint-with-dynamic-dns-from-morpheus.md): a dynamic DNS record kept current by `morpheus`'s own client, on a WAN - address measured to be public rather than carrier-grade NAT โ€” and the client - is not yet configured either. + address measured to be public rather than carrier-grade NAT โ€” but no `rdr` + has been written and the client is not yet configured, so this still + describes a decision rather than a rule on the box. [^modem]: [Xfinity Gateway (XB7)](https://www.xfinity.com/support/articles/broadband-gateways-userguides) [^ProDesk]: [HP ProDesk 600 G4 Mini](https://www.microcenter.com/product/692358/) @@ -360,6 +361,7 @@ Where things get broken on purpose. | shiva | `10.0.30.10` | `94:57:a5:xx:xx:xx` | HPE iLO 4 (DL360 Gen9 BMC)[^Shiva] | iLO 2.82 | Rack U3 | Out-of-band management | | Saruman | `10.0.30.110` | `14:02:ec:xx:xx:xx` | HPE ProLiant DL360 Gen9[^Shiva] | Proxmox VE 9 | Rack U3 | Hypervisor | | alexander | `10.0.30.40` | `bc:24:11:xx:xx:xx` | KVM guest on `Saruman` | Ubuntu 26.04 LTS | Rack U3 | Lab observability | +| phoenix | `10.0.30.70` | `bc:24:11:xx:xx:xx` | KVM guest on `Saruman` | Ubuntu 26.04 LTS | Rack U3 | Deployment host | ### Notes @@ -368,8 +370,10 @@ Where things get broken on purpose. dedicated port, `Saruman` is the Proxmox install. They are separate addresses and separate names, and conflating them is a mistake this document previously made. -- `Saruman` runs one guest, `alexander`, built 2026-09-05 - ([#262](https://github.com/Gerrrt/HomeLab/issues/262)). It runs +- `Saruman` runs two guests: `alexander`, built 2026-09-05 + ([#262](https://github.com/Gerrrt/HomeLab/issues/262)), and `phoenix`, built + 2026-09-20 ([#436](https://github.com/Gerrrt/HomeLab/issues/436)), described + below. `alexander` runs [`stacks/lab`](../stacks/lab) โ€” the lab's own Prometheus, Loki, Grafana and Alloy. **It is a guest and not the hypervisor for a reason**: a compose stack is Docker, and Docker would rewrite the iptables of the box whose own @@ -390,13 +394,13 @@ Where things get broken on purpose. pushes to `alexander` and not to Winterfell โ€” "guests get no such rule" covers it โ€” and ADR-0029's six machines report to it as agents. Every path it needs is intra-segment, so it adds no firewall rule. -- A third guest, `phoenix`, is planned at `10.0.30.70` โ€” the next decade โ€” as +- A third guest, `phoenix`, is at `10.0.30.70` โ€” the next decade โ€” as the deployment host: the Proxmox API token, the SSH key and the checkout that the Packer, OpenTofu and Ansible work after [#436](https://github.com/Gerrrt/HomeLab/issues/436) runs from, placed by - [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md). - **Not built**; it enters the table above when it is - ([`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md)). It runs no stack + [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md) + and built 2026-09-20 by + [`build-the-jumpbox.md`](runbooks/build-the-jumpbox.md). It runs no stack and holds no key that signs anything โ€” the estate's CA stays on `prometheus`, and that ADR says why. It is also where [ADR-0042](adr/0042-terminate-the-remote-path-on-the-lab-and-route-it.md)'s @@ -406,10 +410,13 @@ Where things get broken on purpose. not to Winterfell, and it gets no pass out of this segment: a rule from it into 99 would make it the bastion ADR-0002 and ADR-0012 declined. It adds nothing on `morpheus`. What it adds is on `Saruman`: one line in the - hypervisor's own firewall admitting `10.0.30.70` to `8006`, the first - address on this segment with a path to the Proxmox API, which + hypervisor's own firewall admitting `10.0.30.70` to `8006`, which `firewall-claims.yaml` cannot see because it lives in `/etc/pve` and not in - pf. + pf. That line is decided and not written: on 2026-09-20 the build found + `Saruman`'s firewall disabled and ADR-0014's rules never applied, so today + every address on this segment reaches `8006`, and + [#566](https://github.com/Gerrrt/HomeLab/issues/566) carries closing it + with all four rules. - `Saruman` runs an Alloy agent and is the one host on this segment with a path into Winterfell: a single pass, `10.0.30.110 โ†’ 10.0.99.20` on 9090 and 3100 TCP, unlogged and above the ADR-0014 tripwire. The hypervisor's own telemetry diff --git a/docs/observability.md b/docs/observability.md index abd88edb..61b8c807 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -11,7 +11,12 @@ ADR-0007's decision โ€” lab telemetry stays in the lab, so that deliberately hostile data never lands in the store the estate is actually run from โ€” and [ADR-0020](adr/0020-run-the-lab-stack-in-a-guest-with-its-own-prometheus.md) settles its shape. It runs on `alexander`, the guest -[#262](https://github.com/Gerrrt/HomeLab/issues/262) built on 2026-09-05. +[#262](https://github.com/Gerrrt/HomeLab/issues/262) built on 2026-09-05. Its +first client from off that guest is `phoenix`, the deployment host +([ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md), +built 2026-09-20): its native Alloy pushes over the two ports the lab stack +published for it ([#436](https://github.com/Gerrrt/HomeLab/issues/436)), and +it too reaches nothing here. There is a third, and it reports to the second. [`stacks/soc`](../stacks/soc) is Wazuh and Velociraptor on `odin`, another guest on `Saruman` diff --git a/docs/roadmap.md b/docs/roadmap.md index 218813f8..a10d6ee6 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1179,7 +1179,7 @@ what left this one unfireable for months. - **[#436](https://github.com/Gerrrt/HomeLab/issues/436) Build a deployment jumpbox on ImaginationLAN, and decide where the CA lives.** Decided by [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md); - the guest is not built. The host is `phoenix`, `10.0.30.70`, a guest on + the guest was built 2026-09-20. The host is `phoenix`, `10.0.30.70`, a guest on `Saruman` that holds the estate's first Proxmox API credential and the SSH key the toolchain will inject into what it builds โ€” the prerequisite for the Packer, OpenTofu and Ansible issues, which today have nowhere to run diff --git a/docs/runbooks/build-the-jumpbox.md b/docs/runbooks/build-the-jumpbox.md index 9c9d7f00..2395d697 100644 --- a/docs/runbooks/build-the-jumpbox.md +++ b/docs/runbooks/build-the-jumpbox.md @@ -4,7 +4,7 @@ **Time:** about an hour, most of it the OS installer **You will need:** the Proxmox web UI on `Saruman` (or a shell on it through the KVM or `shiva`), an Ubuntu Server ISO, the pfSense UI on `morpheus` for one -reservation, a shell on `alexander` for ยง5 if the lab's ports are still shut, +reservation, a shell on `alexander` for ยง5, and the Mac for ยง6 โ€” it is the only machine that reaches this segment with a checkout in hand **After this:** [`open-the-remote-path.md`](open-the-remote-path.md), if the @@ -13,6 +13,22 @@ remote path is wanted โ€” terminates the WireGuard tunnel on this host, and its runbook begins where this one ends +> **Status โ€” 2026-09-20: `phoenix` is built and its agent is pushing to +> `alexander`.** +> +> Ubuntu 26.04 LTS, `10.0.30.70`, VMID 170, `bc:24:11` OUI, ISO +> `ubuntu-26.04.1-live-server-amd64.iso`. ยง4's `curl` returned `alexander` +> and `phoenix` โ€” token, ACLs and node name all right. ยง7 passed: `make validate` green in the guest's own checkout (16 +> host-specific skips โ€” no Docker, no age key, by design), both +> `up{instance="phoenix"}` jobs at `1`, and four Loki jobs โ€” `auth.log`, +> `syslog`, `/var/log/*.log` and the journal โ€” so 26.04.1 ships rsyslog as +> 26.04 did. The lab's ports were opened ahead of the guest by #543 and +> applied on `alexander` in ยง5, the first time `deploy-agent.sh`'s +> `--monitoring-host` flag was used. Two things the first run found and this +> runbook now carries: `VM.Monitor` is not a privilege on PVE 9, and the node +> is `Saruman`, capitalised โ€” and one it could not fix, #566: the hypervisor's +> firewall was never on, so the `8006` line was not written. + This builds the host [ADR-0043](../adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md) decided: the one machine whose purpose is to hold credentials for other machines โ€” a Proxmox API token, an SSH key, a checkout โ€” so that the Packer, @@ -125,48 +141,67 @@ the role is its own so that what it can do is a list rather than pveum role add PhoenixBuilder --privs \ "VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Cloudinit \ VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network \ - VM.Config.Options VM.Console VM.Monitor VM.PowerMgmt VM.Audit VM.Snapshot \ + VM.Config.Options VM.Console VM.PowerMgmt VM.Audit VM.Snapshot \ Datastore.AllocateSpace Datastore.AllocateTemplate Datastore.Audit SDN.Use" pveum user add phoenix@pve --comment "deployment host, ADR-0043" pveum acl modify /vms --users phoenix@pve --roles PhoenixBuilder pveum acl modify /storage/local --users phoenix@pve --roles PhoenixBuilder pveum acl modify /storage/local-lvm --users phoenix@pve --roles PhoenixBuilder pveum acl modify /sdn/zones/localnetwork/vmbr0 --users phoenix@pve --roles PhoenixBuilder -pveum acl modify /nodes/saruman --users phoenix@pve --roles PVEAuditor +pveum acl modify /nodes/Saruman --users phoenix@pve --roles PVEAuditor pveum user token add phoenix@pve builder --privsep 0 ``` -Four of those are worth knowing rather than copying: +Five of those are worth knowing rather than copying: +- **No `VM.Monitor`.** Packer's own permission list still names it, and the + first run of this section copied it in; Proxmox VE 8 dropped the privilege, + so on 9 `pveum role add` rejects the whole list and every `acl modify` + after it fails with "role does not exist". The boot command Packer types + goes through `sendkey`, which is `VM.Console`, already granted. A `pveum` + error here means the role was never made โ€” fix the list and rerun the + role and the four ACLs; the `PVEAuditor` line and the token are unaffected. - **`/vms`, the two storages and the bridge, and nothing under `/nodes` but audit.** The token that can create guests must not be able to touch the host firewall ADR-0014 depends on, and `PVEAuditor` on the node is read-only. When Packer or OpenTofu fail with `Permission check failed`, the fix is a privilege added to `PhoenixBuilder`, not the role granted at `/`. -- **`/nodes/saruman`** is the node name as `pvesh get /nodes` prints it, which - is lower-case whatever the documents call the box. +- **`/nodes/Saruman`** is the node name as `pvesh get /nodes` prints it โ€” the + hostname, case and all, and the same string names the directory under + `/etc/pve/nodes/` and the path in every API URL. This runbook first said + lower-case and the ACL went onto a path that does not exist; `pveum acl + modify` does not check. `pveum acl list | grep phoenix` is the check. - **`--privsep 0`** is the issue's choice: the token carries the user's permissions and there is no second set to keep in step. The trade is that it is exactly as powerful as the user, which is why the user is this narrow. - **The secret prints once.** Copy it now; it cannot be shown again, only - regenerated. + regenerated. Copy it into the file in the next block and nowhere else โ€” a + session transcript or a chat is a log, and a secret pasted into one is + rotated, not kept. `pveum user token remove phoenix@pve builder` and the + `token add` line again is the whole rotation. -**Then the door.** ADR-0014 closes `8006` on `Saruman` to `10.0.50.0/24` and -this guest is not on it. ADR-0043 admits one address, on this port and no -other. In `/etc/pve/nodes/saruman/host.fw`, beneath the three rules ADR-0014 -wrote: +**Then the door โ€” which, on the day, had no wall.** ADR-0014 closes `8006` +on `Saruman` to `10.0.50.0/24` and this guest is not on it. ADR-0043 admits +one address, on this port and no other. The line is, in +`/etc/pve/nodes/Saruman/host.fw`, beneath the three rules ADR-0014 wrote: ```ini IN ACCEPT -source 10.0.30.70 -p tcp -dport 8006 -log nolog ``` > [!CAUTION] -> If that file does not exist yet, ADR-0014's rules were never applied on -> `Saruman` and the firewall is off. Do not turn it on as a side effect of -> this line โ€” read [`build-the-playground.md`](build-the-playground.md) ยง4 -> first, with the KVM console to hand, because a `DROP` input policy with the -> rules unrendered locks you out of a machine whose console is a switch away. +> **Check before writing it:** `pve-firewall status` and +> `ls /etc/pve/firewall/cluster.fw /etc/pve/nodes/Saruman/host.fw`. On +> 2026-09-20 the answer was `disabled/running` and neither file โ€” ADR-0014's +> rules had never been applied on `Saruman`, because the runbook that applies +> them, [`build-the-playground.md`](build-the-playground.md) ยง4, is gated on +> #101 and had not run. Do not turn the firewall on as a side effect of this +> line: a `DROP` input policy with the rules unrendered locks you out of a +> machine whose console is a KVM switch away. **The line above was not +> written**; [#566](https://github.com/Gerrrt/HomeLab/issues/566) carries +> enabling the firewall with all four rules, with the console to hand, and +> until it is done every address on this segment reaches `8006`. **On `phoenix`**, the credential and the key: @@ -198,13 +233,15 @@ Prove the door and the token together, from `phoenix`: ```bash set -a; . ~/.config/proxmox/phoenix.env; set +a curl -sk -H "Authorization: PVEAPIToken=${PROXMOX_TOKEN_ID}=${PROXMOX_TOKEN_SECRET}" \ - "${PROXMOX_URL}/nodes/saruman/qemu" | python3 -m json.tool | grep '"name"' + "${PROXMOX_URL}/nodes/Saruman/qemu" | python3 -m json.tool | grep '"name"' ``` `alexander` and `phoenix` at minimum. A connection timeout is the door โ€” the `host.fw` line is missing or the firewall was not reloaded (`pve-firewall -compile` shows what it thinks the rules are). A `401` is the token. An empty -list with a `200` is the ACL: the user can reach the node and see no guests. +compile` shows what it thinks the rules are) โ€” or, while #566 is open, the +network, because there is no door to be shut. A `401` is the token. An empty +list with a `200` is the ACL: the user can reach the node and see no guests, +which is also what a `PVEAuditor` grant on a mis-cased node path looks like. ## 5. Open the lab's doors diff --git a/docs/runbooks/fit-the-saruman-ssds.md b/docs/runbooks/fit-the-saruman-ssds.md index 2433f82b..15709ef0 100644 --- a/docs/runbooks/fit-the-saruman-ssds.md +++ b/docs/runbooks/fit-the-saruman-ssds.md @@ -789,7 +789,11 @@ lvremove /dev/pve/fiotest ## 9. Move the guest -There is exactly one guest, so this is one command rather than a campaign. +There are two guests โ€” `alexander` (140) and, since 2026-09-20, `phoenix` +(170, [#436](https://github.com/Gerrrt/HomeLab/issues/436)), created on +`local-lvm` like the first โ€” so this is one command run twice rather than a +campaign. The commands below name 140; repeat them for 170 once it has +pivoted. ```bash qm config 140