Give the load balancer the real client IP

[lewispb]

Fixes Fizzy: !o 4xx fizzy returns internal IPs, not public IPs.

The bug

Cloudflare puts the caller's address in True-Client-IP. For most of our apps the F5's manage_x_forwarded iRule copies that into X-Forwarded-For, but fizzy's HTTPS virtual server is fastL4 passthrough, so no HTTP iRule can run. kamal-proxy only ever knew about X-Forwarded-For and the TCP peer, so the app tier's access log — the web-* index the !o 4xx report reads — records an internal address. Rails escaped this because TrackTrueClientIp does the copy itself before ActionDispatch::RemoteIp.

The change

kamal-proxy 0.10.0 added --client-ip-header (Kevin's #224 work), which makes the proxy trust a named header both for its own access log and for the X-Forwarded-For it sends on. So, at fizzy-lb:

• --client-ip-header=True-Client-IP in saas/script/configure-lb-*.sh.
• --forward-headers alongside it. Without this the rewrite only fixes the LB's own log: X-Forwarded-For is dropped by default when the proxy terminates TLS, which it does here, so the app tier would keep seeing the peer address. kamal-proxy's own test for the flag pairs the two the same way.
• The LB accessory image moves from basecamp/kamal-proxy:lb to basecamp/kamal-proxy:v0.10.0. Required, not cosmetic — the lb tag was built in Nov 2025 and would reject --client-ip-header outright. Pinning also stops what a host runs depending on when it last rebooted, same reasoning as hotcell's image in deploy.yml.

TrackTrueClientIp stays. The card thread suggested it becomes deletable, and it does become redundant for traffic through the LB, but it still covers the app when reached directly and it collapses the forwarded chain to the one address we trust. Worth removing separately, once this is live.

Worth a look before rolling out

• The app tier's remote_addr becomes a list, "<public>, <lb peer>", because SetXForwarded always appends the peer. The public address is first, but if client.ip is mapped as an ES ip field, a two-element string may fail to index — check the report and the mapping before running the production script. A single clean value at the app tier needs Kamal to expose client_ip_header for the managed proxy; 2.12.0 doesn't, so the app-tier proxy stays on Kamal's default v0.9.2 for now.
• No new trust. --client-ip-header means anything reaching the LB with a True-Client-IP header sets its own logged address — but TrackTrueClientIp already trusts that header unconditionally today, so this places no trust we weren't already placing.
• The lb tag's provenance isn't recorded anywhere. Every flag the configure scripts pass (--read-target, --writer-affinity-timeout, --tls-acme-cache-path) exists in v0.10.0, and the one unmerged load-balancer branch adds X-Kamal-Writer, which fizzy doesn't reference. Still worth staging first.

Rollout

Config changes only — no app deploy needed, but neither file takes effect on its own:

bin/kamal accessory reboot load-balancer -d staging
saas/script/configure-lb-staging.sh

then the same for production.

Testing

bin/ci green locally (every step; gh signoff only wanted the branch pushed).

🤖 Generated with Claude Code

[Copilot]

🔵 Needs a closer look

The rollout changes production proxy trust and header-chain semantics while downstream log-index compatibility remains unverified.

Pull request overview

Updates SaaS load balancers to propagate Cloudflare’s real client IP into application-tier logs.

Changes:

• Trusts True-Client-IP and forwards proxy headers in all environments.
• Pins load balancers to kamal-proxy v0.10.0.
• Documents why TrackTrueClientIp remains necessary.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File summaries

File	Description
saas/script/configure-lb-staging.sh	Configures staging client-IP forwarding.
saas/script/configure-lb-production.sh	Configures production client-IP forwarding.
saas/script/configure-lb-beta.sh	Configures beta client-IP forwarding.
saas/lib/fizzy/saas/true_client_ip.rb	Documents retained Rails middleware behavior.
saas/config/deploy.staging.yml	Pins the staging proxy image.
saas/config/deploy.production.yml	Pins the production proxy image.
saas/config/deploy.beta.yml	Pins the beta proxy image.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.