Skip to content

PostgreSQL Refactor: Standardize all container volume paths and add CI test pipeline - #7

Open
supanadit wants to merge 23 commits into
mainfrom
feat/volume-normalization
Open

supanadit wants to merge 23 commits into
mainfrom
feat/volume-normalization

Conversation

@supanadit

Copy link
Copy Markdown
Owner

What changed

Standardized volume mount paths across all 21 containers to a single
convention under /opt/containers/<category> (data, config, backup,
logs, run). Previously each container used its own path scheme.

Also added a CI pipeline that runs on PRs to automatically test
PostgreSQL container changes — covers smoke, extensions, pgbouncer,
and pgbackrest scenarios.

All containers updated

postgresql, mariadb, pgpool-ii, maxscale, apache-kafka, apache-zookeeper,
apache-flink, apache-spark, apache-cassandra, apache-airflow, prometheus,
thanos, grafana, grafana-alloy, grafana-loki, grafana-mimir,
grafana-pyroscope, grafana-tempo, etcd, minio, wordpress

Migration

This is a breaking change. See MIGRATION_GUIDE.md for the full old→new
path mapping and step-by-step migration instructions.

supanadit added 11 commits June 17, 2026 18:19
BREAKING CHANGE: All default volume mount paths changed to standardized
/opt/containers/<category> convention:
- /opt/containers/data (persistent data)
- /opt/containers/config (configuration files)
- /opt/containers/backup (backup repositories)
- /opt/containers/logs (log output)
- /opt/containers/run (PID files, sockets)

- Updated entrypoints, Dockerfiles, config templates across all 21 containers
- Fixed stale VOLUME declarations in Prometheus (/prometheus->/opt/containers/data)
  and Thanos (/thanos->/opt/containers/data)
- Updated all 47+ example compose files
- Added backward-compatible /content symlink for WordPress
- Added comprehensive migration guide (MIGRATION_GUIDE.md)
AGENTS.md documents the gold standard Docker container patterns
from docker/postgresql/:
- Multi-stage Dockerfile (base → setup → runtime)
- Numbered setup scripts with plugin extension recipe
- Modular entrypoint lifecycle (init → runtime → utils)
- Script template (set -euo pipefail + main function)
- Environment variable conventions and config generation order

openspec/config.yaml populated with project context and per-artifact
rules aligned with the documented patterns.
…pts/

Move from flat test/ directory to clean separation:
  docker/postgresql/tests/
    compose/    <filename>.yaml        (no more compose. prefix)
    scripts/    run-<scenario>.sh

Compose files now live in their own namespace — extensions.yaml
instead of compose.extensions.yaml. Scripts resolve the path
via ../compose/<scenario>.yaml relative to scripts/.
@supanadit supanadit self-assigned this Jun 17, 2026
supanadit added 12 commits June 17, 2026 18:21
Replace the entire bash entrypoint.d/ tree (entrypoint.sh, init/runtime/
scripts, patroni callbacks, sync-reload, healthcheck, pgbouncer.ini) with a
single EZX bootstrap at entrypoint.d/bootstrap/main.js.

- One chain, one root: postgres/patroni node with all sidecars as
  needParentReady children (stanza-init, pgbouncer-userlist, repl-user,
  pgbouncer, sshd, scheduled pgBackRest backups, Patroni role-check).
- Modules mirror the bash structure: main, env, validation, directories,
  database, config (declarative FileProvision), backup (scheduler),
  cluster (role detection), sshd, patroni (role-change reconfig),
  routes (operator api on the health server).
- Dockerfile downloads the pinned ezx release binary (checksum-verified),
  ENTRYPOINT becomes ["ezx", "bootstrap", "main.js"], HEALTHCHECK is
  pg_isready. config/pgbouncer.ini is generated declaratively instead.
- Fixes discovered during container testing:
  - postgresql.conf/pg_hba.conf always overwrite initdb defaults (removed
    createOnly) so unix_socket_directories = PGRUN is applied
  - readiness gate: node.readiness pg_isready probe gates needParentReady
    children on postgres accepting connections
  - pgbouncer runs foreground (no -d daemonize -> no restart loop)
  - sshd runs with -D (no daemonize -> no restart loop)
  - pgbouncer.ini sets unix_socket_dir = PGRUN
  - stanza-init filters PGBACKREST_* env
  - archive_command strips PGBACKREST_ENABLE and PGBACKREST_AUTO_ENABLE
- AGENTS.md documents the PostgreSQL EZX exception.
- Add no_cache=true to forge.toml so the build cache is disabled for the
  whole build (forge never creates cache dirs, no wasted snapshot/tar I/O)
- Bump FORGE_VERSION to 0.6.0 (the release that ships the no_cache feature)
- Drop redundant rm -rf of forge cache dirs from the Dockerfile
Replace the setup.sh + numbered-shell-script build with a forge.toml
component manifest, and remove the now-orphaned setup scripts.

- Rewrite forge.toml to the [[components]] ops model: each component is an
  ordered ops list, install is a polymorphic op (apt/source/binary)
- Distribute component-specific apt deps to their own components with
  build/runtime classification (postgresql, cpython, pgbackrest, pgbouncer,
  patroni, dexter, decoderbufs)
- Add locale setup (en_US.UTF-8) to system-deps
- Add image cleanup component: strip binaries, remove static libs, clear
  Python/Ruby cruft to shrink the image
- Bump FORGE_VERSION 0.6.0 -> 0.9.0
- Delete setup.sh and setup/scripts/*.sh (replaced by forge.toml)
…idate bootstrap

Consolidate the 12-file bootstrap to 7 files and migrate to the new ezx host
API (process.run/capture, fs.write/ensureDir/which, env.int/bool, shell.quote,
scheduler.every, optional children, JS-controlled readiness):

- api.js (new): operator routes + Patroni role-change callback + backup routes;
  role-check is now a native scheduler.every callback (no curl child / HTTP
  round-trip).
- main.js: absorb directories + sshd + cluster setup via fs.ensureDir /
  process.run / fs.which.
- runtime.js: absorb backup node-builders; mark sidecars optional:true so
  pgbouncer/pgbackrest/sshd/backup failures no longer exit the container;
  unbounded shutdown drain (timeout:-1, no force-kill) for large data volumes;
  wire node.health unconditionally so /readyz is always driven by the readiness
  probe (the health server always runs on :8080).
- database.js: process.run args arrays (no shell strings / quoteArg), fs.write
  for whole-file writes.
- validation.js: fs.which + env.int; fix TIMEOUT default regression.
- env.js: env.int/env.bool typed accessors.
- Dockerfile: remove HEALTHCHECK (compose/k8s map probes to ezx /readyz on
  :8080), EXPOSE 8080, bump EZX_VERSION to 0.2.0.
…s superuser

The native-HA replication-user one-shot connected to 127.0.0.1 (TCP) with
PGPASSWORD=REPLICATION_PASSWORD, but authenticated as the postgres superuser —
so password auth failed (exit 2) and the container exited. The original bash
connected via the local unix socket as POSTGRES_USER (trust auth). Fix replUser
to connect via PGRUN socket as the postgres superuser, matching the original
behavior.
Fix the migration regressions found by testing every compose file end-to-end:

- replUser connects via the PGRUN unix socket as the postgres superuser
  (trust auth), not 127.0.0.1 with the replicator password.
- Use the correct 'environment' field (not 'env') so PGPASSWORD reaches
  pg_basebackup and psql.
- Replace require('os') (unavailable in goja) with the HOSTNAME env var.
- configureReplicaAppname appends application_name inside the existing
  primary_conninfo line instead of clobbering host/port/user/password.
- Strip PGBACKREST_* from the postgres process env so archive-push doesn't
  inherit them as unindexed config options.
- Skip pgbackrest stanza-init on native-HA replicas; make it optional in
  Patroni mode (dynamic role) so a replica's failure is non-fatal.
- Tolerate a read-only ~/.ssh mount (SFTP compose provides its own keys).
- Own patroni.yml by postgres (0600) so Patroni can read it.
- In Patroni mode, don't write postgresql.conf/pg_hba.conf into PGDATA
  (Patroni manages its own config and bootstraps the cluster).
…e APIs

- Remove /bin/sh and /bin/true placeholders from all node definitions
- Convert oneshot init scripts to callback-only nodes with onStart handlers
- Convert keepalive sleep to callback-only node
- Replace recursive copy operations with fs.copyTree (preserveOwner)
- All database.js operations now use pure ezx fs module
- Zero external binary dependencies in postgresql entrypoint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant