Add Baseshift extension

[whummer]

Summary

Adds an extension for Baseshift, which creates masked, writable clones of production databases for development. The extension runs Baseshift clones (Docker snapshots of a Dub) next to LocalStack, so app code running in LocalStack can work against realistic, anonymized data.

• Default clone: set BASESHIFT_IMAGE and the clone starts once LocalStack is ready
• Clones API: POST/GET/DELETE http://baseshift.localhost.localstack.cloud:4566/clones to start and stop clones on demand (e.g. one per PR), including from images in the LocalStack ECR registry
• Connectivity: PostgreSQL clones are proxied through the gateway port 4566 (protocol handshake detection, like the ParadeDB extension) and published on host ports. MySQL clones only get a host port, since MySQL is server-speaks-first and can't be detected on the shared port
• Config passthrough: BASESHIFT_ENCRYPTION_PASSWORD maps to the clone's PASSWORD, and BASESHIFT_CLONE_<NAME> vars are passed to clones as <NAME>

Architecture

flowchart LR
    subgraph baseshift["Baseshift"]
        dub["Dub<br/>(masked replica of<br/>production database)"]
    end

    registry[("Snapshot registry<br/>AWS ECR, or<br/>LocalStack ECR")]

    subgraph host["Your machine (Docker)"]
        subgraph localstack["LocalStack container"]
            gateway["Gateway :4566"]
            extension["Baseshift extension<br/>clones API"]
            apps["Your app<br/>(Lambda, ECS, ...)"]
        end
        default[("Clone 'default'<br/>:5432")]
        extra[("Clone 'pr-123'<br/>:15432")]
    end

    client["psql / IDE / tests"]

    dub -- "Docker snapshot" --> registry
    registry -- "pull image" --> extension
    extension -- "start / stop" --> default
    extension -- "start / stop" --> extra
    apps -- "SQL" --> gateway
    client -- "SQL" --> gateway
    client -. "SQL (host port)" .-> extra
    gateway -- "PostgreSQL wire protocol" --> default

Loading

Demo

baseshift/demo/ has an end-to-end flow, all in one LocalStack container (Snowflake emulator image, which also serves RDS/ECR/S3):

RDS Postgres with PII (logical replication enabled) → masked snapshot image in LocalStack ECR → clone via the extension → export to S3 → COPY INTO Snowflake → side-by-side view of the same record at each stage, with a check that no raw PII leaked.

The snapshot step is a stand-in for the Baseshift replication server for now (its image is private). demo/baseshift-selfhosted/ has Helm values for LocalStack (validated against the Baseshift chart schema) and a docker compose translation of the chart, for when we get access. Those are untested.

Testing

• CI runs the integration tests on latest and dev, with postgres:17 as a stand-in clone image (real clone images are private per Baseshift customer), using lstk
• 8 tests: queries via gateway and host port, writes, concurrent connections, clones API (incl. a clone from an image pushed to LocalStack ECR, failed starts, validation errors)
• Tests and the full demo passed locally. The demo isn't in CI yet (it needs a license that includes the Snowflake emulator)

Notes

• LocalStack RDS doesn't apply the rds.logical_replication parameter, so the demo sets wal_level=logical directly and reboots (the master user is a superuser in LocalStack)
• Running together with another extension that serves Postgres on port 4566 (e.g. ParadeDB) isn't supported, both claim the same handshakes
• Not yet added to CODEOWNERS

🤖 Generated with Claude Code

[whummer]

Commands for testing this end to end directly from the branch, no checkout needed. Requires Docker, lstk, psql, and LOCALSTACK_AUTH_TOKEN.

A plain postgres:17 image serves as a stand-in for a Baseshift clone image here (real clone images are private per Baseshift customer). lstk has no extension commands, so the extension is installed at startup via EXTENSION_AUTO_INSTALL, from the branch archive (git+https:// installs currently fail with the latest image due to the dulwich git shim).

# 1. Start LocalStack with the extension installed from the branch, and a default clone
LOCALSTACK_EXTENSION_AUTO_INSTALL="localstack-baseshift @ https://github.com/localstack/localstack-extensions/archive/refs/heads/add-baseshift-extension.tar.gz#subdirectory=baseshift" \
LOCALSTACK_BASESHIFT_IMAGE=postgres:17 \
LOCALSTACK_BASESHIFT_CLONE_POSTGRES_HOST_AUTH_METHOD=trust \
  lstk start

# 2. Check the clone status (starts in the background once LocalStack is ready)
curl -s http://baseshift.localhost.localstack.cloud:4566/clones

# 3. Query the default clone, through the LocalStack gateway and on the host port
psql -h localhost.localstack.cloud -p 4566 -U postgres -c "SELECT version()"
psql -h localhost -p 5432 -U postgres -c "SELECT version()"

# 4. Push a "snapshot image" to LocalStack ECR, and start a second clone from it via the clones API
REPO=000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/baseshift/my-dub
lstk aws ecr create-repository --repository-name baseshift/my-dub
docker tag postgres:17 "${REPO}:latest" && docker push "${REPO}:latest"
docker rmi "${REPO}:latest"   # make sure the extension pulls it from the registry

curl -s -X POST http://baseshift.localhost.localstack.cloud:4566/clones \
  -d "{\"name\": \"pr-123\", \"image\": \"${REPO}:latest\", \"env\": {\"POSTGRES_HOST_AUTH_METHOD\": \"trust\"}}"
curl -s http://baseshift.localhost.localstack.cloud:4566/clones/pr-123   # wait for "status": "running", "hostPort": 15432

psql -h localhost -p 15432 -U postgres -c "SELECT 'hello from pr-123'"

# 5. Stop the second clone, and clean up
curl -s -X DELETE http://baseshift.localhost.localstack.cloud:4566/clones/pr-123
lstk stop

Note: use "${REPO}:latest" with braces as above - in zsh, $REPO:latest is interpreted as the :l modifier.

Optional: the end-to-end demo (RDS source with PII → masked snapshot in ECR → clone → S3 → Snowflake). Requires a license that includes the Snowflake emulator, and runs in the Snowflake emulator image, which also provides RDS, ECR and S3:

LOCALSTACK_EXTENSION_AUTO_INSTALL="localstack-baseshift @ https://github.com/localstack/localstack-extensions/archive/refs/heads/add-baseshift-extension.tar.gz#subdirectory=baseshift" \
  lstk start --type snowflake

mkdir baseshift-demo && cd baseshift-demo
curl -sO https://raw.githubusercontent.com/localstack/localstack-extensions/add-baseshift-extension/baseshift/demo/demo.py
curl -sO https://raw.githubusercontent.com/localstack/localstack-extensions/add-baseshift-extension/baseshift/demo/requirements.txt
python3 -m venv .venv && .venv/bin/pip install -q -r requirements.txt
.venv/bin/python demo.py all   # or step by step: source, snapshot, clone, pipeline, compare

lstk stop

The last step prints the same customer record at every stage (raw in RDS, masked in the clone and in Snowflake), and checks that no raw PII leaked downstream.

Note: lstk start --type snowflake records snowflake as the emulator type in your lstk config. Use lstk start --type aws next time to switch back.