Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ This repository serves as the home for everything related to how we collaborate,
This repo contains:

- [Design proposals](./design-proposals/README.md) — cross-cutting and architectural changes
- [Decision records](./decisions/README.md) — what we decided and why, once a question is settled
- [The community meeting](./community_meeting.md) — when it happens, how to join, how to add a topic
- [The meeting archive](./meetings/README.md) — every past meeting, its topics and its recording
- Community resources and helpful links (below)
Expand All @@ -26,6 +27,7 @@ Cozystack work is split across two repositories. This repo is for **how we decid
| A **bug** or a **concrete, scoped feature** | [cozystack/cozystack issues](https://github.com/cozystack/cozystack/issues/new/choose) |
| A **usage question** or an early idea | [cozystack/cozystack Discussions](https://github.com/cozystack/cozystack/discussions) |
| A **cross-cutting / architectural change** — affects multiple components or APIs, or needs a decision before code | A [design proposal](./design-proposals/README.md) (a PR) **here** |
| A **decision already made** whose reasoning would otherwise survive only in a pull-request thread | A [decision record](./decisions/README.md) (a PR) **here** |
| **Governance, process, or community** matters | An issue **here** |

## Community Resources
Expand Down
44 changes: 44 additions & 0 deletions decisions/0001-computeplane-ships-as-an-operator-owned-module.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# 0001. ComputePlane ships as an operator-owned module, not as preset fields on `kind: Kubernetes`

- **Number:** `0001`
- **Date:** `2026-07-18`
- **Status:** Accepted
- **Deciders:** `@kvaps, @myasnikovdaniil`
- **Proposal:** [`design-proposals/compute-plane/README.md`](../design-proposals/compute-plane/README.md)
- **Implemented in:** not yet — see the proposal's Rollout section

## Context

The first ComputePlane revision ([#17](https://github.com/cozystack/community/pull/17), merged 2026-06-30) delivered a hardened, operator-controlled Kubernetes cluster for running code-executing catalog apps — notebooks, workflow code nodes, plugin systems — behind a per-VM kernel boundary instead of as shared-kernel pods on the management cluster. It was shaped as a tenant module selected by a single string (`computePlane: "<profile>"`), with the cluster's HelmRelease rendered inline in `apps/tenant/templates`.

Two reviews converged on the same objection ([#26](https://github.com/cozystack/community/issues/26), and Timofei Larkin on [#17](https://github.com/cozystack/community/pull/17)): the cluster a ComputePlane runs on *is* an ordinary managed `kind: Kubernetes` — Kamaji control plane, KubeVirt-VM workers, operator-held kubeconfig — so ComputePlane should not become a distinct kind with a parallel cluster implementation, reconcile loop and RBAC surface.

Revision [#27](https://github.com/cozystack/community/pull/27) answered that by putting the posture directly on the existing app as two orthogonal user-facing preset fields: `isolationProfile` (`standard` | `sandbox` | `cluster-meshed`) × `componentProfile` (`minimal` | `standard-addons` | `gpu`). A ComputePlane would then be nothing more than `kind: Kubernetes` with `isolationProfile: sandbox`.

The isolation mechanism itself was never in question at any point — one-way remote Flux apply via `HelmRelease.spec.kubeConfig.secretRef`, untrusted code behind a disposable guest kernel, single-tenant, scoped egress. Only the delivery surface was contested.

## Decision

ComputePlane ships as a Cozystack-owned **tenant module** (`packages/extra/computeplane`) that provisions a hardened `kind: Kubernetes` by wrapping the unchanged `apps/kubernetes` chart with operator-fixed values, sourced through the existing PackageSource source-only-chart mechanism that NATS and SeaweedFS already use.

Because a tenant module is registered through an `ApplicationDefinition`, ComputePlane does get its own `apps.cozystack.io` kind, with its own API endpoint and input schema — the same CRD-free way every managed service is registered. So the accurate claim is not "no new kind": it is **no new CRD, no new controller or reconcile path, and no new fields on `kind: Kubernetes`**. Catalog apps route onto the cluster through a `placement: ManagementPlane | ComputePlane` field on `ApplicationDefinition`.

## Why not the alternatives

- **User-facing preset fields on `kind: Kubernetes` ([#27](https://github.com/cozystack/community/pull/27)).** Putting the posture on the app object turns an operator's choice into tenant-editable API surface. The entire value of a sandbox is that the tenant *cannot* weaken it, and any field that configures the hardening is a field that can relax it. It also loads a hardened posture onto the one `ApplicationDefinition` that has to stay tenant-generic.
- **A distinct, heavyweight `kind: ComputePlane` with its own cluster implementation ([#17](https://github.com/cozystack/community/pull/17) as merged).** Duplicates the cluster reconcile and RBAC path, which was the substance of the #26 / #17-review objection. Note this decision does still register a `ComputePlane` kind — it departs from #26's literal "no new kind at all" — but as a thin wrapper over the unchanged app, which is what the objection was actually about.
- **The single-string `computePlane:` module rendered inline in `apps/tenant/templates` (the first revision's implementation).** Right delivery shape, wrong packaging: the cluster HelmRelease was inlined into the tenant chart instead of shipping as a first-class chart with its own PackageSource, the way `extra/seaweedfs` does.
- **Hardening containers in the tenant namespace instead.** Rejected as the primary boundary: hardening does not make container isolation multi-tenant, and it breaks the apps in scope.
- **gVisor or a sandboxed runtime as the primary boundary.** Rejected for incomplete syscall coverage and no blast-radius containment on a kernel panic. Still valid as an *inner* layer for per-task sandboxes inside the cluster, and tracked as a future runtime option rather than dismissed.

## Consequences

- The hardening is tamper-proof by construction. The cluster's values live entirely in an operator-owned chart and the tenant holds no admin kubeconfig, so "withhold admin" and "own the configuration" become one fact rather than two features to keep in sync.
- `kind: Kubernetes` gains no fields and stays tenant-generic. Node-provisioning and addon changes to the app apply to ComputePlane transparently.
- The cost is composability: several differently-hardened postures per tenant now require separate module variants rather than a combination of two fields. The `placement: <named cluster>` path that would give a tenant N sandbox clusters is deferred.
- The `computeplane` release name is load-bearing (proposal, Design §1). Any later move of the chart between directories must preserve it.
- [#39](https://github.com/cozystack/community/pull/39) (folding `extra` into `apps` as declarative `ApplicationDefinition` capabilities) composes with this rather than conflicting: ComputePlane becomes a directory move plus `visibility: module` and `cardinality: {scope: tenant, max: 1}`, and the two-piece structure carries over unchanged.

## Revisit if

[#39](https://github.com/cozystack/community/pull/39) lands and changes how module kinds are registered, or a concrete need appears for several simultaneous, differently-hardened compute clusters per tenant — the case the single-module shape does not serve and the deferred `placement: <named cluster>` path would.
91 changes: 91 additions & 0 deletions decisions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Cozystack Decision Records

This folder holds **decision records**: short, dated notes on architectural decisions the project has actually made, and why.

A [design proposal](../design-proposals/README.md) is intent — what we think we should build, written before the work. A decision record is history — what we settled on, written once the question is closed. Both are useful; the second is the one we have been missing.

## Why a separate folder

Design proposals get edited. When implementation contradicts the design — and it regularly does — the proposal is revised in place, so it ends up reading as though it always said the current thing. The reasoning that changed our minds (*we tried X, hit a concrete constraint, moved to Y*) then survives only in a pull-request diff that nobody will find in a year.

That reasoning is the most expensive thing we produce and the easiest to lose. A decision record is where it goes.

| | Design proposal | Decision record |
|---|---|---|
| Written | before the work, to get agreement | once the question is settled |
| Answers | "should we, and how might we?" | "what did we decide, and why not the alternatives?" |
| Edited after merge | yes, as the design evolves | never — superseded by a new record |
| Length | as long as it needs to be | one page |
| Review | consensus from maintainers | one maintainer checks it for accuracy |

Both have an alternatives section, and they are not the same list. A proposal weighs the options we could imagine *before* building. A record names the option that lost *during* building — frequently the proposal's own original design.

## When to write one

Write a decision record when a future contributor would otherwise have to reconstruct the reasoning from a pull-request thread. In practice:

- Implementation contradicted an accepted proposal and the design changed course.
- Two viable approaches existed and we picked one for reasons that are not visible in the resulting code.
- We hit a constraint that now shapes the design — an upstream limitation, a Kubernetes semantic, a vulnerability class.
- The code enforces a contract it cannot explain: a field is immutable, a release name is load-bearing, an ordering is required.
- We deliberately decided *not* to do something, and the question keeps coming back.

Do not write one for:

- Routine code choices that the diff and the tests already explain.
- Operator-facing how-to — that is user documentation, and it belongs on [the website](https://cozystack.io/docs/).
- The mechanics of the product repository's own workflow (release process, changelog conventions) — those live next to the code in [cozystack/cozystack](https://github.com/cozystack/cozystack).
- A decision that has not been made yet. That is a design proposal, or an open question inside one.

## How to write one

1. Copy [`template.md`](./template.md) to `decisions/NNNN-short-slug.md`, taking the next free number:

```
./decisions/0007-etcd-is-per-cluster-not-per-tenant.md
```

1. **Title the decision, not the topic.** `storageClass is immutable after creation`, not `storageClass immutability`. Someone scanning the folder should learn what we decided from the filename alone.

1. **Keep it to a page.** Link out to the proposal, the code and the pull requests for detail. A record that grows into a second design document will not get read.

1. **Link both ways.** The record links its proposal and the pull requests that implemented it; the proposal's `Decisions` section links back to the record.

1. As with all commits in CNCF projects, sign the commit for the DCO check:

```bash
git commit --signoff
```

Two pull requests can claim the same number. Whoever merges second renumbers on rebase — a file rename and a couple of link fixes.

## Status and immutability

| Status | Meaning |
|---|---|
| `Accepted` | Current. The decision stands. |
| `Superseded by NNNN` | A later record replaced it. The original text stays exactly as written. |
| `Reverted` | We undid it. Say what we do instead. |

Once merged, **the body of an accepted record is not edited.** Fix typos and broken links; change nothing else. If the decision changes, write a new record and mark the old one `Superseded by NNNN`.

This one rule is what makes the folder a history instead of a second set of documents to keep current. A record that gets quietly rewritten is worth no more than the proposal it was meant to supplement.

## Review

A decision record documents a decision that has already been made. Review therefore checks that the record is **accurate** — not whether the reviewer agrees with it. One maintainer's approval is enough, and it should be quick.

If review turns into re-litigating the decision, that is a signal the decision was not actually settled. Close the pull request and open a design proposal or an issue instead.

## Where this sits relative to everything else

| The question you are answering | Where it is answered |
|---|---|
| "Why is it built this way, and not the obvious other way?" | A decision record, here |
| "What should we build?" | A [design proposal](../design-proposals/README.md), here |
| "How do I use it?" | [User documentation](https://cozystack.io/docs/) on the website |
| "How does this repository release, or write changelogs?" | [cozystack/cozystack](https://github.com/cozystack/cozystack), next to the code |

## Inspiration

The format follows [Michael Nygard's architecture decision records](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) and [MADR](https://adr.github.io/madr/), trimmed to the sections we will actually fill in.
59 changes: 59 additions & 0 deletions decisions/template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<!-- Copy this file to decisions/NNNN-short-slug.md, taking the next free number. -->
# NNNN. State the decision here, as a fact

- **Number:** `NNNN`
- **Date:** `YYYY-MM-DD`
- **Status:** Accepted | Superseded by `NNNN` | Reverted
- **Deciders:** `@your-github-handle, @codecider`
- **Proposal:** `design-proposals/<name>/README.md` — or `none`
- **Implemented in:** `cozystack/cozystack#NNNN` — or `not yet`

<!-- Date is when the decision was made, not when it was written up.
Title the decision, not the topic: "storageClass is immutable after
creation", not "storageClass immutability". -->

## Context

<!-- What was true when we decided. The forces in play: the state of the
code, constraints we were under, what had already shipped.

Write it in the past tense and leave it there — this is a snapshot,
and it is not updated later when the world moves on. Give a reader
who was not present enough to see why the decision was reasonable. -->

## Decision

<!-- One paragraph. Imperative and specific: "ComputePlane ships as …".
State what was decided, not the discussion that produced it. -->

## Why not the alternatives

<!-- One short bullet per alternative: what it was, and the concrete reason
it lost. This is the section that keeps the question from being
reopened every six months, so spend your words here.

If an alternative lost on a judgement call rather than a hard fact,
say so — it tells a future reader how firm this decision really is. -->

## Consequences

<!-- What this now costs or constrains, in both directions: what got
easier, and what we have to live with. Any follow-up work it creates.

Be honest about the downsides. A record that lists no costs reads as
advocacy, and gets trusted accordingly. -->

## Revisit if

<!-- The trigger that would justify reopening this: an upstream feature
landing, a scale threshold crossed, a constraint lifted. If there is
no plausible trigger, say "no foreseeable trigger" — that is useful
information too. -->

---

<!--
Format follows Michael Nygard's architecture decision records
(https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions)
and MADR (https://adr.github.io/madr/).
-->
4 changes: 3 additions & 1 deletion design-proposals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,9 @@ Proposals are reviewed in GitHub PRs. Once a proposal receives consensus from co

We understand that real-world implementation may diverge from initial designs. This is normal and expected.

While it's helpful to update the proposal if the divergence is significant, the **codebase and user documentation are the final source of truth**.
Update the proposal when the divergence is significant. When the divergence came from a decision worth remembering — an approach that failed, a constraint you discovered, an alternative you picked instead — also write a [decision record](../decisions/README.md) and link it from the proposal's `Decisions` section. Editing the proposal alone loses the reasoning: the revised text reads as though it always said the current thing, and *why* the design changed course survives only in the pull-request diff.

The **codebase and user documentation remain the final source of truth** for what the system does. The decision records are the source of truth for why it does it that way.

## Inspiration

Expand Down
4 changes: 4 additions & 0 deletions design-proposals/compute-plane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ The capability is generic and intended to live in Cozystack core as a reusable p
- **`design-proposals/kubernetes-nodes-split`** / **`kubernetes-nodes-hybrid-clusters`** (PR #8/#9): the substrate is the existing managed-`kubernetes` app (Kamaji + CAPI/KubeVirt); node-provisioning changes apply transparently.
- **Deferred:** billing/metering of cluster resource and API consumption; secret delivery of managed-service connection strings into sandbox workloads; the per-instance/label granularity of the visibility control (Design §6). (Cross-tenant *sharing* of a cluster is **not** deferred — it is rejected by design; see Non-goals.)

## Decisions

- [0001. ComputePlane ships as an operator-owned module, not as preset fields on `kind: Kubernetes`](../../decisions/0001-computeplane-ships-as-an-operator-owned-module.md) — why the preset-field revision (#27) was set aside.

## Context

Today Cozystack already has every primitive needed *except* the glue that ties them into "deploy this catalog app onto a hardened, operator-controlled `kind: Kubernetes` the tenant does not administer":
Expand Down
12 changes: 12 additions & 0 deletions design-proposals/template.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,18 @@ A reader who stops here should know what you're asking for and roughly why. -->
Link them by repo path or URL. If this proposal must land before or after
another, say so. Omit the section if there are no related proposals. -->

## Decisions

<!-- Leave this empty in the initial PR; fill it in as implementation
proceeds. Link each decision record that came out of building this,
newest first:

- [0001. ComputePlane ships as an operator-owned module](../../decisions/0001-computeplane-ships-as-an-operator-owned-module.md)

Where implementation changed the design, the record is what explains why
— this section is how a reader of the proposal finds that out.
See ../../decisions/README.md. -->

## Context

<!-- Background a reader needs to evaluate the proposal.
Expand Down