sql: Revamp LIR-pinning design doc to reflect new plans

[mgree]

With LIR stabilized, plans have shifted slightly: all plans should be pinned by default.

Rendered document.

[antiguru]

I think we need to iterate on the COPY CLUSTER concept. As it stands it's underspecified, and some questions need to be resolved first.

What COPY CLUSTER seems to do is to create a set of objects using the same plans, but with new names. We could turn this around. Instead of trying to rename for the user, we let the user express what they like to achieve:

CREATE CLUSTER new ...;
CREATE INDEX foo_new IN CLUSTER new COPY foo;
CREATE MATERIALIZED VIEW mv1_new IN CLUSTER new COPY mv1;
CREATE MATERIALIZED VIEW mv2_new IN CLUSTER new COPY mv2;

Under the hood, this would desugar to the full CREATE statement, and not retain a reference the ancestor. It would allow the materialized view to pick up the indexes, but mv2 would still depend on mv1 if it did so. A very hand-wavy REPLACE mv1 WITH mv1_new syntax might help, but getting this right is work, and might hit blockers:

CREATE MATERIALIZED VIEW mv2_new IN CLUSTER new COPY mv2 REPLACE (mv1 WITH mv1_new, ...);

Asking a rhetorical question: What is the goal of this work? One interpretation could be to turn optimized artifacts into first-class citizens, although this might be hard to reconcile with SQL syntax.

[antiguru]

A COPY CLUSTER command has some wrinkles we need to explore. Unfortunately, clusters are not schemas, meaning that they're not a namespace. This raises questions what is supposed to happen to the objects a cluster hosts:

• Indexes can be renamed, and everything would still work as expected, as we're not naming indexes explicitly.
• Materialized views are more challenging. Should we use replacements automatically, so that the new cluster would continue writing the same shards, or would we create new shards? I think it's important to resolve this early because we likely have downstream dependents on the materialized view, and we need to come up with a pattern to move them over to a new definition.
• What happens to sources and sinks? Can we exclude them from this design?