feat(isthmus): convert nested struct and map expressions to and from Calcite

[nielspardon]

Stacked on #1062, which adds the NestedMap POJO and the proto import this builds on.

Of the three nested expression kinds, only nested lists survived a trip through Calcite. In the Substrait to Calcite direction ExpressionRexConverter had no case for NestedStruct or NestedMap, so both reached visitFallback and threw UnsupportedOperationException. Coming back the other way, CallConverters.ROW rejected any ROW whose fields were not all literals with "ROW operands must be literals.", and the map value constructor cast every operand to Expression.Literal — so SELECT ROW(a + 1, b) and SELECT MAP['key', a + 1] failed with an IllegalArgumentException and a ClassCastException respectively.

Nested structs now become Calcite ROW calls and nested maps become MAP_VALUE_CONSTRUCTOR calls, each carrying its own nullability in the call type. In reverse, a ROW or map constructor whose operands are not all literals falls back to a NestedStruct or NestedMap — the same treatment array value constructors already gave nested lists — while the all-literal cases still collapse to a StructLiteral or MapLiteral.

Nullability has to survive that collapse

nullable on a Substrait literal marks the literal's type as nullable, not the value as null — a value that is actually null is a NullLiteral. The ROW and map converters were hardcoding nullable=false, so a nullable struct or map of literals silently came back non-nullable. They now take it from the call type, which is what the array value constructor already did for lists.

The same gap existed in the other direction: the list and map literal visitors let Calcite infer the container type from the elements, which discards the container's own nullability, so they now pass the converted type like their struct and nested counterparts. All five paths are pinned by tests.

Struct-level nullability is deliberately kept inside StructLiteral rather than routed out to NestedStruct. Routing looks tidier and gives an exact round trip, but it breaks the user-defined type struct encoding, which recognises its payload by it being a StructLiteral: a UDT literal with a nullable struct field produces a nullable inner ROW, that ROW becomes a NestedStruct, the enclosing UDT ROW then fails its all-literals test too, and REINTERPRET can no longer extract the literal — IllegalArgumentException: Unable to convert call Reinterpret(struct<struct<i32>?, i32>). structEncodedUdtWithNullableStructFieldRoundTrip locks that shape down.

One consequence to be aware of: Calcite makes every field of a nullable record type nullable, so a nullable struct widens its field types on the way through. That is Calcite's representation of a nullable row, not something this change can avoid.

Known limitation, pre-existing

A nested value that mixes literal and non-literal entries of the same declared nullable type comes back with mismatched entry types, because Calcite gives literals NOT NULL types while preserving nullability on everything else. NestedMap then rejects it (All values in a NestedMap must have the same type). This is not new here — NestedList has behaved this way since nested lists landed, and it needs literals to keep their nullability across the Calcite boundary to fix properly. Filed separately as #1066 rather than widening this PR.

Incidental fixes in the map converter

The literal map is built in operand order rather than HashMap order, so the emitted key_values no longer depend on hash iteration order, and an odd operand count throws IllegalArgumentException instead of relying on an assert that is disabled outside test JVMs. That last one overlaps #1058 — whichever lands second can drop its copy.

Closes #375

🤖 Generated with AI