isthmus: literal nullability is dropped converting to Calcite, producing non-conformant plans

[nielspardon]

A Substrait literal's nullable flag does not survive a trip through Calcite. Most literals come back non-nullable, which is a silent type change, and in a nested container it turns a conformant input plan into a non-conformant one that substrait-java then rejects.

Verified on main at 33e2f6a.

Minimal reproduction

Round-tripping a single nullable literal through SubstraitToCalcite and back (Project over a namedScan, remapped so the projection survives):

input	comes back as
ExpressionCreator.i32(true, 5)	I32Literal{nullable=false, value=5}
ExpressionCreator.bool(true, true)	BoolLiteral{nullable=false, value=true}
ExpressionCreator.fp64(true, 1.5)	FP64Literal{nullable=false, value=1.5}
ExpressionCreator.string(true, "x")	Cast{type=Str{nullable=true}, input=StrLiteral{nullable=false, value=x}}

Only the character path keeps the nullable type, and it does so by picking up a Cast wrapper rather than staying a nullable literal.

Why it matters: nested containers stop round-tripping

Expression.NestedList and Expression.NestedMap require homogeneously-typed elements, matching the spec (Nested.List is "A homogeneously-typed list of one or more expressions"). Calcite drops nullability from literals but preserves it on everything else, so a container that was homogeneous on input comes back mixed — and the resulting plan is rejected:

// both values are i32? on input, so this plan is conformant
Expression.NestedList.builder()
    .addValues(ExpressionCreator.i32(true, 5))    // literal      -> returns as i32
    .addValues(sb.fieldReference(table, 0))       // nullable i32 -> returns as i32?
    .build();

• NestedList → AssertionError: All values in NestedList must have the same type
• NestedMap → IllegalArgumentException: All values in a NestedMap must have the same type
• NestedStruct → no error (structs are heterogeneous by design), but the round trip is not identity: the literal field silently loses its nullability

The nested-list case reproduces on main today. The map and struct cases become reachable with #1062 / #1063, which add NestedMap and the Calcite conversions for nested structs and maps — the underlying defect is the same and predates them.

Note the inconsistency in how the two containers reject it: NestedList.check() uses assert, so outside a -ea JVM it does not fire at all and getType() later reads values().get(0) instead. See #1047 / #1058.

Root cause

ExpressionRexConverter's literal visitors hand the nullable type to RexBuilder.makeLiteral(value, type), but Calcite gives every non-null literal a NOT NULL type, so the nullability is discarded at that point:

// isthmus/src/main/java/io/substrait/isthmus/expression/ExpressionRexConverter.java:204-207
public RexNode visit(Expression.I32Literal expr, Context context) {
  return rexBuilder.makeLiteral(expr.value(), typeConverter.toCalcite(typeFactory, expr.getType()));
}

Coming back, LiteralConverter faithfully reads the nullability off the RexLiteral's type (LiteralConverter.java:110-124), which by then is non-nullable. Nothing on the return path is wrong; the information is already gone.

Possible directions

1. Represent nullable literals the way Calcite does — wrap them in a CAST to the nullable type on the way out, which is what already happens incidentally for CHAR/VARCHAR, and unwrap that shape on the way back so it does not surface as a Expression.Cast. Fixes the root cause for every literal, but touches every literal visitor and needs care to avoid a cast appearing in plans that previously had none.
2. Compare element types ignoring nullability in NestedList/NestedMap, deriving the container's element type as the least-restrictive of its entries. Much smaller, and it stops the hard failure, but it relaxes a check that mirrors the spec and leaves the underlying nullability loss in place.

(1) is the real fix; (2) would be a targeted mitigation for the container case if (1) is too broad to take on now.