Skip to content

MergeJoin.getRecordType() reports the right input twice and omits the left #1067

Description

@nielspardon

MergeJoin.getLeftTypes() is a verbatim copy of getRightTypes(): it reads getRight() instead of getLeft(), and switches on the left-oriented join types instead of the right-oriented ones. As a result MergeJoin.getRecordType() reports the wrong arity as well as the wrong types — the left input's columns are missing entirely and the right input's are duplicated.

HashJoin, which is otherwise structurally identical, has the correct version of both methods.

Observed

With left = one column i64 and right = two columns string, fp64, the two relations disagree for every join type:

join type MergeJoin.getRecordType() HashJoin.getRecordType() (correct)
INNER [string, fp64, string, fp64] [i64, string, fp64]
OUTER [string?, fp64?, string?, fp64?] [i64?, string?, fp64?]
LEFT [string?, fp64?, string?, fp64?] [i64, string?, fp64?]
RIGHT [string, fp64, string, fp64] [i64?, string, fp64]
LEFT_SEMI [] [i64]
LEFT_ANTI [] [i64]
RIGHT_SEMI [string, fp64, string, fp64] [string, fp64]
RIGHT_ANTI [string, fp64, string, fp64] [string, fp64]

Note LEFT_SEMI and LEFT_ANTI produce an empty struct, because both halves take the Stream.empty() branch.

Cause

MergeJoin.getLeftTypes() and MergeJoin.getRightTypes() are byte-for-byte identical:

private Stream<Type> getLeftTypes() {
  switch (getJoinType()) {
    case LEFT:                 // should be RIGHT
    case OUTER:
      return getRight().getRecordType()...  // should be getLeft()
    case LEFT_ANTI:            // should be RIGHT_ANTI
    case LEFT_SEMI:            // should be RIGHT_SEMI
      return Stream.empty();
    default:
      return getRight().getRecordType()...  // should be getLeft()
  }
}

HashJoin.getLeftTypes() is the shape MergeJoin.getLeftTypes() should have: pad the left for RIGHT/OUTER, drop it for RIGHT_SEMI/RIGHT_ANTI, and read getLeft() throughout.

Impact

Anything that resolves a field position against a MergeJoin's output is wrong, not merely imprecisely typed, because the field count differs:

  • a field reference in a parent relation's expressions, whose offset now indexes into the wrong column;
  • Project/Aggregate record types derived above a merge join;
  • proto → POJO conversion of a relation above a merge join, which types root references from the child's record type.

The physical join relations appear to have no round-trip or record-type test coverage, which is why the copy-paste survived.

Suggested fix

Give MergeJoin HashJoin's getLeftTypes(), and add a record-type test for both relations covering all nine join types — the table above works as the expectation for HashJoin. Deduplicating the two implementations onto a shared helper would stop them diverging again; note MergeJoin.JoinType and HashJoin.JoinType are separate enums with the same nine constants.

Found while reviewing #1061.

Metadata

Metadata

Assignees

No one assigned

    Labels

    corePull requests that update java code

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions