MergeJoin.getLeftTypes() is a verbatim copy of getRightTypes(): it reads getRight() instead of getLeft(), and switches on the left-oriented join types instead of the right-oriented ones. As a result MergeJoin.getRecordType() reports the wrong arity as well as the wrong types — the left input's columns are missing entirely and the right input's are duplicated.
HashJoin, which is otherwise structurally identical, has the correct version of both methods.
Observed
With left = one column i64 and right = two columns string, fp64, the two relations disagree for every join type:
| join type |
MergeJoin.getRecordType() |
HashJoin.getRecordType() (correct) |
INNER |
[string, fp64, string, fp64] |
[i64, string, fp64] |
OUTER |
[string?, fp64?, string?, fp64?] |
[i64?, string?, fp64?] |
LEFT |
[string?, fp64?, string?, fp64?] |
[i64, string?, fp64?] |
RIGHT |
[string, fp64, string, fp64] |
[i64?, string, fp64] |
LEFT_SEMI |
[] |
[i64] |
LEFT_ANTI |
[] |
[i64] |
RIGHT_SEMI |
[string, fp64, string, fp64] |
[string, fp64] |
RIGHT_ANTI |
[string, fp64, string, fp64] |
[string, fp64] |
Note LEFT_SEMI and LEFT_ANTI produce an empty struct, because both halves take the Stream.empty() branch.
Cause
MergeJoin.getLeftTypes() and MergeJoin.getRightTypes() are byte-for-byte identical:
private Stream<Type> getLeftTypes() {
switch (getJoinType()) {
case LEFT: // should be RIGHT
case OUTER:
return getRight().getRecordType()... // should be getLeft()
case LEFT_ANTI: // should be RIGHT_ANTI
case LEFT_SEMI: // should be RIGHT_SEMI
return Stream.empty();
default:
return getRight().getRecordType()... // should be getLeft()
}
}
HashJoin.getLeftTypes() is the shape MergeJoin.getLeftTypes() should have: pad the left for RIGHT/OUTER, drop it for RIGHT_SEMI/RIGHT_ANTI, and read getLeft() throughout.
Impact
Anything that resolves a field position against a MergeJoin's output is wrong, not merely imprecisely typed, because the field count differs:
- a field reference in a parent relation's expressions, whose offset now indexes into the wrong column;
Project/Aggregate record types derived above a merge join;
- proto → POJO conversion of a relation above a merge join, which types root references from the child's record type.
The physical join relations appear to have no round-trip or record-type test coverage, which is why the copy-paste survived.
Suggested fix
Give MergeJoin HashJoin's getLeftTypes(), and add a record-type test for both relations covering all nine join types — the table above works as the expectation for HashJoin. Deduplicating the two implementations onto a shared helper would stop them diverging again; note MergeJoin.JoinType and HashJoin.JoinType are separate enums with the same nine constants.
Found while reviewing #1061.
MergeJoin.getLeftTypes()is a verbatim copy ofgetRightTypes(): it readsgetRight()instead ofgetLeft(), and switches on the left-oriented join types instead of the right-oriented ones. As a resultMergeJoin.getRecordType()reports the wrong arity as well as the wrong types — the left input's columns are missing entirely and the right input's are duplicated.HashJoin, which is otherwise structurally identical, has the correct version of both methods.Observed
With
left= one columni64andright= two columnsstring, fp64, the two relations disagree for every join type:MergeJoin.getRecordType()HashJoin.getRecordType()(correct)INNER[string, fp64, string, fp64][i64, string, fp64]OUTER[string?, fp64?, string?, fp64?][i64?, string?, fp64?]LEFT[string?, fp64?, string?, fp64?][i64, string?, fp64?]RIGHT[string, fp64, string, fp64][i64?, string, fp64]LEFT_SEMI[][i64]LEFT_ANTI[][i64]RIGHT_SEMI[string, fp64, string, fp64][string, fp64]RIGHT_ANTI[string, fp64, string, fp64][string, fp64]Note
LEFT_SEMIandLEFT_ANTIproduce an empty struct, because both halves take theStream.empty()branch.Cause
MergeJoin.getLeftTypes()andMergeJoin.getRightTypes()are byte-for-byte identical:HashJoin.getLeftTypes()is the shapeMergeJoin.getLeftTypes()should have: pad the left forRIGHT/OUTER, drop it forRIGHT_SEMI/RIGHT_ANTI, and readgetLeft()throughout.Impact
Anything that resolves a field position against a
MergeJoin's output is wrong, not merely imprecisely typed, because the field count differs:Project/Aggregaterecord types derived above a merge join;The physical join relations appear to have no round-trip or record-type test coverage, which is why the copy-paste survived.
Suggested fix
Give
MergeJoinHashJoin'sgetLeftTypes(), and add a record-type test for both relations covering all nine join types — the table above works as the expectation forHashJoin. Deduplicating the two implementations onto a shared helper would stop them diverging again; noteMergeJoin.JoinTypeandHashJoin.JoinTypeare separate enums with the same nine constants.Found while reviewing #1061.