[fix](topn) Skip TopN lazy materialization when light_schema_change=false#64441
Open
HappenLee wants to merge 1 commit into
Open
[fix](topn) Skip TopN lazy materialization when light_schema_change=false#64441HappenLee wants to merge 1 commit into
HappenLee wants to merge 1 commit into
Conversation
…alse TopN lazy materialization relies on BE adding GLOBAL_ROWID_COL to the tablet schema via columns_desc. When light_schema_change=false, table columns have col_unique_id=-1, causing BE to skip the schema rebuild from columns_desc, so GLOBAL_ROWID_COL is never added and the scan fails with "field name is invalid". Skip the optimization at FE for tables with light_schema_change=false to avoid the BE error. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
HappenLee
requested review from
924060929,
englefly,
morrySnow and
starocean999
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
morrySnow
reviewed
Jun 12, 2026
| StatementContext threadStatementContext = StatementScopeIdGenerator.getStatementContext(); | ||
| for (Relation relation : relationToLazySlotMap.keySet()) { | ||
| // TopN lazy materialization relies on BE adding a GLOBAL_ROWID_COL to the | ||
| // tablet schema. When light_schema_change=false, the table columns have |
Contributor
There was a problem hiding this comment.
When a table has light_schema_change=false, does that mean lazy materialization cannot be used for any tables involved in the query?
Contributor
Author
There was a problem hiding this comment.
@morrySnow Light schema change was introduced in doris version 1.2.
Legacy tables are rare, and this is not the default behavior. Simply avoid this optimization.
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29192 ms |
Contributor
TPC-DS: Total hot run time: 169601 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
TopN lazy materialization relies on BE adding
GLOBAL_ROWID_COLto the tablet schema throughcolumns_desc. Whenlight_schema_change=false, table columns havecol_unique_id=-1, which causes BE to skip rebuilding the schema fromcolumns_desc. As a result,GLOBAL_ROWID_COLis not added and the scan can fail withfield name is invalid.This PR skips TopN lazy materialization for Olap tables with
light_schema_change=false, so FE keeps the regular scan output and avoids generating a plan that requires the missing row id column.It also adds FE unit test coverage for:
light_schema_change=falseIssue Number: close #xxx
Related PR: #xxx
Release note
None
Check List (For Author)
sh run-fe-ut.sh --run org.apache.doris.nereids.postprocess.TopnLazyMaterializeTestBehavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)