Antalya 26.5: DataLakeCatalog: avoid full catalog read for UNKNOWN_TABLE typo hints#1938
Open
zvonand wants to merge 2 commits into
Open
Antalya 26.5: DataLakeCatalog: avoid full catalog read for UNKNOWN_TABLE typo hints#1938zvonand wants to merge 2 commits into
UNKNOWN_TABLE typo hints#1938zvonand wants to merge 2 commits into
Conversation
…next commit) --- Original cherry-pick message follows: Merge pull request #1675 from Altinity/feature/antalya-26.3/ClickHouse-ClickHouse-pr-100452 Antalya 26.3: DataLakeCatalog: avoid full catalog read for `UNKNOWN_TABLE` typo hints # Conflicts: # src/Interpreters/DatabaseCatalog.cpp
The source PR's change (gate getAllRegisteredNames on show_data_lake_catalogs_in_system_tables via isDatalakeCatalog) is already present on antalya-26.5 in generalized superset form: commit d5f46f6 (backport ClickHouse#104416) renamed isDatalakeCatalog -> isRemoteDatabase and show_data_lake_catalogs_in_system_tables -> show_remote_databases_in_system_tables, broadening the skip to all remote databases (data lake catalogs, MySQL, PostgreSQL). The bucket-2 token-swap of the source PR's lines lands exactly on the existing antalya-26.5 code, so the conflict resolves to "ours". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zvonand
added
releasy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Avoid scanning the whole remote data lake catalog for “Maybe you meant …” table hints when
show_data_lake_catalogs_in_system_tablesis disabled (ClickHouse#100452 by @alsugiliazova).Cherry-picked from ClickHouse#100452.
--- (#1675 by @zvonand).
CI/CD Options
Exclude tests:
Regression jobs to run:
Cherry-picked from #1675.
Documentation entry for user-facing changes
When
show_data_lake_catalogs_in_system_tables = 0, the server must not implicitly scan the whole remote data lake catalog.Previously, building the “Maybe you meant …” hint for a missing table in a
DataLakeCatalogdatabase still calledgetAllTableNames()→DatabaseDataLake::getTablesIterator(), which lists the entire catalog and loads per-table metadata—heavy work and can OOM on large catalogs, only to enrich an error message.This change makes
TableNameHints::getAllRegisteredNames()return an empty name list for data lake catalogs when that setting is off, so hint generation does not trigger a full catalog listing.Query examples
Drop non-existent table:
Previously (undesired): server could answer with
UNKNOWN_TABLEand a “Maybe you meant …” suggestion after scanning catalog names:After the fix:
UNKNOWN_TABLEwithout hint when the setting is0:Optional follow-up
If local hints are skipped for data lake + setting
0,getHintForTablecan still fall back togetExtendedHintForTable, which only scans non–data-lake databases. In edge cases a suggestion could point at another database’s table. If that is undesirable, a follow-up could skip extended hints under the same condition as local enumeration.