fix(webinar): filter hash tree sync response to requested dataset - #5136
Open
WeijuanShao wants to merge 3 commits into
Open
fix(webinar): filter hash tree sync response to requested dataset#5136WeijuanShao wants to merge 3 commits into
WeijuanShao wants to merge 3 commits into
Conversation
A sync request is scoped to a single dataset, but Locus may include additional datasets in the response (e.g. "unjoined" shares core Locus elements with "main" and advances to the same version). Processing the response as-is applied the shared elements once per dataset, producing duplicate LOCUS objects and triggering the LOCUS_HASH_TREE_UNSUPPORTED_OPERATION metric. Filter the sync response down to the requested dataset before processing.
|
This pull request is automatically being deployed by Amplify Hosting (learn more). |
xinhyao
approved these changes
Jul 31, 2026
marcin-bazyl
approved these changes
Aug 4, 2026
| * @returns {HashTreeMessage} a new message containing only the requested dataset | ||
| */ | ||
| private filterMessageToDataSet(message: HashTreeMessage, dataSetName: string): HashTreeMessage { | ||
| return { |
Collaborator
There was a problem hiding this comment.
this is good, but I think it would be useful to add a log that prints what data we are filtering out (dataset name, element types, ids, versions)
github-actions
Bot
removed
the
validated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A sync request is scoped to a single dataset, but Locus may include additional datasets in the response (e.g. "unjoined" shares core Locus elements with "main" and advances to the same version). Processing the response as-is applied the shared elements once per dataset, producing duplicate LOCUS objects and triggering the LOCUS_HASH_TREE_UNSUPPORTED_OPERATION metric.
Filter the sync response down to the requested dataset before processing.
COMPLETES # https://jira-eng-gpk2.cisco.com/jira/browse/SPARK-841547
This pull request addresses
filter the sync response data
by making the following changes
When we send a sync request for the main dataset, the response may contain both the main and unjoin datasets. To ensure the data is processed correctly, we should first filter the returned datasets based on the requested dataset, and then process only the filtered results.
Change Type
The following scenarios were tested
< ENUMERATE TESTS PERFORMED, WHETHER MANUAL OR AUTOMATED >
The GAI Coding Policy And Copyright Annotation Best Practices
I certified that
Make sure to have followed the contributing guidelines before submitting.