Never read a tree the RPC peer lost as a deletion - #8776
Merged
Merged
Conversation
A source file was deleted from a repository when the peers' object tables drifted apart mid-run, because every way the transport can fail to hand a tree back looked, to the scheduler, like the recipe having deleted it: - `RewriteRpc.getObject` returned null on a NO_CHANGE answer for an id this side holds no baseline for, and `RecipeRunCycle.flushBatch` returned that null as the file's new state even though no batch result reported a deletion. - The Python facade's `_hub_pull_child_edit` forgot a file whenever the child answered its pull with DELETE, which the child also answers for an id it no longer holds; the host's next GetObject then saw DELETE. NO_CHANGE without a baseline is now an error, a batch whose results carry no deletion fails the file instead of dropping it when the remote cannot produce the tree, and the facade only lets a pull delete a file on the single-Visit path, where the child has no other way to report one; a batch reports its deletions in its results and is dropped from the hub without a pull.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
tornado/template.py,src/textual/widget.py,xonsh/built_ins.py). Running every leaf recipe of the composite directly over the same files returns a tree for each of them; none returnsNone. The run that deleted them was also full of ref-table drift between the peers ("Received a reference to an object that was not previously sent" on the Java side, "Received reference to unknown object" on the Python side), and re-running the same recipe over the same LST with the facade ref-table fix from Fix the Python RPC facade's ref-table asymmetries #8768 produced no errors and no deletions.Why a lost tree became a deletion
Every way the transport can fail to hand a tree back looked, to the scheduler, like the recipe having deleted it:
RewriteRpc.getObjectanswerednullwhen the remote sentNO_CHANGEfor an id this side holds no baseline for (a failed receive drops the baseline; the remote still records the object as delivered).RecipeRunCycle.flushBatchthen returned thatnullas the file's new state, although no batch result had reported a deletion, and the file was removed._hub_pull_child_editforgot a file whenever the child answered its pull withDELETE. A child answersDELETEboth for a file its visitor deleted and for an id it no longer holds, and once the facade forgets the file the host's nextGetObjectseesDELETEtoo.Fix
RewriteRpc.getObject:NO_CHANGEwithout a baseline is anIllegalStateException, nevernull(RpcReceiveQueue.peekadded to look at the head state).RecipeRunCycle.flushBatch: when a batch's results carry no deletion but the remote cannot produce the modified tree, the file is marked with the error through the same path a failedBatchVisittakes, instead of being dropped.hub_drop);_hub_pull_child_editrefuses aDELETEanswer unless the caller allows it, which only the single-Visitpath does, since there the child has no other way to report a deletion.Tests
RewriteRpcTest.getObjectRejectsNoChangeWithoutABaselinetest_facade.py:test_batch_visit_drops_a_deleted_file_instead_of_pulling_it,test_hub_pull_refuses_a_child_that_lost_the_tree; the existing single-Visit deletion test now states that contract.