FileProvider: Use a per-process unique temporary lodev file name - #512
Open
mateo-moon wants to merge 1 commit into
Open
FileProvider: Use a per-process unique temporary lodev file name#512mateo-moon wants to merge 1 commit into
mateo-moon wants to merge 1 commit into
Conversation
Multiple satellite processes may share the same /var/lib/linstor.d directory: the special satellites (e.g. EBS targets) that the controller spawns all run within the controller's container. Their concurrent device manager cycles race on the shared loop_device_mapping.tmp: one process renames the file away and the others' rename then fails with a NoSuchFileException, which aborts their entire device manager cycle. For EBS target satellites this leaves freshly dispatched resources undeployed (stuck Unknown) even though the corresponding EBS volume may already have been created. Using a per-process unique temporary file name keeps the atomic-rename pattern while making concurrent cycles of co-located satellites independent of each other.
This was referenced Jul 20, 2026
Author
|
E2E-validated live on a 3-AZ Kubernetes cluster (arm64/Nitro, v1.34.1 base + all four of our EBS fix PRs): full validation report in #506 (comment) (latest comment). Highlights relevant to this PR are listed there; the four fixes together make native EBS work end-to-end on latest. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #511.
Problem
Special satellites spawned by the controller (e.g.
EBS_TARGETnodes) all run within the controller's container and share/var/lib/linstor.d. Every device-manager cycle ends inFileProvider.clearCache(), which writesloop_device_mapping.tmpand atomically renames it — even with zero loop devices. Concurrent cycles of co-located satellites race on the shared temp file: one renames it away, the other's rename throwsNoSuchFileException, and that satellite's whole device-manager cycle aborts. With three per-AZ EBS targets this fires regularly and leaves freshly dispatched resources stuckUnknown(live evidence in #511).Fix
Use a per-process unique temporary file name (
loop_device_mapping.tmp.<pid>), keeping the atomic-rename pattern while making co-located satellites' cycles independent. Behavior for the normal one-satellite-per-host case is unchanged.Testing
:satellite:compileJava+:satellite:checkstyleMainclean (no new findings).