Fix indefinite parent hang when child dies mid-handshake - #5391
Closed
nebojsaj1726 wants to merge 1 commit into
Closed
nebojsaj1726 wants to merge 1 commit into
nebojsaj1726 wants to merge 1 commit into
Conversation
If the container's init process dies mid-handshake (e.g. killed by a seccomp filter) while some other reference to its end of the sync socket is still held open elsewhere, the parent can block forever in recvfrom(), since the kernel never delivers EOF on a socket with a live reference. Add waitForSyncReady, which polls the sync socket alongside the child's own /proc state before each read in parseSync, so the parent can detect the child's death directly instead of relying on the socket's own EOF behavior. If the child is confirmed dead without its socket ever becoming ready, parseSync now returns a clear error instead of hanging or silently reporting success. Fixes opencontainers#5087 Signed-off-by: Nebojsa Jacovic <nebojsa.jacovic@gmail.com>
Contributor
|
The description and the patch is somewhat misleading. What happens (I believe) is this: #5087 (comment) |
Author
|
Thanks - I've read issue comment and now it's clearer. That's a different mechanism than what I described (leaked fd reference) - I'll update the description. So, the fix itself doesn't need to be changed - waitForSyncReady already checks for Zombie state via system.Stat, same as waitForFifoReadyPolling, and both the test and manual repro confirm it resolves the hang. Please let me know if I'm missing something. |
Contributor
|
Closing in favor of #5431, thank you @nebojsaj1726 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5087.
When the container's init process is killed by a seccomp filter mid-handshake, only the thread that made the offending syscall is terminated (SCMP_ACT_KILL / SECCOMP_RET_KILL_THREAD kills a single thread, not the whole process - see seccomp(2)). Since runc init is a Go binary, other runtime threads (GC, sysmon, etc.) keep running regardless, so the process can never fully exit - the kernel never delivers EOF on the sync socket, and the parent blocks forever in recvfrom().
This adds waitForSyncReady, which polls the sync socket alongside the child's own /proc state before each read in parseSync, so the parent can detect the child's death directly (via Zombie state) rather than relying on the socket's own EOF behavior - the same approach waitForFifoReadyPolling already uses for the exec-fifo handoff. If the child is confirmed dead without its socket ever becoming genuinely ready, parseSync now returns a clear error instead of hanging or silently reporting success.
Verified two ways:
What's not yet done / open questions for maintainers:
Opening as a draft to get feedback on the approach before finishing this up.