improve observability and readiness timeout for buildkitd startup#112
Draft
ajwerner wants to merge 1 commit into
Draft
improve observability and readiness timeout for buildkitd startup#112ajwerner wants to merge 1 commit into
ajwerner wants to merge 1 commit into
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the supabase/supabox variance investigation: a job that hit the buildkitd worker-readiness timeout fell back to a cold local builder (13m29s build) even though the daemon was still alive and serving 20 minutes later in the post step — the readiness window was simply too short, and the failure left no evidence and no backend report.
Changes (all zero-customer-change; picked up via the moving
v1tag):buildkitdTimeoutMs: 30s → 120s. Slow worker init (loading a large sticky-disk cache) previously triggered fallback; a cold local build + the skipped sticky-disk commit that poisons downstream runs costs far more than waiting longer.startAndConfigureBuildkitd:/tmp/buildkitd.loginto the job log (previously this only happened in the post step when the process was gone, so slow-but-alive daemons left no evidence)reportBuildPushActionFailure("BUILDER_STARTUP", ...)so these fallbacks are visible to the backend (previously only fork failures were reported)BPA_BUILDKITD_READY_DURATION_MSis now reported in afinally, so timeout cases show up in the metric instead of only successes.logBuildkitdLogTailinsetup_builder.ts(replaces the post-step-onlylogBuildkitdCrashLogsinmain.ts).Link to Devin session: https://app.devin.ai/sessions/f9e26ad52787476784fe0848cd40551d
Requested by: @ajwerner
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled. (Staging)