fix(go): take a heap snapshot, not a 30-second delta - #130
Merged
Merged
Conversation
Every Go heap profile the collector produced could come back empty:
Type: inuse_space
Duration: 30.02s, Total samples = 0
Showing nodes accounting for 0, 0% of 0 total
The URL was /debug/pprof/heap?seconds=30. On the heap endpoint the seconds
parameter does not mean "sample for this long" - Go samples, waits, samples
again and returns the DIFFERENCE. A service whose heap is steady across the
window deltas to exactly zero, so the profile renders as an empty
flamegraph and the pod's Profiler tab shows nothing.
A heap profile is a snapshot of what the process is holding, so it takes no
duration. Dropping seconds and adding gc=1 - which collects first, so the
snapshot describes live memory rather than whatever the last GC left -
turns the same request into a real profile.
There was a problem hiding this comment.
Code Review
This pull request updates the heap profiling URL in internal/agent/profiler/go_pprof.go to request a snapshot of live memory using ?gc=1 instead of a delta over a duration. Feedback suggests correcting the added comment to accurately describe Go's heap profiling behavior regarding garbage collection and to warn about potential latency spikes caused by forcing a garbage collection on large heaps.
RamanKharchee
approved these changes
Sep 19, 2026
blue4209211
approved these changes
Sep 19, 2026
Merged
4 tasks
mayankpande88
added a commit
to nudgebee/k8s-agent
that referenced
this pull request
Sep 19, 2026
ca8a3b3 asks Go for /debug/pprof/heap?seconds=30, which returns the delta over that window rather than a snapshot - so every heap profile of a service with a steady heap came back with Total samples = 0 and rendered as an empty flamegraph. 2ca753a is the first tag with nudgebee/application-profiler#130, which takes the snapshot instead. No version bump: 0.1.25 is the latest release and 0.1.26 is still unreleased, so this correction belongs in 0.1.26 rather than burning another number for a chart nobody has installed yet.
mayankpande88
added a commit
to nudgebee/k8s-agent
that referenced
this pull request
Sep 19, 2026
…es (#623) * fix(chart): pick up the profiler image whose heap profiles have samples ca8a3b3 asks Go for /debug/pprof/heap?seconds=30, which returns the delta over that window rather than a snapshot - so every heap profile of a service with a steady heap came back with Total samples = 0 and rendered as an empty flamegraph. 2ca753a is the first tag with nudgebee/application-profiler#130, which takes the snapshot instead. No version bump: 0.1.25 is the latest release and 0.1.26 is still unreleased, so this correction belongs in 0.1.26 rather than burning another number for a chart nobody has installed yet. * chore: update image tags for main release --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Shiv <3078106+blue4209211@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Go heap profiles come back empty — the Profiler tab and any event evidence show a flamegraph with nothing in it:
The request was
/debug/pprof/heap?seconds=30. On the heap endpointsecondsdoes not mean "sample for this long" — Go samples, waits, samples again, and returns the difference. A service whose heap is steady across that window deltas to exactly zero.A heap profile is a snapshot of what the process is holding, so it takes no duration. Dropping
secondsand addinggc=1— which collects first, so the snapshot describes live memory rather than whatever the last GC cycle left behind — turns the same request into a real profile.Type of change
Test plan
Reproduced against a real Go service with a steady ~64MB live heap and
net/http/pprof, then confirmed the fix on the same process:The empty output is character-for-character what the dev cluster produced: profiles of
llm-serveralternated between ~300 and ~16000 base64 bytes across four runs, the small ones decoding to a valid pprof with only sample-type headers and zero samples — i.e. whenever the heap happened to be steady for the 30s window.go build ./...andgo vetpass. Not unit-testable without an HTTP fixture: the URL is built inline inheapProfileand the function shells out tonsenter/wget.Checklist
go build ./...passes locallyRelated issues
The last of the profiler defects from this series: #128 (image shipped no async-profiler), #129 (read-only rootfs / non-root targets), nudgebee/k8s-agent#619 (language detection).