fix(profiler): reach the target through its own mount namespace

[mayankpande88]

Summary

Files we stage for the target — and files the target writes for us — were resolved through the container runtime's overlay directory (util.ContainerFileSystem). That is only the same directory when nothing is mounted over the container's root. A pod with readOnlyRootFilesystem: true and an emptyDir at /tmp is the common counter-example, and there everything we put in the target's /tmp lands somewhere the process cannot see.

Java flamegraphs fail exactly that way on such pods, with Could not start attach mechanism: No such file or directory: both jattach's .attach_pid<pid> handshake file and libasyncProfiler.so, which the JVM dlopens, were written through the overlay.

This resolves those paths via /proc/<pid>/root instead — the target's whole mount namespace, volumes included. It is the same directory as before when nothing is mounted there, and the correct one when something is, so it replaces the old behaviour rather than branching on it. The same change applies to jcmd (the JVM writes heap dumps and JFR recordings itself) and to the Node heapsnapshot (written by the Node process into its working directory).

It also hands the staged profiler to the user the target runs as: we stage as root, and the JVM has to read the library and write its output into that directory. Hardened images rarely run as root.

SetUp is reordered in the two JVM profilers — PIDs are now resolved first, because the target's mount namespace is only reachable through one of its PIDs. One consequence: with an explicit --pid, the container runtime is no longer consulted during SetUp at all.

ContainerFileSystem is left in place (it is still the honest answer to "where does the runtime keep this container") but now carries a comment saying what it is not, so the next profiler does not reach for it by reflex.

Type of change

• Bug fix

Test plan

Unit tests cover the new path helper, the credentials read, and the chown step (root target → no chown; non-root → handed over; unreadable pid → error, not a silent skip). The reordering changed three existing expectations, each updated to the new contract rather than loosened.

Verified against a real read-only-rootfs pod — ECK Elasticsearch on the dev cluster, readOnlyRootFilesystem: true, an emptyDir mounted at /tmp, running as uid 1000 — where a Java flamegraph currently fails. Evidence to follow in a comment once the test image finishes rolling.

Ahead of that, the mechanism was confirmed directly from a hostPID pod on the same node:

ls /proc/12275/root/tmp        →  elasticsearch-954707…, final-flags…, hsperfdata_elasticsearch
kubectl exec es -- ls /tmp     →  elasticsearch-954707…, final-flags…, hsperfdata_elasticsearch   (identical)
/proc/12275/status             →  Uid: 1000 1000 1000 1000

The overlay directory shows none of that.

Checklist

• go build ./... and go test ./... pass locally (TestAsyncProfiler_CleanUp and TestExecuteWhenError fail on macOS both before and after this change — pre-existing, verified against main in a clean worktree; both pass on Linux CI)
• No breaking changes to the agent's stdout JSON envelope

Related issues

Follows #128, which fixed the image itself; this fixes the next failure it exposed.

[gemini-code-assist]

Code Review

This pull request refactors how the target container's filesystem is accessed by switching from the container runtime's overlay directory to the target's mount namespace via /proc/<pid>/root. It also introduces ownership adjustment (chown) of the staged profiler files to match the target process's UID/GID, allowing non-root JVMs to access them. The review feedback correctly identifies potential runtime panics in async_profiler.go and jcmd.go if the target PID list is empty, and suggests a safety check for an empty root PID in node_dummy.go.

[mayankpande88]

Verified against the pod that could not be profiled

Ran against ECK Elasticsearch on the dev cluster — readOnlyRootFilesystem: true, an emptyDir mounted at /tmp, running as uid 1000 — with a test image built from this branch.

Before (published ca8a3b3, i.e. main):

pod_profiler: profiler reported error: could not launch profiler:
Could not start attach mechanism: No such file or directory
FdTransfer accept(): Resource temporarily unavailable

After (this branch), same pod, 60s CPU profile:

status   COMPLETED
tool     async-profiler
output   176,246 bytes, 2142 frames

with real stacks, not an empty template:

java/lang/Thread.run
org/elasticsearch/transport/netty4/NettyByteBufSizer.channelRead
org/elasticsearch/http/netty4/Netty4HttpServerTransport$HttpChannelHandler$1.createMessage
org/elasticsearch/http/netty4/Netty4HttpHeaderValidator.channelRead
java/util/zip/Inflater.inflateBytesBytes

No regression on the ordinary case: demo/fraud-detection (writable rootfs, root) still returns a full flamegraph.

CI is green (11/11). The test tag has been removed and the dev cluster put back on ca8a3b3.