[ML] Periodically report pytorch_inference RSS

[edsavage]

Summary

Adds a periodic memory reporter to pytorch_inference so Elasticsearch can track the process's real resident set size (RSS) for a trained model deployment, independent of inference traffic.

• A dedicated timer thread emits process stats every 10s (a single RSS read; negligible cost), so memory is observed even when a deployment is idle or between requests.
• Each report carries both the current RSS (memory_rss) and the OS peak / high-water mark (memory_max_rss). The peak is transmitted explicitly rather than derived on the ES side from the sample stream, so transient spikes between reports are not lost — this is the signal used to keep model assignment and adaptive scaling OOM-safe.
• The process-stats document key is renamed from process_stats to stats to match the Elasticsearch PyTorchResult / InferenceProcessStats parser.
• The reporter thread is shut down promptly (condition-variable signalled) before the rest of the process tears down, so it can never write to a closing stream.

Relates #2885

Release-ordering note (important)

ES's InferenceProcessStats parser is strict and treats an unknown field as a fatal parse error for the result stream. Because this PR (re)introduces memory_max_rss, the Elasticsearch-side change that accepts memory_max_rss as an optional field must be released before the ml-cpp version emitting it is bumped into Elasticsearch. See the companion Elasticsearch PR. Merging this PR in the ml-cpp repo is safe on its own; the constraint applies to the ml-cpp version bump inside ES.

Test plan

• CResultWriterTest.testWriteProcessStats updated for the stats schema + memory_max_rss; passes (all 7 CResultWriterTest cases pass).
• Manual: run pytorch_inference with a model and confirm periodic stats docs are emitted ~every 10s with memory_rss and memory_max_rss. Observed {"request_id":"ignore","stats":{"memory_rss":0,"memory_max_rss":93339648}} after one 10s tick on macOS (memory_rss is 0 on macOS — platform-specific; both fields will be non-zero on Linux).
• Confirm prompt shutdown (no stray writes / hangs on exit). Process exits with code 0 cleanly after stdin closes.

Made with Cursor

[edsavage]

buildkite run_serverless_tests ES_SERVERLESS_BRANCH=feature/ml-cpp-repo-override

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[elasticsearchmachine]

Hi @edsavage, I've created a changelog YAML for you.

[edsavage]

buildkite run_serverless_tests ES_SERVERLESS_BRANCH=feature/ml-cpp-repo-override

[edsavage]

buildkite run_serverless_tests

[Copilot]

Pull request overview

Adds periodic current and peak RSS reporting for pytorch_inference, with updated stats serialization and evaluation tooling.

Changes:

• Adds a stoppable 10-second memory reporter thread.
• Renames process-stat output to stats and includes RSS fields.
• Updates tests, evaluator parsing, and changelog documentation.

Reviewed changes

Copilot reviewed 2 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
docs/changelog/3160.yaml	Adds changelog metadata.
docs/CHANGELOG.asciidoc	Adds the release-note entry.
bin/pytorch_inference/unittest/CResultWriterTest.cc	Updates stats serialization expectations.
bin/pytorch_inference/Main.cc	Adds periodic memory reporting and shutdown handling.
bin/pytorch_inference/evaluate.py	Parses and displays updated memory statistics.
bin/pytorch_inference/CResultWriter.h	Documents memory stats output.
bin/pytorch_inference/CResultWriter.cc	Updates the stats output schema.

The periodic stats documents use request_id ignore, but evaluate.py counts them as request-triggered samples, which can misassociate request sizes and cause an IndexError. These samples must be distinguished before incrementing stats_count.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[edsavage]

buildkite build this