Skip to content

CVS-177198 LLM Memory Usage tests - #4360

Merged
porlows1 merged 7 commits into
mainfrom
porlows1_memory_tests_update
Aug 14, 2026
Merged

CVS-177198 LLM Memory Usage tests#4360
porlows1 merged 7 commits into
mainfrom
porlows1_memory_tests_update

Conversation

@porlows1

@porlows1 porlows1 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

🛠 Summary

CVS-177198 Update long tests for LLMs memory usage
Changes in tests related files required by https://github.com/intel-innersource/frameworks.ai.openvino.model-server.tests/pull/2684

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

Copilot AI review requested due to automatic review settings July 8, 2026 13:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the functional-test resource monitoring utilities to better support LLM long-tests focused on memory usage, including additional memory breakdown metrics and Windows support.

Changes:

  • Extend Docker resource monitoring to capture additional memory metrics (private/anon vs cache) and add an efficient “sample all metrics” path.
  • Introduce a Windows resource monitor that collects process memory stats via PowerShell and writes them to artifacts.
  • Update OVMS run context to attach the correct resource monitor depending on Docker vs Windows execution.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
tests/functional/object_model/resource_monitor.py Adds extra Docker memory metrics, introduces sample_all(), and implements a new Windows resource monitor with CSV logging.
tests/functional/object_model/ovms_instance.py Attaches Docker vs Windows resource monitors based on runtime environment and available OVMS PID.

Comment thread tests/functional/object_model/resource_monitor.py Outdated
Comment thread tests/functional/object_model/resource_monitor.py
@porlows1
porlows1 force-pushed the porlows1_memory_tests_update branch from 3125006 to 3f5b5c7 Compare July 9, 2026 08:11
@porlows1
porlows1 force-pushed the porlows1_memory_tests_update branch 2 times, most recently from ba38983 to 511eab6 Compare July 20, 2026 18:36
@porlows1
porlows1 force-pushed the porlows1_memory_tests_update branch 2 times, most recently from 5aac6ff to ca77445 Compare July 31, 2026 17:55
@porlows1
porlows1 force-pushed the porlows1_memory_tests_update branch 3 times, most recently from c1ac16a to daac116 Compare August 12, 2026 08:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

tests/functional/object_model/resource_monitor.py:267

  • When writing CSV files on Windows, opening the file without newline="" can produce extra blank lines due to newline translation. Use newline="" when opening the file for csv.DictWriter.
        log_path = Path(artifacts_dir, f"windows_stats_pid_{self.ovms_pid}.log")
        with log_path.open("w") as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=self.FIELDS)

tests/functional/object_model/ovms_log_monitor.py:389

  • get_all_logs() always calls self._proc.get_output() first, which drains stdout from the shared stream queue via pop(). That defeats the mirror-file logic below (and contradicts the comment) because other monitors / log-saving code will still miss those stdout lines even when a mirror file exists. Reorder the logic so stdout is only popped when there is no mirror, and in the mirror case only drain stderr (if needed) while reading stdout from the mirror tail.
    def get_all_logs(self):
        stdout, stderr = self._proc.get_output()
        if stderr:

@porlows1
porlows1 merged commit e5e9afa into main Aug 14, 2026
2 checks passed
dtrawins pushed a commit that referenced this pull request Aug 17, 2026
### 🛠 Summary

[CVS-177198](https://jira.devtools.intel.com/browse/CVS-177198) Update
long tests for LLMs memory usage
Changes in tests related files required by
intel-innersource/frameworks.ai.openvino.model-server.tests#2684

### 🧪 Checklist

- [ ] Unit tests added.
- [ ] The documentation updated.
- [ ] Change follows security best practices.
``
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants