Vla test

README.md

@@ -241,6 +241,12 @@ telefuser/
 | `FlashVSR` | VSR | Streaming video super-resolution via [examples/flashvsr/README.md](examples/flashvsr/README.md) |
 | `SwiftVR` | Causal video restoration | Single-GPU BF16 offline and direct streaming restoration via [examples/swiftvr/README.md](examples/swiftvr/README.md) |
 
+### Robotics and Action Models
+
+| Pipeline | Task | Notes |
+|----------|------|-------|
+| `LingBot-VLA v2` | Vision-language-action inference | RobotWin observations to normalized canonical action chunks via the direct SDK or native structured service; see [examples/lingbot_vla_v2/README.md](examples/lingbot_vla_v2/README.md) |
+
 ### Video Generation
 
 | Pipeline | Task | Notes |
@@ -277,6 +283,7 @@ See [examples/README.md](examples/README.md) for the example runner and baseline
 - [docs/en/adding_new_model.md](docs/en/adding_new_model.md): integrating new models
 - [docs/en/adding_new_example.md](docs/en/adding_new_example.md): authoring examples and pipeline contracts
 - [docs/en/abot_world.md](docs/en/abot_world.md): ABot-World single-GPU interactive pipeline, controls, and tests
+- [examples/lingbot_vla_v2/README.md](examples/lingbot_vla_v2/README.md): LingBot-VLA v2 inference, structured service, parity, and validation boundaries
 - [examples/swiftvr/README.md](examples/swiftvr/README.md): SwiftVR checkpoint loading, streaming usage, performance, and acceleration options
 
 ## Known Limitations

benchmarks/telefuser_aiperf/README.md

@@ -113,9 +113,52 @@ TELEFUSER_AIPERF_CONCURRENCY=2 \
 ```
 
 Each terminal result is required to contain a finite `50x55` action chunk and the frozen structured result fields.
-The adapter retains an action hash, bounds, dimensions, verification status, target inference time, and peak memory;
-it does not copy full action arrays or Base64 cameras into AIPerf response records. This validates service execution
-and normalized action structure, not physical robot control semantics.
+The VLA adapter registers `vla_inference_time` and `vla_peak_memory` as AIPerf record metrics, so the normal AIPerf
+artifact contains their per-request values and aggregated p50/p95/p99 summaries alongside request latency, throughput,
+and success rate. It retains an action hash, bounds, dimensions, and verification status; it does not copy full action
+arrays or Base64 cameras into AIPerf response records. This validates service execution and normalized action structure,
+not physical robot control semantics.
+
+For a target-process RSS and per-GPU process-memory artifact, pass the native service PID. This is an external bounded
+sampler and does not change the TeleFuser service or any shared metric endpoint:
+
+```bash
+TELEFUSER_AIPERF_SERVICE_PID=<service-pid> \
+  bash benchmarks/telefuser_aiperf/scripts/run_vla_structured_bench.sh
+```
+
+The resource summary is written to
+`artifacts/telefuser_aiperf/vla_structured/resource_summary.json` and includes sample count, RSS mean/p50/p95/p99,
+and per-GPU process-memory mean/p50/p95/p99. It is intentionally separate from AIPerf server metrics because RSS is
+an observer-side process-tree fact, while GPU telemetry remains target-side Prometheus data.
+
+The checked-in AIPerf configuration defaults to two excluded warmup requests and 20 measured requests. For a longer
+distribution, override the request count and concurrency without editing the configuration:
+
+```bash
+TELEFUSER_AIPERF_REQUESTS=100 \
+TELEFUSER_AIPERF_CONCURRENCY=2 \
+  bash benchmarks/telefuser_aiperf/scripts/run_vla_structured_bench.sh
+```
+
+## VLA GPU Parallelism Assessment
+
+The current VLA policy validates `world_size == 1`, so this assessment is deliberately read-only and does not enable
+FSDP, tensor parallelism, or pipeline parallelism. It reports visible GPU capacity, current free memory, checkpoint
+size, and how many complete replicas fit using a measured resident-memory estimate:
+
+```bash
+.venv-vla/bin/python tools/validation/inspect_lingbot_vla_v2_gpu_plan.py \
+  --model-root /hhb-data/aigc/model_zoo/lingbot/lingbot-vla-v2-6b \
+  --replica-memory-mib 13302 \
+  --output work_dirs/vla_gpu_plan.json
+```
+
+On the validated four-H100 host, the report showed four visible 80 GB GPUs and four estimated complete replicas at
+the 13,302 MiB measured process-memory baseline. This supports request-level multi-replica service capacity; it is
+not evidence that one model replica can be split across GPUs. Any future FSDP or tensor-parallel implementation must
+be introduced behind the VLA pipeline boundary and re-run the frozen preprocessing, velocity, action, and HTTP
+contract tests before it is considered equivalent.
 
 ## LingBot-World v2 Streaming
 

benchmarks/telefuser_aiperf/scripts/run_vla_structured_bench.py

@@ -0,0 +1,169 @@
+"""Run the VLA AIPerf workload and sample target process resources."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import subprocess
+import threading
+import time
+from pathlib import Path
+from statistics import fmean
+from typing import Any
+
+import psutil
+
+_MIB = 1024.0 * 1024.0
+
+
+def _percentile(values: list[float], fraction: float) -> float | None:
+    if not values:
+        return None
+    ordered = sorted(values)
+    position = (len(ordered) - 1) * fraction
+    lower = int(position)
+    upper = min(lower + 1, len(ordered) - 1)
+    return ordered[lower] + (ordered[upper] - ordered[lower]) * (position - lower)
+
+
+def _summary(values: list[float]) -> dict[str, float | int | None]:
+    return {
+        "count": len(values),
+        "mean": fmean(values) if values else None,
+        "p50": _percentile(values, 0.50),
+        "p95": _percentile(values, 0.95),
+        "p99": _percentile(values, 0.99),
+        "max": max(values) if values else None,
+    }
+
+
+def _process_tree(root_pid: int) -> tuple[set[int], float]:
+    try:
+        root = psutil.Process(root_pid)
+        processes = [root, *root.children(recursive=True)]
+    except psutil.Error:
+        return set(), 0.0
+    process_ids: set[int] = set()
+    rss_bytes = 0
+    for process in processes:
+        try:
+            process_ids.add(process.pid)
+            rss_bytes += process.memory_info().rss
+        except psutil.Error:
+            continue
+    return process_ids, rss_bytes / _MIB
+
+
+def _gpu_memory(process_ids: set[int]) -> dict[str, float]:
+    if not process_ids:
+        return {}
+    try:
+        result = subprocess.run(
+            [
+                "nvidia-smi",
+                "--query-compute-apps=pid,gpu_uuid,used_gpu_memory",
+                "--format=csv,noheader,nounits",
+            ],
+            check=True,
+            capture_output=True,
+            text=True,
+            timeout=5,
+        )
+    except (OSError, subprocess.SubprocessError):
+        return {}
+    memory: dict[str, float] = {}
+    for line in result.stdout.splitlines():
+        fields = [field.strip() for field in line.split(",")]
+        if len(fields) != 3:
+            continue
+        try:
+            pid = int(fields[0])
+            value = float(fields[2])
+        except ValueError:
+            continue
+        if pid in process_ids and value >= 0:
+            memory[fields[1]] = memory.get(fields[1], 0.0) + value
+    return memory
+
+
+def _sample(root_pid: int, started_at: float) -> dict[str, Any]:
+    process_ids, rss = _process_tree(root_pid)
+    return {
+        "offset_seconds": time.perf_counter() - started_at,
+        "process_ids": sorted(process_ids),
+        "cpu_rss_mib": rss,
+        "gpu_memory_mib": _gpu_memory(process_ids),
+    }
+
+
+def run(args: argparse.Namespace) -> int:
+    root = Path(__file__).resolve().parents[3]
+    python = os.environ.get("TELEFUSER_AIPERF_PYTHON", str(root / ".venv-aiperf/bin/python"))
+    command = [python, "-m", "telefuser_aiperf.cli", "profile", "--config", str(args.config)]
+    environment = os.environ.copy()
+    adapter_root = root / "benchmarks/telefuser_aiperf"
+    environment["PYTHONPATH"] = f"{adapter_root}{os.pathsep}{environment.get('PYTHONPATH', '')}".rstrip(os.pathsep)
+
+    started_at = time.perf_counter()
+    process = subprocess.Popen(command, cwd=root, env=environment)
+    samples: list[dict[str, Any]] = []
+    stop = threading.Event()
+
+    def sample_loop() -> None:
+        while not stop.is_set():
+            samples.append(_sample(args.service_pid, started_at))
+            stop.wait(args.sample_interval)
+
+    sampler = threading.Thread(target=sample_loop, name="vla-aiperf-resource-sampler", daemon=True)
+    sampler.start()
+    return_code = process.wait()
+    stop.set()
+    sampler.join(timeout=max(args.sample_interval * 2.0, 1.0))
+    samples.append(_sample(args.service_pid, started_at))
+
+    rss_values = [float(sample["cpu_rss_mib"]) for sample in samples]
+    gpu_values: dict[str, list[float]] = {}
+    for sample in samples:
+        for gpu, value in sample["gpu_memory_mib"].items():
+            gpu_values.setdefault(gpu, []).append(float(value))
+    report = {
+        "schema_version": 1,
+        "benchmark": "lingbot_vla_v2_aiperf_structured",
+        "aiperf_command": command,
+        "service_pid": args.service_pid,
+        "return_code": return_code,
+        "elapsed_seconds": time.perf_counter() - started_at,
+        "cpu_rss_mib": _summary(rss_values),
+        "gpu_process_memory_mib": {gpu: _summary(values) for gpu, values in sorted(gpu_values.items())},
+        "samples": samples[-args.max_samples :],
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+    print(json.dumps(report, indent=2, sort_keys=True))
+    return return_code
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--config",
+        type=Path,
+        default=Path("benchmarks/telefuser_aiperf/configs/vla_structured_e2e.yaml"),
+    )
+    parser.add_argument("--service-pid", type=int, required=True)
+    parser.add_argument("--sample-interval", type=float, default=1.0)
+    parser.add_argument("--max-samples", type=int, default=600)
+    parser.add_argument(
+        "--output",
+        type=Path,
+        default=Path("artifacts/telefuser_aiperf/vla_structured/resource_summary.json"),
+    )
+    args = parser.parse_args()
+    if args.service_pid < 1 or args.sample_interval <= 0 or args.max_samples < 2:
+        parser.error("service PID must be positive, interval must be positive, and max-samples must be at least 2")
+    raise SystemExit(run(args))
+
+
+if __name__ == "__main__":
+    main()

benchmarks/telefuser_aiperf/scripts/run_vla_structured_bench.sh

@@ -22,4 +22,15 @@ if command -v curl >/dev/null 2>&1; then
 fi
 
 export PYTHONPATH="${ADAPTER_ROOT}${PYTHONPATH:+:${PYTHONPATH}}"
+if [[ -n "${TELEFUSER_AIPERF_SERVICE_PID:-}" ]]; then
+    RESOURCE_PYTHON="${TELEFUSER_VLA_PYTHON:-${ROOT_DIR}/.venv-vla/bin/python}"
+    if [[ ! -x "${RESOURCE_PYTHON}" ]]; then
+        echo "The VLA resource sampler interpreter is unavailable: ${RESOURCE_PYTHON}" >&2
+        exit 1
+    fi
+    exec "${RESOURCE_PYTHON}" benchmarks/telefuser_aiperf/scripts/run_vla_structured_bench.py \
+        --config "${CONFIG_PATH}" \
+        --service-pid "${TELEFUSER_AIPERF_SERVICE_PID}" \
+        --output "${TELEFUSER_AIPERF_RESOURCE_OUTPUT:-artifacts/telefuser_aiperf/vla_structured/resource_summary.json}"
+fi
 exec "${AIPERF_PYTHON}" -m telefuser_aiperf.cli profile --config "${CONFIG_PATH}"

benchmarks/telefuser_aiperf/telefuser_aiperf/vla_structured.py

@@ -14,17 +14,21 @@
 from urllib.parse import quote, urlsplit, urlunsplit
 
 import orjson
-from aiperf.common.exceptions import NotInitializedError
+from aiperf.common.enums import MetricFlags, MetricSizeUnit, MetricTimeUnit
+from aiperf.common.exceptions import NoMetricValue, NotInitializedError
 from aiperf.common.models import (
     BaseResponseData,
     ErrorDetails,
     InferenceServerResponse,
     ParsedResponse,
+    ParsedResponseRecord,
     RequestInfo,
     RequestRecord,
     TextResponse,
 )
 from aiperf.endpoints.base_endpoint import BaseEndpoint
+from aiperf.metrics import BaseRecordMetric
+from aiperf.metrics.metric_dicts import MetricRecordDict
 from aiperf.plugin.schema.schemas import TransportMetadata
 from aiperf.transports.aiohttp_transport import AioHttpTransport
 
@@ -60,6 +64,43 @@ class VlaActionResponseData(BaseResponseData):
     peak_memory_mb: float | None = None
 
 
+class VlaInferenceTimeMetric(BaseRecordMetric[float]):
+    """Expose server-measured VLA inference time to AIPerf."""
+
+    tag = "vla_inference_time"
+    header = "VLA Inference Time"
+    short_header = "VLA Inference"
+    unit = MetricTimeUnit.SECONDS
+    display_unit = MetricTimeUnit.MILLISECONDS
+    display_order = 310
+    flags = MetricFlags.NONE
+
+    def _parse_record(self, record: ParsedResponseRecord, record_metrics: MetricRecordDict) -> float:
+        del record_metrics
+        for response in reversed(record.responses):
+            if isinstance(response.data, VlaActionResponseData) and response.data.inference_time_s is not None:
+                return response.data.inference_time_s
+        raise NoMetricValue("VLA inference time is not available in the structured response.")
+
+
+class VlaPeakMemoryMetric(BaseRecordMetric[float]):
+    """Expose server-measured peak accelerator memory to AIPerf."""
+
+    tag = "vla_peak_memory"
+    header = "VLA Peak Memory"
+    short_header = "VLA Peak Memory"
+    unit = MetricSizeUnit.MEGABYTES
+    display_order = 311
+    flags = MetricFlags.NONE
+
+    def _parse_record(self, record: ParsedResponseRecord, record_metrics: MetricRecordDict) -> float:
+        del record_metrics
+        for response in reversed(record.responses):
+            if isinstance(response.data, VlaActionResponseData) and response.data.peak_memory_mb is not None:
+                return response.data.peak_memory_mb
+        raise NoMetricValue("VLA peak memory is not available in the structured response.")
+
+
 def _finite_number(value: Any, *, name: str, allow_none: bool = False) -> float | None:
     if value is None and allow_none:
         return None

benchmarks/telefuser_aiperf/tests/test_vla_structured.py

@@ -39,6 +39,11 @@ def test_registration_uses_aiperf_endpoint_and_transport_plugins() -> None:
     assert transport_class is TeleFuserStructuredHttpTransport
     assert plugins.get_endpoint_metadata("telefuser_vla_structured").requires_polling is True
 
+    from aiperf.metrics.metric_registry import MetricRegistry
+
+    assert MetricRegistry.get_class("vla_inference_time").__name__ == "VlaInferenceTimeMetric"
+    assert MetricRegistry.get_class("vla_peak_memory").__name__ == "VlaPeakMemoryMetric"
+
 
 def test_build_vla_payload_reuses_inline_image_for_three_cameras() -> None:
     encoded = base64.b64encode(b"image bytes").decode()

benchmarks/telefuser_aiperf/vla_structured_contract.yaml

@@ -33,7 +33,10 @@ metrics:
   - request_latency
   - request_throughput
   - success_rate
+  - vla_inference_time
+  - vla_peak_memory
   - server_metrics
 artifacts:
   config: benchmarks/telefuser_aiperf/configs/vla_structured_e2e.yaml
   dataset: benchmarks/telefuser_aiperf/data/vla_structured.jsonl
+  resource_summary: artifacts/telefuser_aiperf/vla_structured/resource_summary.json