Add Grok Build harness - #2366
Conversation
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| f"mkdir -p {GROK_DIR} && " | ||
| f'until ln -s "$$" {lock} 2>/dev/null; do ' | ||
| f"owner=$(readlink {lock}); " | ||
| f'if ! kill -0 "$owner" 2>/dev/null; then ' |
There was a problem hiding this comment.
🟠 High grok_build/harness.py:80
A stale install.lock can make every later setup hang indefinitely: after the installer dies without running its EXIT trap, a reused or unrelated live PID makes kill -0 "$owner" keep succeeding even though no process owns the lock. Use a kernel-owned lock primitive such as flock, or validate ownership beyond PID liveness and bound the wait.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/harnesses/grok_build/harness.py around line 80:
A stale `install.lock` can make every later `setup` hang indefinitely: after the installer dies without running its `EXIT` trap, a reused or unrelated live PID makes `kill -0 "$owner"` keep succeeding even though no process owns the lock. Use a kernel-owned lock primitive such as `flock`, or validate ownership beyond PID liveness and bound the wait.
b98682e to
0b1a8a0
Compare
|
|
||
| INSTALL = r""" | ||
| set -e | ||
| root=/var/tmp/vf-grok-build |
There was a problem hiding this comment.
🟡 Medium grok_build/harness.py:31
Concurrent or sequential mixed-version runs launch the wrong Grok Build release: setup A can verify version A, then setup B replaces the shared /var/tmp/vf-grok-build/bin/grok, so A later starts B's binary through BINARY. The install lock only serializes installation and is released before rollout, while INSTALL mutates the shared path whenever .version differs; use a version-specific install/binary path or an immutable versioned binary reference so each harness retains its configured version.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/harnesses/grok_build/harness.py around line 31:
Concurrent or sequential mixed-version runs launch the wrong Grok Build release: setup A can verify version A, then setup B replaces the shared `/var/tmp/vf-grok-build/bin/grok`, so A later starts B's binary through `BINARY`. The install lock only serializes installation and is released before rollout, while `INSTALL` mutates the shared path whenever `.version` differs; use a version-specific install/binary path or an immutable versioned binary reference so each harness retains its configured `version`.
0b1a8a0 to
039491d
Compare
2f63e73 to
e46f3e2
Compare
039491d to
b1ea7ec
Compare
1 participant
Summary
Streaming watchdog compatibility
The training-streaming PR, #2369, opens SSE immediately and emits valid content-free protocol events while renderer generation is buffered. Grok Build 1.0.3 treats parsed Responses lifecycle events as activity, which was confirmed against the exact pinned binary.
Newer Grok behavior is stricter: its content watchdog ignores lifecycle and empty-delta events. This harness therefore sets
inference_idle_timeout_secsto 25 hours—just beyond the 24-hour remote sandbox limit—so Verifiers remains the deadline owner. This avoids corrupting assistant content or tool state with fake nonempty deltas.Live validation
Using the repository's
PRIME_API_KEY:Contributor checks
uv run pytest tests/— 914 passed, 77 skipped (test-created commits used process-local signing disable)uv run pytest tests/v1 -m 'not e2e'— 70 passed, 77 deselected after rebasetyhooks passedThe commit is signed with the contributor's verified SSH signing key.