docs(evidence): record first hosted mount latency measurement - #407
Draft
khaliqgant wants to merge 8 commits into
Draft
docs(evidence): record first hosted mount latency measurement#407khaliqgant wants to merge 8 commits into
khaliqgant wants to merge 8 commits into
Conversation
…surement The public "sub-200ms end-to-end including measurement overhead that exceeds the signal" was never a one-way measurement. It came from the 2026-07-26 evidence, which measured a round trip (median 315.5 ms, n=12, 5 ms polling echo) and halved it. This replaces the inference with a direct one-way measurement, sf-mini as receiver. Both halves of the old claim are wrong, in opposite directions. "sub-200ms" is false for realistic change sets. A single small file propagates in a median of 20.2 ms (p95 161.7 ms, n=20), but a repo-sized change set of 11 files / ~14 KB takes a median of 216.7 ms (p95 303.9 ms, n=20). The receive path fetches each file with its own server round trip (syncer.go:3310,3320,3339), so cost scales with file count, not bytes. Agents commit change sets; small single files were never the case that mattered. "measurement overhead that exceeds the signal" is false by more than an order of magnitude and is retired. The watcher's own detection delay was measured against a local control, not assumed: median 1.225 ms, max 2.455 ms over 25 pairs. Every figure holds only under the stated topology precondition, which is repeated beside each median in RESULTS.md: the server ran on the sender's own machine, so sender->server was loopback and the only network hop was server->receiver over a Tailscale LAN. These are LAN best cases and do not license any claim about the hosted product, faster or slower. RESULTS.md states the three prerequisites a product claim would need. No sub-100ms claim is made. Two things the method caught that a simpler one would have missed: - The hosts' clock offset drifted 8.323 ms across a ~21 minute run (-6.441 ms to -14.765 ms), comparable to the small-file signal itself. The offset is therefore interpolated to each trial's own send time rather than pinned. A single offset measurement would have looked reasonable and been quietly wrong. - sf-mini's status/live flipped online<->offline four times per gate window while its heartbeat advanced monotonically, and an MCP query_nodes call reported it flatly offline with a 38 s old, advancing heartbeat. Liveness is gated on heartbeat advance across >=90 s, before and after the trials. Trusting status would have abandoned the run against a healthy host. 52 trials, all HTTP 202, zero lost files, zero partial change sets, no sample near the ~30 s websocket-off fallback. The headline batches are the clean 20+20; an earlier 12-trial batch cut short by an interrupt is preserved as correctness evidence only and contributes to no percentile. 26/26 named assertions pass via harness/assertions.py. Isolation: fresh server on a separate port with a separate state dir, fresh workspace, distinct mount path. The pre-existing .dev-collab-stack/ and .salvaged-from-minis/ trees, their processes and ports, were not touched or reused. Teardown in CLEANUP.md. Test credentials were ephemeral, kept outside the repo, and are absent from every artifact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Relayfile Eval ReviewRun: Passed: 4 | Needs human: 0 | Reviewable: 0 | Missing output: 0 | Failed: 0 | Skipped: 0 Human Review CasesNo reviewable human-review cases captured Relayfile output. |
added 3 commits
August 7, 2026 23:01
Base automatically changed from
evidence/mount-latency-one-way-20260807
to
main
August 8, 2026 19:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The first successful one-way latency measurement through the hosted Relayfile deployment at
https://api.relayfile.dev, using the existing 2026-08-07 baseline harness and a distinct sender and receiver host.Do not merge. The merge gate belongs to the principal.
This is a stacked draft PR based on
evidence/mount-latency-one-way-20260807/ #405. It should be retargeted to the default branch only after #405 lands. The hosted deployment build is recorded as undetermined because the public health body and response headers exposed no application build identifier; the local receiver CLI commit is not presented as the server build.Result
Nothing is halved. Leg A is the real sender-to-hosted-service WAN request; leg B is hosted acknowledgement to the receiver-side last-file observation.
Failed and incomplete attempts are retained
The clean headline populations are not the whole record:
Every sender and receiver JSONL record is committed. Incomplete observations are named and excluded from percentiles, not deleted or silently counted as loss. The r4 headline distribution is explicitly conditional on a fresh, successfully completing mount.
Method
/trials;Claim boundary
This PR licenses only a factual statement about the measured deployment, workspace, date, and topology. It does not license a product-wide SLA, a tail guarantee, or a claim about an identifiable release. Completed-trial percentiles do not erase the 403, 500, websocket, reconcile, or incomplete-observation outcomes.
Both measured hosted populations are far above 200 ms, so this run supplies no support for a sub-200 ms hosted-product claim.
Verification
cd docs/evidence/mount-latency-hosted-20260807 python3 harness/assertions.py54/54 named assertions pass. All raw JSONL and analysis JSON parse successfully, the raw records contain only
sender/receiveraliases, and the post-capture hardened analysis reproduces the same headline figures from byte-unchanged raw data.