A shared clipboard between the development host and ToyOS under QEMU, through QEMU's vdagent

[Japabu]

Design round 2. Nothing is built: the branch is one empty commit, and the build begins only once this design is accepted in a comment here. The owner gave his go for a shared clipboard between the development host and ToyOS under QEMU: copy on the host and paste in ToyOS, and the reverse. Round 1 was reviewed in #894 (comment) and ruled on in #894 (comment); the mapping from each finding to this body is a comment below.

What the published definitions say

The design follows three published definitions and nothing else: the virtio 1.2 specification (OASIS, cs01), spice-protocol's spice/vd_agent.h at 499cc832, and QEMU v11.1.1 (build/ci/qemu-version), which makes both the device and the host end of the channel. No other operating system's guest agent or driver is followed.

• QEMU's host side is the qemu-vdagent chardev, clipboard=on (QAPI ChardevQemuVDAgent, compiled only with spice-protocol: qapi/char.json:460-474, ui/meson.build:68).

  • It bridges one type, UTF-8 text (ui/vdagent.c:313-321).
  • It announces CLIPBOARD_BY_DEMAND, CLIPBOARD_SELECTION and CLIPBOARD_GRAB_SERIAL together (:192-196). A capability changes the wire only where the guest announced it too (:303-311).
  • It writes chunks of at most 1024 bytes (:167-176), and drops any message to the guest past its 1 MiB buffer with buffer full, dropping message (:162-165).
  • It stores a CLIPBOARD its owner sends unrequested (:585-606, ui/clipboard.c:232-257). Once the data is stored, a host request returns at once (ui/clipboard.c:196-209). If it owns nothing the guest asked about, it answers a guest REQUEST with no data (ui/vdagent.c:556-583).

• The grab serial. spice-protocol gives a GRAB a serial under CLIPBOARD_GRAB_SERIAL (vd_agent.h:301-310) and does not say what it means. QEMU's side is the definition the agent talks to:

  • QEMU resets its last_serial when the guest announces its capabilities (ui/vdagent.c:724-729).
  • A host grab carries last_serial, which then goes up by one (:342-351).
  • A guest grab whose serial is below last_serial is discarded, and any other one is kept and sets last_serial (:516-530). A CLIPBOARD sent after a discarded grab is dropped too (:602-605).
  • QEMU closes and reopens its end to reset the serials (:455-465). The guest sees PORT_OPEN(0), and QEMU reopens only once the guest has closed its side (:874-895, hw/char/virtio-console.c:111-125, hw/char/virtio-serial-bus.c:264-290). While the host end is closed, the guest's transmits are discarded (:479-495).

• The host's UI peers wait with no bound for text the guest owns and has not sent:

  • cocoa does so under QEMU's big lock (ui/cocoa.m:1793-1821);
  • GTK spins QEMU's main loop (ui/gtk-clipboard.c:51-57).

  Both waits end as soon as the data is stored.

• GTK's clipboard is off by default, and why. Commit 29e0bfffab87 disabled it at build time because it "may cause guest hangs": QEMU issue #1150, guest soft lockups. Its reads of the host's clipboard then waited on GTK inside QEMU's main loop (gtk_clipboard_wait_for_text, gtk_clipboard_wait_is_text_available). In 11.1, 9a007326e83f made those reads asynchronous. Then 335e32cbd0a2 turned the build option into -display gtk,clipboard=on, still off by default (qapi/ui.json:1322-1323). All three commits are in v11.1.1. The one wait left in 11.1.1 is the loop above, and the push after each guest grab (§4) closes it: QEMU holds the text before any paste can ask for it.

• virtio-serial (virtio 1.2 §5.3):

  • Port 0's queues always exist. The control queues and every further port's queues exist only under VIRTIO_CONSOLE_F_MULTIPORT (§5.3.2, §5.3.3).
  • QEMU offers MULTIPORT only when max_ports > 1 (hw/char/virtio-serial-bus.c:541-543).
  • A multiport driver creates the queues max_nr_ports names and sends DEVICE_READY. The device then adds ports, and the driver answers each with PORT_READY (§5.3.5, §5.3.6.2.2). The device then names the port (PORT_NAME) and opens it (PORT_OPEN, both directions).
  • All queue setup precedes DRIVER_OK (§3.1.1).

• The vdagent stream (vd_agent.h:44-61, :83-105, :271-326, :399-402):

  • A VDIChunkHeader {port, size} frames a VDAgentMessage {protocol = 1, type, opaque, size}, and VD_AGENT_MAX_DATA_SIZE is 2048.
  • The clipboard messages are GRAB [serial][types], REQUEST [type], CLIPBOARD [type, data] and RELEASE [].
  • A GRAB tells the remote to "empty clipboard and wait for one of the types passed", and a CLIPBOARD "set[s] clipboard data (both directions)".

The design

1. The QEMU launch (build/qemu.rs, tests/common/qemu.rs)

Every virtio profile adds three arguments, created just before the console's function:

-chardev qemu-vdagent,id=vdagent0,clipboard=on,mouse=off
-device virtio-serial-pci-non-transitional,id=vdagent-serial,max_ports=2{platform}
-device virtserialport,bus=vdagent-serial.0,nr=1,chardev=vdagent0,name=com.redhat.spice.0

cargo run also passes -display cocoa on macOS and -display gtk,clipboard=on on other hosts, one choice per host as audio_backend() makes. The harness keeps -display none.

• The agent's port is on a virtio-serial function of its own. The console's function is the kernel's: it is the log channel from early boot, and a PCI function belongs to the kernel or to one claim.
• The order makes every virtio boot run the kernel's console rule. The agent's multiport function enumerates first, so a kernel that took the first 1af4:1043 would drive it and lose its console. Nothing else moves: the harness's slots (tests/common/iommu.rs:195-199, :292) are all created earlier.
• The arguments, the port name and max_ports = 2 come from one function in the toyos_build library. It reads the codec crate's declaration (§3), which the agent reads too, and both launchers call it.
• The prerequisite check asks QEMU's -chardev help for qemu-vdagent before cargo run or the harness launches anything. cargo run on a host that is not macOS also asks its -display help for gtk. A missing one is refused by name: "built without spice-protocol" or "no GTK display; Debian ships it as qemu-system-gui". Nothing falls back.
• Measured, every run EXIT=0. Each run is round 1's command line with only info pci and quit, then cargo run's Virtio profile on x86_64 and on aarch64 as designed here, with no firmware and the drives on null-co. The agent's function is at 00:06.0 before the console's at 00:07.0 on q35, and at 00:07.0 before 00:08.0 on virt.
  • Development host, QEMU 11.1.1: log (:69-109, :111-186, :188-254).
  • CI's pinned guest image (debian@sha256:a2aa4626…, its snapshot's QEMU 1:11.1.1+ds-1): log. It lists qemu-vdagent for both architectures (:50, :78). The same three runs exit 0 (:143, :220, :288). The deps step's packages pull in qemu-system-gui (:29).
  • Windows: MSYS2's UCRT64 QEMU 11.1.2, the build QEMU's download page names beside Stefan Weil's installers. It was built with spice-protocol 0.14.5 installed, and both system binaries carry chardev-qemu-vdagent and GTK's clipboard peer (log, read as bytes, never run).

2. The kernel: two functions with one ID

Two 1af4:1043 functions break two things today:

• the console takes the first one enumerated (kernel/src/drivers/virtio_console.rs:199);
• a claim on an ID that two functions carry is refused Ambiguous, and the kernel's own function counts (kernel/src/pcidev/mod.rs:715-747).

Each rule is a pure decision in toyos_pci (kernel/pci/), beside slot, with host tests that fail without it.

• (a) The console is the first virtio console function whose device features offer no VIRTIO_CONSOLE_F_MULTIPORT. That is the only shape its single-port driver drives: a multiport device names its console ports by CONSOLE_PORT messages on queues this driver has none of (virtio 1.2 §5.3.6.2).
  • The kernel reads each candidate's feature word by §3.1.1's first four steps: reset, ACKNOWLEDGE, DRIVER, read. That happens before bus mastering and before an address space, because enable_bus_master is what records a function as kernel-driven (kernel/src/drivers/pci.rs:253-257, kernel/src/pcidev/mod.rs:480-486).
  • A function it declines is reset again, named in the log and left to a claim.
  • toyos_pci::console(features: &[u64]) -> Option<usize>.
• (b) A claim's vendor:device resolves among the functions the kernel does not drive. One undriven match is the claim's, two are Ambiguous, and none among driven ones is KernelDriven. The existing KernelDriven control, usbd_drives_the_spare, still holds: that machine has one 1b36:000d. toyos_pci::claimed(driven: impl IntoIterator<Item = bool>) -> Result<usize, Unclaimable>.
• Fence. The build needs kernel/src/drivers/virtio.rs beside the three paths the ruling widened the fence to. Reading a feature word without bus mastering needs VirtioPciConfig::parse, which is private there (:84, :194). VirtioDevice::init enables bus mastering before it reads any feature (:721-731).

The alternative is the kernel's console driver going multiport and passing port 1 to a process. That is a kernel addition userland can own.

3. The codec crate, toyos-vdagent (lib/vdagent)

An input boundary of its own (Fit), pure, no_std, host-tested. Every word in it comes from the host:

• The channel's declaration: the port name com.redhat.spice.0, MAX_PORTS = 2 and the agent's port id 1. The launch (§1) and the agent read these.
• virtio-serial control messages (§5.3.6.2):
  • decode {id, event, value} plus PORT_NAME's bytes, bounded and UTF-8 checked;
  • an id at or past max_nr_ports is refused by name;
  • encode DEVICE_READY, PORT_READY and PORT_OPEN.
• The vdagent stream, parsed as a stream, because a chunk header can straddle two receive buffers:

  • A chunk over VD_AGENT_MAX_DATA_SIZE (2048) leaves the stream unframed. The program then ends by name.

  • A chunk for a port other than VDP_CLIENT_PORT is skipped by name.

  • A message needs protocol 1 and at most 4 + MAX_CLIPBOARD_BYTES (2 MiB + 4) bytes. A larger one is counted off without being buffered and refused by name, and the next message still decodes.

  • The buffer grows with the bytes that arrive, never to a size a header declares.

  • Each type's size follows its layout in vd_agent.h:

    • REQUEST: exactly 4;
    • RELEASE: 0;
    • ANNOUNCE_CAPABILITIES: at least 8;
    • GRAB: a serial plus a multiple of 4, at most 16 types;
    • CLIPBOARD: at least 4.

    An unknown type is refused by name.

  • The encoders write capabilities (CLIPBOARD_BY_DEMAND | CLIPBOARD_GRAB_SERIAL), GRAB [serial][UTF8_TEXT], REQUEST, CLIPBOARD and RELEASE, each in chunks of at most 2048 bytes.

4. The agent, /system/bin/vdagent (system/vdagent)

• The driver (the program) runs over toyos-pci-claim and toyos-virtio, with netstack's virtio_net.rs as the pattern.

  • It negotiates VERSION_1 | MULTIPORT | ACCESS_PLATFORM and refuses by name a function whose max_nr_ports is not 2.
  • Before DRIVER_OK it creates the six queues those two ports name (§5.3.5, §3.1.1), all on the one MSI-X entry the kernel programs.
  • It runs §5.3.6.2's handshake and opens only the port whose PORT_NAME is com.redhat.spice.0.
  • A CONSOLE_PORT, a DEVICE_REMOVE of its port, or a used ring it does not believe ends the program by name.
  • The host's PORT_OPEN(0) is answered with the guest's PORT_OPEN(0), so QEMU can reopen; its PORT_OPEN(1) is answered with PORT_OPEN(1). The session restarts at the host's next capabilities.

• Transmit is one buffer, 2 MiB + 4 KiB. It carries what the bridge owes the host, encoded from the bridge's state when the device has taken the previous one back.

  • A guest grab and its text go out in that one buffer. QEMU handles the whole buffer under its big lock (hw/char/virtio-serial-bus.c:160-212, util/main-loop.c:307-315), and a host paste needs the same lock (ui/cocoa.m:1799). So no paste can arrive between the GRAB and its CLIPBOARD.
  • Each kind of message is owed at most once. QEMU asks once per grab (ui/clipboard.c:199-208).
  • A transmit the device has not taken back within a bound ends the program loudly.

• The bridge is a pure crate nested in the program, system/vdagent/bridge, host-tested. The owner is None, Host or Guest(text). It keeps last, the serial of the guest's last grab in this session or none, and next, the serial its next grab carries.

  event	what the bridge does
  host capabilities	last := none, next := 0, and it answers with its own capabilities. A host without CLIPBOARD_BY_DEMAND or CLIPBOARD_GRAB_SERIAL is named once, and nothing flows. Owner Guest(t) (a copy came before the port opened): GRAB + CLIPBOARD(t). Otherwise the owner becomes Host and it sends REQUEST UTF8_TEXT, which QEMU answers from the host's clipboard, or with no data.
  host GRAB s	Kept iff last is none or s ≥ last. A kept grab sets next := max(next, s + 1). The owner becomes Host, the guest clipboard is emptied, and REQUEST UTF8_TEXT goes out if the grab offers it. A grab that is not kept is discarded by name.
  host CLIPBOARD	The data becomes the guest clipboard (empty included) only while the owner is Host and the data is UTF-8. Anything else is refused by name, so a reply that lost to a guest copy cannot land.
  host REQUEST	Always answered: the text when the owner is Guest and the type is UTF8_TEXT, no data otherwise.
  host RELEASE	Owner Host becomes None, and the guest clipboard is emptied. Any other owner ignores it, by name.
  guest copy (from the compositor, §5)	Text t: the owner becomes Guest(t), and GRAB [next] + CLIPBOARD(t) go out in one transmit, with last := next and next += 1. Empty: RELEASE if the owner was Guest, and the owner becomes None. A copy made while the port is closed only sets the owner. The next opening sends it.
  host PORT_OPEN(0)	The serials are cleared and the owner is kept. Nothing is owed until the next capabilities.

  Why the serial rule is right. QEMU keeps a crossing guest grab g iff g ≥ s + 1, that is iff g > s (ui/vdagent.c:342-351, :524-530). The bridge keeps the host's crossing grab s iff s ≥ g. Exactly one side wins, the same on both. Without a crossing, QEMU's grabs carry at least the serial of the last guest grab it kept. So the bridge keeps them, and its next grab carries one past them, which QEMU keeps. Serials compare as QEMU compares them, with no wrap (:524). A session that would pass u32::MAX ends by name.

• The loop. One Poller covers the claim's interrupt and the compositor connection. The agent connects to the compositor and watches (§5) before it opens the port. It is not restarted: one that ended says why and stays down for the boot.

5. The compositor and sdk/window

Today the compositor holds the clipboard as one String. Any holder of a compositor connector sets it, inline up to 4096 bytes and through the COPY_BEGIN/REGION/COMMIT exchange up to 2 MiB (system/compositor/src/session.rs:1051-1127). It is read only by GUI+V. Nothing can learn that it changed.

• A third port, clipboard, with one connection: the watcher.
  • Its first frame is MSG_CLIPBOARD_WATCH. A second connection while the watcher lives is refused by name.
  • It waits for that first frame in a slot of its own, as the prompt's does (toyos_desktop::pending_admits).
  • Its acceptor and its connection are two more poll registrations, so MAX_WINDOW_SLOTS goes from 218 to 216 (system/compositor/src/main.rs:46-63).
• One notification outstanding. MSG_CLIPBOARD_WATCH asks for the next copy, and it is the only frame that does.
  • The compositor keeps unsent: set by a client's set, cleared by the bridge's set and by a delivery.
  • While the watcher has asked and unsent is set, it is sent the clipboard, in the paste's two encodings, empty included. That clears the ask.
  • What WATCH delivers: the clipboard the compositor holds, as soon as a client's set is unsent. At boot that means a copy made before the agent watched.
  • The pipe holds at most one delivery, so a client copying in a loop moves only the latest copy and can never fill the watcher's pipe. A refused try_send still drops the watcher by name under the server doctrine. It cannot happen to a watcher that asked.
• The bridge's sets travel on the watcher's own connection: MSG_CLIPBOARD_SET, or MSG_COPY_BEGIN … MSG_COPY_REGION … MSG_COPY_COMMIT. The region is held by the watcher, not by a waiting connection, and it is the same exchange copy_begin/copy_commit serve.
  • A bridge set lands only while the watcher's ask is outstanding: only when the bridge has been sent every client copy before it. A set made before the bridge saw a client copy that is now in its pipe is refused by name. Without this rule, a set crossing a delivery would leave the guest with the host's text while the host is given the guest's copy. The rule can be decided only because the ask and the set travel on one ordered connection.
  • A bridge set is never sent back. Otherwise a host copy would return as a guest grab and take the host's clipboard from the app that owns it.
  • Both rules, and unsent, are one pure decision in toyos_desktop, host-tested.
• sdk/window gets MSG_CLIPBOARD_WATCH and the watcher's client side beside clipboard_set. It never blocks: every send is a try_send, and the compositor's MSG_COPY_REGION arrives as an event on the agent's Poller. One set is in flight at a time, and a newer host text replaces the one queued behind it. clipboard_set's blocking exchange (sdk/window/src/lib.rs:445-471) stays the apps'.

6. Authority

[programs.vdagent]
devices = ["pci:1af4:1043"]
receives = ["clipboard"]
exempt.owns = "the virtio-serial function QEMU's clipboard channel is on, and the compositor's clipboard port"

The row is in [boot] start, and the compositor's row also serves clipboard.

• The claim is the agent's virtio-serial function, the one the kernel does not drive under 2(b).
• clipboard reads every copy made in the session, so one row holds it. build/build.rs's HELD_ALONE gains (clipboard, vdagent), with the port's name and holder declared in toyos_manifest, as the prompt port's are.
• Nothing else: no compositor connector (it opens no window), no syscap and no folder.

7. Data types

UTF-8 text, and nothing else crosses. QEMU's clipboard has one type, its vdagent maps only VD_AGENT_CLIPBOARD_UTF8_TEXT, and the compositor's clipboard is a String. A host grab that offers no text empties the guest clipboard and requests nothing, because the last copy wins. Line endings pass untouched: QEMU converts nothing, and the agent announces no GUEST_LINEEND_*.

8. Security

• Host bytes are bounded before they are buffered (§3). Every refusal names itself, and no host word is unwrapped or used as an index. On host input the agent ends only on an unframed stream or a device it does not believe. The kernel is never involved.
• Text is checked as UTF-8 twice: by the agent, and again by the compositor.
• Memory stays bounded at any host rate. At most it holds:
  • one host message being assembled (≤ 2 MiB + 4);
  • the guest's text;
  • one transmit buffer;
  • one set in flight and one queued.
• The compositor never blocks on the agent, nor the agent on the compositor.
• What crosses the boundary. Every guest copy goes to the host, eagerly. Every host copy enters guest memory, eagerly, whether or not it is pasted there, a password manager's included, and every cargo run boot pulls in the host's current clipboard. In the guest only the compositor keeps it, and the agent only while it passes it on. A guest program gets it only by GUI+V. A guest program can write the developer's host clipboard; that is the feature, and clipboard=off turns it off.
• Neither the agent nor the compositor ever logs a clipboard byte: lengths and refusals only, a rule each module header states.

9. Metal

QEMU only. On the T14 the supervisor's mint gets NotFound and logs no pci:1af4:1043 on this machine. The agent finds no claim, says so in one line, and exits 0.

• Rule 2(a) finds no virtio console function and changes nothing.
• Rule 2(b) changes how every claim resolves on every machine. On the T14 it changes nothing only if no vendor:device named in a devices list is carried by two functions.
• The build round's T14 run therefore carries the boot's PCI inventory against images/system.toml's pci: entries, with each entry's count of functions. It also carries the boot rows, which show what the feature costs metal: one process start and its binary in the image.

10. Tests, under the owner's ruling in #890

• Host unit tests:

  • toyos-vdagent:
    • each refusal by name: a chunk over 2048, another port, protocol ≠ 1, a declared size past the bound counted off with the next message decoded, each type's size, an unknown type, a control id past max_nr_ports, a PORT_NAME that is not UTF-8;
    • a valid stream decodes the same at every split point;
    • each encoding byte-for-byte against vd_agent.h's packed layouts and §5.3.6.2's control layout, chunks included.
  • toyos-vdagent-bridge, one test per row of the table, among them:
    • the_answer_to_the_opening_request_lands;
    • the_opening_grabs_a_copy_made_before_the_port_opened;
    • host_text_that_lost_to_a_guest_copy_is_refused;
    • a_crossing_host_grab_older_than_the_guests_is_discarded;
    • a_crossing_host_grab_as_old_as_the_guests_is_kept;
    • a_guest_grab_carries_its_text_in_the_same_transmit;
    • every_request_is_answered;
    • the_hosts_close_clears_the_serials_and_keeps_the_owner.
  • toyos_desktop:
    • a_bridge_set_is_never_sent_back;
    • a_client_set_is_sent_once_asked;
    • only_the_latest_of_two_client_sets_is_sent;
    • nothing_is_sent_unasked;
    • a_bridge_set_made_before_a_delivery_was_seen_is_refused.
  • toyos_pci:
    • the_console_is_the_first_function_without_multiport;
    • multiport_functions_alone_give_no_console;
    • one_undriven_match_among_driven_ones_is_claimed;
    • two_undriven_matches_are_ambiguous;
    • only_driven_matches_are_kernel_driven.

• The kernel's rule (a) runs on every virtio guest boot, because the agent's function enumerates first (§1). A kernel that took the first function would boot with no console.

• One smoke row, on a new boot. No boot the suite runs today has both a compositor and a virtio function: every compositor boot is Metal-shaped, and Headless has no framebuffer. So a new tests/images/clipboardcase.toml on Profile::HeadlessVirtioGpu starts logkeeper, the compositor, vdagent and test-runner (for the ready marker). The row waits on the guest for two lines:

  • the agent's port open with the host's capabilities;
  • the length of the host's answer to the opening REQUEST, which is 0, because -display none gives QEMU's clipboard no peer.

  Why it needs QEMU: the device exists only under QEMU, so no metal row reaches it. The other end of the handshake and of the request is QEMU's own vdagent and virtio-serial, which no host test runs.

• Negative control: the smoke row run on the base, with the agent absent, is red. Each of these mutations must turn the named unit test red:

  • lifting the declared-size bound;
  • dropping the owner-is-Host check;
  • keeping every host grab;
  • sending the grab without its text;
  • sending a bridge set back;
  • delivering without an ask;
  • landing a stale bridge set;
  • taking the first console function;
  • refusing one undriven match as Ambiguous.

• Independent oracles:

  • virtio 1.2 and spice-protocol's header, two external specifications;
  • QEMU 11.1.1's vdagent and virtio-serial, live in the smoke: the device maker's own implementation.

  The full round trip, copying on the host and pasting in ToyOS and back, is not driven by any test, per the ruling.

Roast

• The cost. It is estimated at about 1,300 production lines, an estimate. On top of that come a codec crate, one boot process in every image (metal's included) and one new boot in the guest suite, all for a comfort only cargo run uses. The cheaper-looking alternatives are worse: the kernel passing on a port of its console, or pci-serial, which is a legacy UART off the published transport, with no open or close for the agent to restart on.

• Three kernel changes for a QEMU-only feature. Rule (a) reads a policy off a feature bit, and needs a feature read before bus mastering in virtio.rs. Rule (b) changes what a claim's name means on every machine.

• Three of the design's steps rest on QEMU's implementation, not on spice-protocol's text:

  • the opening REQUEST with no grab before it;
  • the unrequested CLIPBOARD after each grab, which leaves the by-demand reading: QEMU stores it, and spice-protocol's CLIPBOARD "set[s] clipboard data (both directions)" and names no request before it;
  • the serial rule, which spice-protocol leaves undefined.

  The module header cites the QEMU lines for each. A QEMU that changes the first breaks the smoke; one that changes either of the others fails with nothing visible.

• The eager push moves every guest copy, up to 2 MiB, to QEMU's memory whether or not the host pastes it. That is what removes the freeze: a halted, panicked or debugger-stopped guest no longer holds cocoa's or GTK's paste. No residue of the freeze is left to record.

• A host copy past QEMU's 1 MiB never arrives. The guest clipboard is then empty, not stale, and QEMU's stderr log says buffer full, dropping message.

• The compositor's stale-set rule refuses a host text that crossed a guest copy. The guest copy then wins on both sides, because the agent grabs it next and QEMU keeps that grab.

• Claim slots. A cargo run boot then claims all four of pcidev::MAX_FUNCTIONS (kernel/src/pcidev/mod.rs:146): sound, net and disk (images/system.toml:85, :95, :235), and the agent.

  • QEMU gives a virtio function PM and FLR only behind an Express port (hw/virtio/virtio-pci.c:2197-2198, :2260-2307). An agent that ended therefore keeps its slot as residue (kernel/pci/src/slot.rs:4-9), as netstack's and soundserver's functions do on that machine.
  • Nothing else claims there, so nothing is refused today. A fifth claim on that machine would be Exhausted.
  • More slots reach both architectures' interrupt tables (kernel/src/arch/x86_64/idt/user_dev.rs, kernel/src/arch/aarch64/irqchip/its.rs), which is outside the fence. The build round files this in issues/ with an exit: a cargo run boot leaves a claim slot free.

• Every development host and CI need a QEMU built with spice-protocol, and cargo run on a host other than macOS needs one with GTK. The prerequisite check says so by name. On those hosts the display becomes GTK, chosen explicitly.

What I am unsure of

• The Linux and Windows host path is unrun. Both -display gtk,clipboard=on and GTK's clipboard peer are unrun: QEMU ships the peer off by default, and this project's one interactive host is a Mac. The build round files that gap in issues/.
• Stefan Weil's Windows installers are not measured. Only MSYS2's build is.
• The CI image ran emulated. Its measurement ran under the development host's podman VM, which runs the amd64 image under qemu-user. The packages and binaries are the image's, but this was not CI's runner.
• Two parts of the fence. The build needs kernel/src/drivers/virtio.rs (§2), which the ruling's widened fence does not name. It also needs tests/images/clipboardcase.toml and two new issues/ files.

Gates

None run: this is a design round, so nothing is built or tested. Net lines: zero (one empty commit).

🤖 Generated with Claude Code

https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy

[Japabu]

Design review, round 1, head 0b12adca4.

Net lines (git diff --shortstat origin/main...0b12adca4): none, one empty commit. Production 0, tests 0.

Evidence: qemu-shape.log at ToyOSOrg/evidence 57668f44, whose commit message names head 0b12adca4. Both QEMU runs of the shape end with EXIT=139, and the cause is under the crash question below. QEMU was read at v11.1.1, spice-vdagent at e940b009 and spice-protocol at 499cc832.

The brief's questions

• The two kernel changes. This shape needs both, and no design without them is better. The only shape that needs neither puts the agent's function on transitional 1af4:1003, which the console never matches and a claim names alone. That puts a legacy interface on the machine to avoid a kernel decision. A multiport console is the kernel addition that §2 rightly refuses. Rule (a) draws the spec's line: a function without MULTIPORT has one port, and that port is its console; a multiport function names its console port by control message (hw/char/virtio-serial-bus.c:403-405). Two defects remain: no test reaches (a) (BLOCKER 5), and (b) is not limited to virtio (NOTE 4).

• The clipboard port. It is the least authority the feature allows. Any design that answers a host paste needs every guest copy. The port carries no window and no input, and the agent holds no compositor connector. HELD_ALONE keeps the port to one row. How the port delivers copies is not least (BLOCKER 4).

• Answering every request. It is necessary. Cocoa (ui/cocoa.m:1804-1810) and GTK (ui/gtk-clipboard.c:51-57) wait with no bound, and an empty answer ends both waits (ui/clipboard.c:249-253). The roast's "a killed agent closes the port" holds, through the bus-master clear rather than a reset:

  • tear_down clears bus mastering first (kernel/src/pcidev/mod.rs:1454);
  • QEMU drops DRIVER_OK when it is cleared (hw/virtio/virtio-pci.c:811-815);
  • virtio-serial then closes the guest end (hw/char/virtio-serial-bus.c:622-623, :585-600, hw/char/virtio-console.c:117-118);
  • the vdagent unregisters (ui/vdagent.c:874-884).

  Answering is not sufficient: see BLOCKER 3.

• The crash. This configuration does not cause it. QEMU 11.1.1's info virtio-status dereferences NULL on every virtio device (NOTE 1). cargo run never sends that command, so the crash gives no reason to call the launch change unsafe.

• QEMU builds without qemu-vdagent. See NOTE 2.

BLOCKER

1. PR body §3, the bridge table's "host capabilities" row against its "host CLIPBOARD" row — the answer to the opening REQUEST is refused, so the host's text from before boot never lands — Why: no grab precedes that REQUEST, so the answer arrives while the owner is None, and the table's own rule sets the guest clipboard "only when the owner is Host". Fix: the opening REQUEST makes the owner Host, as a host grab would. If the owner is already Guest, because a copy was made between WATCH and the port's open, the agent grabs instead. Say whether WATCH delivers the clipboard the compositor already holds. Name the unit test of each transition.

2. PR body §3 wire, "capabilities (CLIPBOARD_BY_DEMAND alone)", and §"Linux's guest", "departs from it in one place" — when a host copy and a guest copy cross, both clipboards end up empty — Why: this is a race, and a departure from the state of the art (A design follows what its hardware's manufacturers or the published state of the art back, and a bench verdict never blocks it #885) that the body neither follows nor cites.

   • spice-vdagent announces CLIPBOARD_GRAB_SERIAL (src/vdagentd/vdagentd.c:160) and orders crossing grabs by it (:290-305, :715-727). QEMU honours it by discarding a lower-ordered guest grab (ui/vdagent.c:518-530).
   • Without it, each side believes the other owns. The guest, holding Host, requests, and QEMU answers empty because the vdagent owns (ui/vdagent.c:569-582). A host paste then asks the guest, which answers with no data.

   Fix: announce CLIPBOARD_GRAB_SERIAL and keep vdagentd's serial rule. That includes QEMU's serial reset, which closes and reopens the host end (ui/vdagent.c:455-466), so the serial module must handle a host PORT_OPEN(0). Otherwise, show that the crossing cannot happen.

3. PR body, Roast, "The agent's rule of answering every request is the only defence" — that defence fails whenever the guest is not running — Why: QEMU's peers wait on the UI thread for a guest that cannot run, so a host paste of guest-owned text freezes QEMU's window, and the paste gets nothing:

   • cargo run --debug parks a triple-faulted guest with -action shutdown=pause (build/qemu.rs:286), and the window stays frozen for the rest of the session;
   • a panicked kernel holds its panel for 60 s before it resets (lib/tco/src/lib.rs:137), and the window is frozen that long;
   • a gdb stop on the always-on -s (build/qemu.rs:269) freezes the window until the debugger continues.

   Fix: weigh sending the text right after each GRAB. QEMU stores a CLIPBOARD that its owner sends unrequested (ui/vdagent.c:585-605, ui/clipboard.c:232-257), and after that qemu_clipboard_request returns at once, so neither loop waits. Weigh it against the protocol's by-demand reading. The design already relies on QEMU's implementation for the opening REQUEST. Choose one, and record whatever stays in issues/ with an owner and an exit.

4. PR body §4, "A refused send drops the watcher by name", and the roast's "the bridge is gone for the boot" — any program can end the bridge for the boot — Why: every program holding compositor can do it by copying in a loop, and every installed package holds it (images/system.toml:37). Until the drop, each copy past 4096 bytes pins a fresh region of up to 2 MiB in the pipe. The roast found this compromise and neither removes nor records it. Fix: keep one notification outstanding. The compositor keeps only a changed flag, and it sends the latest non-bridge clipboard when the watcher asks for the next one. Then the pipe never fills, and the server doctrine's drop never fires on a live watcher.

5. PR body §2(a) and §9, "A console chosen wrong is a boot with no console" — no test can fail on rule (a), and none covers rule (b)'s refusals — Why:

   • The launch creates the console's function before the agent's, so the console enumerates first (qemu-shape.log:47-60: the console at 00:01.0, the agent's at 00:02.0). The existing first match (kernel/src/drivers/virtio_console.rs:199) already takes it.
   • The mutation "drop rule (a) and keep devices.iter().find(|d| d.is_id(VIRTIO_VENDOR, VIRTIO_CONSOLE_DEVICE))" therefore stays green on every boot.
   • Rule (b) has three outcomes: two undriven matches give Ambiguous, all driven gives KernelDriven, and one undriven among driven matches is chosen. Only the smoke test, through its success path, would fail on a missing (b).

   Fix: write (b)'s resolution as a pure decision beside toyos_pci::slot (kernel/pci/src/slot.rs), host-tested on all three outcomes. Write (a)'s choice over the functions' feature words the same way, or make the launch create the agent's function first so that every boot runs rule (a).

NOTE

1. PR body, "What I am unsure of", last bullet — "which step faults is not known" — Why: QEMU's source answers it, and the evidence has no clean run of the shape.

   • QEMU 11.1.1's qmp_x_query_virtio_status never sets backend_features (hw/virtio/virtio-qmp.c:697-768); v10.0.0 set it at :749-750.
   • hmp_virtio_status prints "Backend features:" and then reads features->transports through that NULL (hw/virtio/virtio-hmp-cmds.c:154-155, :54).
   • The last line before both EXIT=139 results is that header (qemu-shape.log:109-110, :182-183). It faults the same way on the console function as on the agent's.

   Still, no run of the shape exits 0. The build round's evidence should carry the same command line with only info pci and quit, exiting 0.

2. PR body, "What I am unsure of", CI and Windows — qemu-vdagent is unmeasured on both — Why:

   • CI. Debian builds QEMU with spice on amd64 (debian/control:93-94 of qemu 1:11.1.2+ds-1, one revision past the pin). vdagent.c compiles into the main binary whenever spice-protocol is found (ui/meson.build:68). So CI very likely has it, but nobody has measured it. "The image moves" cannot add a compile option to the Debian 11.1.1 build that build/ci/qemu-version pins, so measure in the pinned image now: that decides whether the harness half of the design exists at all.
   • Windows. Nothing falls back by design, so a Windows QEMU built without spice-protocol stops cargo run at launch. Cite the Windows build's spice protocol support line (meson.build:4915) or measure it. If the Windows build lacks it, file that gap.

3. PR body §1, -display gtk,clipboard=on — the body cites the GTK clipboard's off default but not why it is off — Why:

   • Upstream shipped GTK's clipboard as "EXPERIMENTAL, MAY HANG" through 11.0 (meson_options.txt:270-271 at v11.0.0), and 11.1 still defaults it off (qapi/ui.json:1322-1323).
   • No Linux or Windows host has run it.
   • A Linux QEMU without the GTK module (Debian packages it in qemu-system-gui) now stops cargo run at launch.

   Name which hang it is, and that BLOCKER 3's choice closes it. Have the build's prerequisite check name the spice-protocol and GTK requirements before a launch can fail.

4. PR body §8, "rules 2(a) and 2(b) find no virtio function there and change nothing" — this is false for rule (b) — Why: (b) changes how every claim resolves on every machine, not only virtio claims. The T14 run that the build round asks for should carry its inventory, showing that no vendor:device named in a devices list is carried by two functions.

5. kernel/src/pcidev/mod.rs:146 — MAX_FUNCTIONS is 4, and the agent takes the last free slot on every cargo run boot — Why: a cargo run virtio boot already claims three functions (images/system.toml:85, :95, :235). QEMU adds PM and FLR only on an Express port (hw/virtio/virtio-pci.c:2197-2198, :2260-2307), so the kernel finds no reset for a root-bus virtio function. An agent that ended therefore keeps its slot as residue (kernel/pci/src/slot.rs:4-9). The body says none of this.

6. PR body §9, "one smoke assertion, on a boot the guest suite already runs" — the smoke test names no boot — Why: the shared boot starts no compositor (tests/images/testcases.toml:3), and the agent opens the port only after it holds the compositor (§3, "The loop"). Name a boot that runs the compositor, and add the row to that boot's config.

7. PR body §3, wire, and serial's control-message decoding — both decode words from outside the guest's trust — Why: Fit gives an input boundary a pure crate of its own whoever uses it, as toyos-acpi and toyos-elf are. Here both are modules of system/vdagent. That crate is also where max_ports = 2 and com.redhat.spice.0 can be declared once and read by both the launch and the agent. Today each is two declarations.

8. sdk/window/src/lib.rs:458-471 — the bridge's set reuses clipboard_set's exchange, which blocks — Why: past 4096 bytes the exchange blocks in recv_header with no bound, inside the agent's single loop. A stalled compositor then stalls every host request the agent owes, which contradicts §3's "There is no flat wait". Route the set through the Poller, or name its bound.

9. PR body §7, "no secret is newly exposed" — the eager fetch exposes host secrets to the guest — Why: every host copy, including a password manager's, moves into guest memory whether or not it is pasted there, and every cargo run boot pulls in the host's current clipboard. State that neither the agent nor the compositor ever logs a clipboard byte.

10. PR body §2, "which the brief does not name" — the build round would fall outside its fence — Why: the comment that accepts this design must widen the fence by name, to kernel/src/drivers/virtio_console.rs, kernel/src/pcidev/mod.rs and kernel/pci/.

DESIGN NEEDS CHANGES: let the opening REQUEST's answer land (owner Host, or GRAB if the guest already copied); announce CLIPBOARD_GRAB_SERIAL with vdagentd's serial rule and QEMU's close-and-reopen, or show the crossing cannot happen; answer the halted guest (push the text after GRAB, or record the freeze with an owner and exit); give the watcher one outstanding notification so no client can drop it; make rules (a) and (b) pure decisions with host tests that fail without them; and fix NOTEs 1–10 before landing.

[Japabu]

The orchestrator's rulings on round 1 (#894 (comment)), following spice-vdagent and QEMU per #885's rule:

• B1: the opening REQUEST makes the owner Host, or the agent grabs if the guest already copied. Say what WATCH delivers, and unit-test each transition.
• B2: announce CLIPBOARD_GRAB_SERIAL, keep vdagentd's serial rule, and handle QEMU's close-and-reopen (PORT_OPEN(0)).
• B3: push the text right after each GRAB. QEMU stores an owner's unrequested CLIPBOARD, so a halted guest (debug pause, panic panel, gdb stop) never freezes QEMU's window. That beats the by-demand reading here: the host is the developer's own machine, and the guest's text going to it eagerly costs nothing the owner minds.
• B4: one outstanding notification. The compositor keeps a changed flag and sends the latest copy when the watcher asks.
• B5: rules (a) and (b) become pure decisions with host tests that fail without them.
• NOTEs 1–10 as the review names them, including the clean QEMU run with exit 0, measuring qemu-vdagent in CI's pinned image, a prerequisite check that names spice-protocol and GTK, the T14 inventory under rule (b), the MAX_FUNCTIONS residue, the boot the smoke test runs on, a pure codec crate, a bounded set, and never logging a clipboard byte.
• The build's fence is widened to kernel/src/drivers/virtio_console.rs, kernel/src/pcidev/mod.rs and kernel/pci/.

[Japabu]

Design round 2, head 0b12adca4 (unchanged: still one empty commit). The body is rewritten. This maps each finding of the round-1 review (#894 (comment)) to the place in the body that closes it, as ruled (#894 (comment)).

New measurements, all on ToyOSOrg/evidence pr-894 at 7b1ccc19:

• qemu-shape-r2-devhost.log;
• qemu-shape-r2-ci-image.log;
• qemu-windows-msys2.log.

The owner's later direction (ToyOS does not follow Linux). The design now rests on three definitions only: virtio 1.2, spice-protocol's vd_agent.h, and QEMU v11.1.1, which makes the device and the host end. Round 1 leaned on the SPICE project's Linux guest agent for the chunk bound, the per-type sizes and one-chunk-per-message, and on Linux's console driver for queue setup before DRIVER_OK. Each is now taken from vd_agent.h or from virtio 1.2 §3.1.1 and §5.3.5. One behaviour changed: outgoing messages are chunked at VD_AGENT_MAX_DATA_SIZE (2048), not sent as one chunk.

BLOCKER

1. The opening REQUEST's answer lands. §4's table: at the host's capabilities the owner becomes Host before the REQUEST goes out, or the bridge grabs and pushes if the owner is already Guest. §5 says what WATCH delivers: the clipboard the compositor holds, as soon as a client's set is unsent, which at boot is a copy made before the agent watched. Every row has a unit test, named in §10, among them the_answer_to_the_opening_request_lands and the_opening_grabs_a_copy_made_before_the_port_opened.
2. CLIPBOARD_GRAB_SERIAL is announced (§3), and QEMU's close-and-reopen is handled: the host's PORT_OPEN(0) is answered with ours, and its PORT_OPEN(1) with ours (§4).
   • One departure from the ruling's wording, by the owner's later direction. The ruling said "keep vdagentd's serial rule", and vdagentd is that Linux agent. So the bridge's rule is derived from QEMU's host side instead (ui/vdagent.c:342-351, :516-530): a host grab is kept iff its serial is at least the guest's last grab's.
   • §4 shows why both sides then pick the same winner of a crossing.
   • On every serial sequence QEMU 11.1.1 sends, it keeps and discards the same grabs as the rule the ruling named. It differs only on a host serial QEMU does not send.
3. The text is pushed right after each GRAB, in the same transmit buffer (§4). QEMU handles one buffer under the lock a host paste needs, so no paste falls between the two. Cocoa's and GTK's waits end at once, a halted guest no longer freezes the host's window, and nothing is left to record.
4. One outstanding notification (§5): MSG_CLIPBOARD_WATCH asks, and the compositor answers with the latest client copy while its unsent flag is set, so no client can fill the watcher's pipe.
   • While designing this I found a race round 1 also had. A bridge set crossing a delivery leaves the guest with the host's text and the host with the guest's copy.
   • The bridge's sets now travel on the watcher's own ordered connection, and land only while its ask is outstanding.
   • Unit test: a_bridge_set_made_before_a_delivery_was_seen_is_refused.
5. Rules (a) and (b) are pure decisions in toyos_pci with host tests that fail without them (§2, §10). The launch also creates the agent's function before the console's (§1), so every virtio guest boot runs rule (a). Measured: the agent's function at 00:06.0 before the console's at 00:07.0 on q35. The harness's other slots do not move.

NOTE

1. A clean run exits 0. Round 1's command line with only info pci and quit exits 0, and so does cargo run's Virtio profile as designed, on x86_64 and aarch64. That holds on the development host's QEMU 11.1.1 and in CI's pinned image.
2. CI and Windows are measured.
   • CI's pinned image (debian@sha256:a2aa4626…, its snapshot's QEMU 1:11.1.1+ds-1) lists qemu-vdagent for both architectures, so the harness half of the design stands. The image was run under the development host's podman VM, emulated.
   • Windows: MSYS2's QEMU 11.1.2 was built with spice-protocol 0.14.5, and its binaries carry chardev-qemu-vdagent and GTK's clipboard peer. Stefan Weil's installers are not measured, which "What I am unsure of" says.
3. The GTK hang is named. 29e0bfffab87 turned GTK's clipboard off for guest hangs (QEMU #1150). Its synchronous host reads became asynchronous in 11.1 (9a007326e83f), and the option became -display gtk,clipboard=on, still off (335e32cbd0a2). The one wait left, gd_clipboard_get_data's loop, is the one BLOCKER 3's push closes. The prerequisite check names spice-protocol, and GTK for cargo run off macOS, before any launch (§1).
4. Rule (b) is now stated as changing every claim on every machine (§9). The build round's T14 run carries the PCI inventory against every pci: entry in images/system.toml.
5. The slot residue is stated (Roast, "Claim slots"). A cargo run boot claims all four slots, and an ended agent keeps its slot as netstack's and soundserver's do. More slots reach both architectures' interrupt tables, outside the fence, so the build round files it with an exit.
6. The smoke row has a boot. None exists that has both a compositor and a virtio function. A new tests/images/clipboardcase.toml on HeadlessVirtioGpu starts the compositor and the agent, and the row waits for the agent's open line and for the length of QEMU's answer to the opening REQUEST (§10).
7. There is a codec crate. toyos-vdagent (lib/vdagent) decodes and encodes the control messages and the vdagent stream. It also declares the port name and max_ports = 2 once, for the launch and the agent (§3).
8. The bridge's set no longer blocks. It goes over the watcher's connection with try_send, and MSG_COPY_REGION arrives as a Poller event, so nothing waits and there is nothing to bound (§5). clipboard_set's blocking exchange stays the apps'.
9. The secrets that cross are stated (§8): every host copy enters guest memory, a password manager's included, and every boot pulls in the host's clipboard. Neither the agent nor the compositor logs a clipboard byte.
10. The fence. The ruling widened it to kernel/src/drivers/virtio_console.rs, kernel/src/pcidev/mod.rs and kernel/pci/. The build also needs kernel/src/drivers/virtio.rs. Rule (a) has to read a feature word before bus mastering, VirtioPciConfig::parse is private there, and VirtioDevice::init enables bus mastering, which marks a function kernel-driven, before it reads any feature (§2). The build also needs tests/images/clipboardcase.toml and two new issues/ files.