Beanjamin Module

The viam:beanjamin module provides these models for arm-based automation workflows:

1. viam:beanjamin:coffee - A generic service that orchestrates a full coffee brew cycle by sequentially moving through all poses on a pose switcher.
2. viam:beanjamin:multi-poses-execution-switch - A switch component that moves an arm between predefined poses using the Motion service.
3. viam:beanjamin:maintenance-sensor - A sensor component that reports whether the system is safe for maintenance (arm idle, no orders running or queued).
4. viam:beanjamin:order-sensor - A sensor that yields one reading per completed order (start/end timestamps and outcome) when wired from the coffee service.
5. viam:beanjamin:dial-control-motion - A generic service that translates Stream Deck dial inputs into relative arm motions.
6. viam:beanjamin:customer-detector - A generic service that identifies return customers via facial recognition using the viam:vision:face-identification vision service.

---

Model: viam:beanjamin:multi-poses-execution-switch

API: rdk:component:switch

Moves an arm (or any movable component) between a list of named poses via the Motion service. Each "position" of the switch corresponds to a pose. Only one movement can execute at a time.

Configuration

{
  "component_name": "<string>",
  "motion": "<string>",
  "reference_frame": "<string>",
  "poses": [
    {
      "pose_name": "<string>",
      "pose_value": { ... }
    }
  ]
}

Name	Type	Required	Description
component_name	string	Yes	Name of the arm component to move.
motion	string	Yes	Name of the motion service (typically "builtin").
reference_frame	string	No	Reference frame for poses. Defaults to "world".
poses	array	Yes	One or more named poses. Pose names must be unique.

Defining poses

Each pose in the poses array must have a pose_name and exactly one of two definition styles:

Absolute pose (pose_value)

Define the pose directly with position and orientation coordinates:

{
  "pose_name": "home",
  "pose_value": {
    "x": 0, "y": 0, "z": 500,
    "o_x": 0, "o_y": 0, "o_z": 1,
    "theta": 0
  }
}

Pose value fields: x, y, z are in millimeters. o_x, o_y, o_z define the orientation axis, theta is the rotation angle in degrees.

Relative pose (baseline)

Define a pose relative to another pose in the same poses array. Optionally add a translation (offset added to the baseline position) and/or an orientation (replaces the baseline orientation entirely). The baseline can appear anywhere in the array — before or after the pose that references it.

{
  "pose_name": "left-of-home",
  "baseline": "home",
  "translation": { "x": -100 }
}

Field	Type	Required	Description
baseline	string	Yes (instead of pose_value)	Name of another pose in the poses array.
translation	object	No	Position offset added to the baseline. Fields: x, y, z (millimeters along world axes, default 0), and along_orientation (millimeters along the baseline's normalized orientation vector, default 0).
orientation	object	No	Orientation that replaces the baseline orientation. Fields: o_x, o_y, o_z, theta.

The along_orientation component is projected onto the baseline's orientation vector, not onto any orientation override set on the same pose — translation is applied before the orientation replace. If the baseline's orientation vector has zero norm, the along_orientation offset is silently skipped.

Baselines can be chained — a relative pose can itself be used as a baseline for another pose. Multiple poses can share the same baseline.

Validation rules:

• A pose must have either pose_value or baseline, not both.
• translation and orientation are only allowed with baseline.
• The baseline must reference an existing pose_name in the poses array.
• Circular baseline references are not allowed (e.g. A → B → A).

Example Configuration

{
  "component_name": "my-arm",
  "motion": "builtin",
  "reference_frame": "world",
  "poses": [
    {
      "pose_name": "home",
      "pose_value": {
        "x": 0, "y": 0, "z": 500,
        "o_x": 0, "o_y": 0, "o_z": 1,
        "theta": 0
      }
    },
    {
      "pose_name": "above-home",
      "baseline": "home",
      "translation": { "z": 100 }
    },
    {
      "pose_name": "backed-off-home",
      "baseline": "home",
      "translation": { "along_orientation": -50 }
    },
    {
      "pose_name": "pour",
      "baseline": "home",
      "translation": { "x": 200, "y": 100, "z": -150 },
      "orientation": { "o_x": 0, "o_y": 1, "o_z": 0, "theta": 90 }
    }
  ]
}

In this example:

• home is defined absolutely at (0, 0, 500) with orientation (0, 0, 1, 0°).
• above-home inherits home's position and orientation, then adds z: +100 → final position (0, 0, 600).
• backed-off-home inherits home's pose and translates -50 mm along home's orientation vector (0, 0, 1) → final position (0, 0, 450).
• pour inherits home's position, adds a translation → (200, 100, 350), and overrides the orientation to (0, 1, 0, 90°).

Switch Interface

Method	Description
GetNumberOfPositions	Returns the total number of poses and their names.
GetPosition	Returns the index of the current pose (0-based).
SetPosition(index)	Moves the arm to the pose at the given index.

DoCommand

set_position_by_name - Move to a pose by name.

{ "set_position_by_name": "home" }

get_current_position_name - Get the name of the current pose.

{ "get_current_position_name": true }

Returns:

{ "position_name": "home" }

get_pose_by_name - Get the pose coordinates, reference frame, and component name for a named pose.

{ "get_pose_by_name": "home" }

Returns:

{
  "x": 0, "y": 0, "z": 500,
  "o_x": 0, "o_y": 0, "o_z": 1,
  "theta": 0,
  "reference_frame": "world",
  "component_name": "my-arm"
}

---

Model: viam:beanjamin:coffee

API: rdk:service:generic

Orchestrates a full coffee brew cycle using a multi-poses-execution-switch component. Supports preparing espresso and lungo orders, executing individual actions, stopping a run (cancel), and driving the arm back to a clean start (rewind).

has_separate_brew_buttons selects which coffee machine the arm is driving, and with it which claw poses the switcher must carry:

	false (default)	true
Hardware	one toggle switch	one momentary button per shot size
Claw poses required	coffee_button_approach, coffee_button_on, coffee_button_off	espresso_button_approach / _press, lungo_button_approach / _press
Who sets the dose	the hold duration — brew_time_sec / lungo_brew_time_sec	the machine; the brew times only wait the pour out
Extra actions	turn_coffee_button_on, turn_coffee_button_off	press_espresso_button, press_lungo_button, brew_lungo

Only the selected machine's poses are validated at startup, and only its actions are registered for execute_action. Under true, both shot sizes are required even if only one is on the menu — the drink is known per-order, so a switcher that cannot reach the lungo button is misconfigured regardless of what is queued.

When keepalive is configured, the filter pose switcher must additionally carry purge_approach and purge_press. These are filter-frame poses, not claw-frame ones: the arm keeps the portafilter in its claws and presses the machine's 1 CUP button with the assembly in hand, so nothing is parked in the group head and no filter basket gets wet. Author purge_press so the press is a straight-in linear move from purge_approach. See "Keeping the machine at brew temperature" below.

Configuration

{
  "pose_switcher_name": "multi-pose-execution-switch",
  "claws_pose_switcher_name": "claws-switch",
  "arm_name": "my-arm",
  "gripper_name": "my-gripper",
  "speech_service_name": "speech",
  "has_separate_brew_buttons": true,
  "brew_time_sec": 25,
  "lungo_brew_time_sec": 40,
  "button_press_hold_sec": 0.5,
  "grind_time_sec": 7.5,
  "slow_movement_vel_degs_per_sec": 25,
  "portafilter_shake_sec": 2.5,
  "lock_overshoot_degs": 3,
  "save_motion_requests_dir": "/tmp/motion-requests",
  "order_sensor_name": "order-events",
  "cam_storage_mux_name": "video-store-mux",
  "slack_notifier_name": "slack-notifier",
  "cup_vision_service_name": "cup-vision",
  "src_camera_name": "cam",
  "camera_observe_pose_switcher_name": "camera-observe-switch",
  "cup_approach_relative_pose": { "x": -80, "y": 0, "z": 0, "o_x": 0, "o_y": 0, "o_z": 1, "theta": 0 },
  "cup_grab_relative_pose": { "x": -20, "y": 0, "z": 0, "o_x": 0, "o_y": 0, "o_z": 1, "theta": 0 },
  "serving_approach_relative_pose": { "x": -80, "y": 0, "z": 0, "o_x": 0, "o_y": 0, "o_z": 1, "theta": 0 },
  "serving_grab_relative_pose": { "x": -20, "y": 0, "z": 0, "o_x": 0, "o_y": 0, "o_z": 1, "theta": 0 },
  "input_range_override": {
    "my-arm": {
      "5": { "min_degs": -270, "max_degs": 270 }
    }
  }
}

Add a viam:beanjamin:order-sensor component to the machine, put it in the coffee service depends_on, and set order_sensor_name to that component’s name. When an order attempt finishes, one reading is queued with start_time, end_time, order_ok, duration_ms, and — for observability — failed_step, operator_cancelled, trace_id, the decaf path flag, and error_message (if applicable).

Usage sensor. The optional usage_sensor_name field points at a single sensor resource that holds several counters, one per key, updated through the brew lifecycle. Setting the field automatically registers the sensor as a dependency of the coffee service, so no manual depends_on entry is required. The sensor must support both the Readings API and a DoCommand({"set": {<key>: <value>}}) that overwrites the named counter (and preserves the others). The coffee service updates each counter with a best-effort read-modify-write: it reads the current value via Readings, computes the new value, and writes it back via DoCommand. The keys are:

• regular_grinds — +1 after each regular (non-decaf) grind
• decaf_grinds — +1 after each decaf grind
• usage — +1 after a regular brew (espresso/decaf), +1.5 after a lungo brew (lungo/decaf_lungo, and the iced drinks, which pour a lungo)
• cleanings — +1 after each cleaning cycle
• ice_dispenses — +1 after each ice dispense (iced coffee only)
• ice_dispense_timeouts — +1 when a watched ice dispense hits either ceiling — ice_dispense_max_sec from the pin opening, or ice_after_first_seen_max_sec from the first sighting — without the surface passing the stop row (ice_vision_enabled only). The glass is served as it is, so a climbing counter is how an empty hopper — or a stop row that no longer matches how the glass is being gripped — shows up. The log line names which ceiling fired, and the saved frame is tagged ice_timeout or ice_surface_cap.
• ice_shadow_disagreements — +1 per watched dispense where the brightness shadow and the contrast step differed in kind rather than in timing: the shadow never saw ice, saw it but never reached the stop row, or reached the stop row on the first surface it ever saw. The last means it locked onto something already there — the rim or a reflection — and is the one that would rule out ever promoting it. Timing gaps are expected and are not counted. See "Brightness shadow" below.
• milk_pours — +1 after each milk pour (iced latte only)
• drip_tray_brews — +1 after each brew
• espresso_cups_used — +1 after the brew cup is placed, on every drink
• latte_glasses_used — +1 after a latte glass is fetched, on iced drinks only (iced_coffee/iced_latte)
• successful_consecutive_orders — +1 after each successful order, reset to 0 after any failed or operator-cancelled order

Consumable counters increment only after their step completes successfully, so a brew that fails partway leaves the consumables it actually used counted and does not roll them back. A missing counter key is treated as 0 (so the first update lands a fresh count). All updates are best-effort: a read/write failure logs a warning and never fails the brew. When usage_sensor_name is unset, every update is skipped.

Configure a viam:video:storage camera on the machine. After each order attempt, the coffee service saves a clip via a save DoCommand issued from a background goroutine, so it never blocks the queue. Each clip includes a fixed N seconds of pre-roll (ring-buffer permitting) and N seconds of post-roll. The save is synchronous (async: false) so slice failures surface in the logs instead of being dropped silently; because a synchronous slice can only read segment files that have already closed on disk, the goroutine waits roughly one video-store segment (~30s) past the clip's end before issuing the save.

The save request includes a tags entry with the order UUID — this is what links clips to orders for cloud data filtering — and a minimal JSON metadata blob containing only order_id and order_status (ok or failed), which the video-store appends to the clip filename. Clips are saved after every attempt, including failed brews or panics. Failure detail (the error and the step it failed at) is not stored in the clip metadata; it is recorded separately on the order sensor.

Slack notifications. The optional slack_notifier_name field points at a viam:notifications:slack generic service. Setting the field automatically registers it as a dependency of the coffee service, so no manual depends_on entry is required. When set, the coffee service sends a best-effort Slack message on every non-successful order attempt — both genuine faults and operator cancels — via DoCommand({"command": "send", "blocks": [...], "text": ...}). The message is laid out with Slack Block Kit and mirrors the per-attempt fields the order sensor records, so it's a self-contained record: a header that distinguishes a fault (:x: Order failed) from an operator cancel (:warning: Order cancelled by operator), a fields section with the drink, customer, the step it failed (or was cancelled) at, the duration, and the decaf flag, the error in a code block (faults only), and a context footer with the order ID, trace ID, start time, and — when the module is cloud-connected — clickable app.viam.com deep-links to this machine's logs (built from the VIAM_MACHINE_ID / VIAM_PRIMARY_ORG_ID environment variables Viam injects) and, when cam_storage_mux_name is configured, to the order's video clip (a data page filtered by the order-ID tag, scoped to VIAM_LOCATION_ID). All links are omitted on a local or test machine where those environment variables are unset. Because the clip uploads asynchronously after the notification is sent, the clip link may show no results for the first ~15–60s. The flat text value is sent alongside as the notification/accessibility fallback Slack uses when blocks can't render. Sends run off the queue goroutine (so a slow Slack call never stalls the next order) and are bounded by a 10-second timeout; a send failure logs a warning and never affects brewing. The Slack channel/credentials (bot token or webhook URL) are configured on the notifier service itself. When slack_notifier_name is unset, no notifications are sent.

Top-level fields:

Name	Type	Required	Description
pose_switcher_name	string	Yes	Name of the multi-poses-execution-switch component.
claws_pose_switcher_name	string	Yes	Name of the claws pose switcher component.
arm_name	string	Yes	Name of the arm component used for motion planning and execution.
gripper_name	string	Yes	Name of the gripper component.
speech_service_name	string	No	Name of a text-to-speech generic service for spoken greetings.
has_separate_brew_buttons	bool	No	true when the machine has one momentary button per shot size (poked, then released); false (default) for a single toggle switch the claw holds down for the whole brew. Changes which claw poses are required and which actions exist — see the table above.
brew_time_sec	float	No	Espresso brew duration in seconds (default: 8). Under a toggle this is the hold time and therefore the dose; with has_separate_brew_buttons the machine controls the dose and this must be at least as long as its actual pour, or the arm reaches for the cup mid-stream.
lungo_brew_time_sec	float	No	Same, for a lungo — also used by iced_coffee and iced_latte, which pour a lungo (default: 15).
button_press_hold_sec	float	No	How long the claw dwells on a brew button so the momentary switch registers, in seconds (must be >= 0; unset means no dwell). Raise if presses are intermittent. Ignored unless has_separate_brew_buttons is set.
grind_time_sec	float	No	Bean grinding duration in seconds, applied to both regular and decaf grinders (default: 7.5).
gripper_hold_min_pos	float	No	Gripper jaw position (0–850) below which the gripper is considered closed/empty. Positions in [min, max] mean an object (cup or glass) is held; used to verify grabs and self-heal an open gripper at brew-cycle start (default: 430).
gripper_hold_max_pos	float	No	Gripper jaw position (0–850) above which the gripper is considered open (default: 685).
gripper_open_timeout_sec	float	No	How long the portafilter handoff waits for the jaws to actually read open before giving up (default: 3). release_filter and grab_filter both open the jaws and then travel along the filter handle at claw clearance, so moving while the jaws are still closing drags the filter against the bayonet on the way out or strikes the handle on the way in. Rather than sleeping a fixed interval, both poll the real jaw position every 500ms until it clears gripper_hold_max_pos and fail with a "gripper did not open" error at this timeout. Raise it on a build whose jaws travel slowly — clamp force and jaw range both lengthen the open stroke.
slow_movement_vel_degs_per_sec	float	No	Max joint velocity (degrees/sec) for the slow tier: a step with a LinearConstraint, a pivot or circular motion, and the spill/scatter-prone carries — the no-spill cup/glass carry, the milk bottle, and loose grounds on the way to the tamper. Everything else (an ordinary empty traverse, the tamped puck) runs at the arm's own default speed. Raise carefully — precision, contact, and the spill-free carries all rely on this (default: 25).
slow_movement_acc_degs_per_sec2	float	No	Max joint acceleration (degrees/sec²) for the slow tier. Gentle acceleration matters more than top speed for both precision and sloshing, so this defaults to a value below the arm's own (default: 25). Lower it further to ease into and out of contact and carries.
portafilter_shake_sec	float	No	Duration in seconds of each circular shake during unlock_portafilter, to dislodge a stuck puck. The arm shakes twice, once at coffee_shake and once at coffee_shake_left, returning to coffee_in in between so both shakes run the same move out of the group head. A puck stuck on either side of the basket then gets worked from both. When set, requires both poses in the filter pose switcher; author them to lean opposite ways and keep the tilted filter clear of the group head. Defaults to 0, which skips the shakes and the travel to those poses: the arm withdraws straight from coffee_in to coffee_approach.
lock_overshoot_degs	float	No	Extra rotation applied to the coffee_locked_final lock pivot, unwound back onto the authored angle within the same planned trajectory. The claws slip on the portafilter handle once the bayonet is under load, so the arm has to over-rotate for the filter to seat; the unwind re-zeroes the grip, since a seated filter out-holds the claws and the handle slides back through them. Defaults to 0 (no overshoot) — tune it on the machine.
save_motion_requests_dir	string	No	Directory to save debugging payloads. Each plan writes a single request+response JSON (RDK's WriteRequestAndResponseToFile; readable back with ReadRequestAndResponseFromFile, and the response is absent when planning failed), nested under tag=<order-id>/tag=step_<step>/tag=motion_<move|pivot|circular|carry>/tag=planning_<success|failure>/. When this directory is a Viam data-synced capture dir, the data manager reads those tag= segments and tags each uploaded file, so plans are searchable on the data page by order, step, motion type, and planning outcome — and a failed order's Slack notification deep-links to that order's plan requests (which the reader can narrow to planning_failure or a specific step). Also writes a visualization_snapshot_<timestamp>_<cup|glass>.pb.gz snapshot on each cup/glass observation: the whole frame system resolved at the joint configuration the arm held when the photo was taken, plus every detection's point cloud (as captured, anchored at the camera's world pose) and the world-frame bounding box the grasp was derived from. Drag the file onto a visualization viewer to replay the observation — the visualization_snapshot prefix is what its drag-and-drop loader keys off.
order_sensor_name	string	No	Name of a viam:beanjamin:order-sensor sensor to notify when each order attempt completes (must appear in depends_on).
usage_sensor_name	string	No	Name of a single sensor whose per-key counters are updated through the brew lifecycle: regular_grinds, decaf_grinds, usage, cleanings, ice_dispenses, ice_dispense_timeouts, ice_shadow_disagreements, drip_tray_brews, espresso_cups_used, latte_glasses_used, and successful_consecutive_orders. See "Usage sensor" below.
cam_storage_mux_name	string	No	Name of a viam:multiplexer:resource-multiplexer generic service whose dependencies are viam:video:storage stores; when set, saves a clip per order attempt (synchronous save) to all configured stores.
data_dir	string	No	Directory for persistent module data. When set alongside cam_storage_mux_name, a pending-clip record is written under <data_dir>/pending-clips when each order starts and removed only once that order's clip has been saved successfully — a save that fails (or never runs because the process died first) leaves the record in place. Use with a Viam scheduled job calling cleanup_pending_clips to recover clips for any order whose save was interrupted or failed.
slack_notifier_name	string	No	Name of a viam:notifications:slack generic service. When set, the coffee service sends a best-effort Slack message on every non-successful order attempt (faults and operator cancels). See "Slack notifications" above.
chore_wheel	object	No	Roster and chores for the weekly maintenance rota posted by send_weekly_chores: { "people": [...], "chores": [...] }. Requires slack_notifier_name. See "Weekly chore wheel in Slack" below.
customer_detector_name	string	No	Name of a viam:beanjamin:customer-detector service. When set, the coffee service credits each successfully completed order (when the prepare_order carried a customer_email) to that customer's order history via the detector's record_order DoCommand, powering "the usual". Setting the field automatically registers it as a dependency. Unset disables order-history recording.
delivery_handler_name	string	No	Name of a generic service on a peer delivery machine, reached through a remote part (e.g. "delivery-bot:mission-control" after adding the peer machine as a remote named delivery-bot in app.viam.com). Setting the field automatically registers it as an (optional) dependency. When a fulfillment: "delivery" order's drink lands in the serving area, the coffee service sends that service a delivery_request DoCommand (see send_delivery_message below for the payload and the manual test hook). Unset disables outbound peer messaging; delivery orders then just announce and rely on pickup from the serving area.
input_range_override	object	No	Narrows joint limits on named frames before motion planning. Outer key is the frame name (typically the arm); inner key is either the joint name or its stringified index (e.g. "5" for the last joint of a 6-DoF arm). Each value is { "min_degs": number, "max_degs": number }.
conversational	bool	No	When true, the coffee service speaks its own greetings, almost-ready prompts, order-received lines, and rejection quips through speech_service_name. When false (default), the service stays silent except for the drink-ready announcement at cup handoff — leaving the rest of the talking to an external orchestrator (e.g. viam:conversation-bundle:voice-command).
cup_vision_service_name	string	Yes	Name of a rdk:service:vision segmenter that returns cup detections via GetObjectPointClouds. Cup pickup is always vision-guided — the arm detects the empty cup rather than grabbing from a fixed pose.
src_camera_name	string	Yes	Source camera the vision service segments from. Must be present in the frame system.
camera_observe_pose_switcher_name	string	Yes	Switcher holding the camera observation vantages. Poses are swept one at a time and vision run at each (within-pose near-duplicates within 40 mm collapsed); at each pose that sees a cup the arm tries to grab the candidates closest-first, and the sweep stops as soon as a cup is in hand. When a pose's cups are all unreachable the sweep continues to the remaining poses, so a cup reachable only from a later vantage is still found before the machine gives up. Must include a pose named cup_observe (the home/recovery pose), and all poses must move the cam frame (set the switch's component_name to cam)
cup_approach_relative_pose	object	Yes	6-DoF offset composed onto the detected cup centroid for the pre-grab pose. Shape { "x", "y", "z", "o_x", "o_y", "o_z", "theta" }; same gripper orientation as the grab pose but translated further back from the cup. Not stored on the pose switch — it's an offset, not a real world-frame pose.
cup_grab_relative_pose	object	Yes	6-DoF offset composed onto the detected cup centroid for the final grab pose. Same shape as cup_approach_relative_pose; gripper orientation for a side-grab with a small translation onto the cup.
cup_pickup_max_attempts	int	No	Cap on full observe-and-grab attempts per order. Each attempt sweeps every observe pose, grabbing the first reachable cup across all of them (closest-first within each pose, continuing to later poses when a pose's cups are all unreachable). When a whole sweep grabs nothing the machine re-observes and retries — asking the customer to place a cup (none seen anywhere) or to nudge the cups (some seen but none reachable) between attempts. Default 3.
cup_dimensions	object	Yes	Known cup size. Shape { "diameter_mm", "height_mm" } (both must be > 0). The held-item bounding box is built with width = depth = diameter_mm and height = height_mm (a square-footprint box approximating the round cup), centered on the grasp centroid (the point the gripper is sent to). Required because every carried container is tracked as a held item (see Held-item geometry below), and modeling a known size around the grasp point avoids a box skewed by a partially-observed point cloud. The known height_mm also drives resting-surface seating (below): the grasp Z is derived from the surface the cup stands on instead of the noisy detected Z.
max_batch_size	int	No	Cap on prepare_order.count — how many identical drinks one DoCommand may enqueue at once. Defaults to 10 when unset. Protects the queue against runaway voice commands or LLM hallucinations.
door_open_angle_degs	float	No	Swing angle for the open_door command, in degrees. Defaults to 90.
door_pivot_degrees_per_step	float	No	Per-step θ increment for the open_door sweep, in degrees. Smaller values re-plan the door pose more finely (smoother tracking, more planning calls). Defaults to 10.
door_grasp_frame_name	string	No	Frame the gripper aims at for open_door — its center is the grasp target, and it's the frame tracked through the sweep and allowed to contact the gripper. Must be a child frame within the fridge-door subtree so it rides the swing. Defaults to fridge-handle-ball.
door_approach_relative_pose	object	For open_door	A RelativePose (x,y,z,o_x,o_y,o_z,theta) offset composed onto the grasp frame's center (world axes) to produce the pre-grasp standoff — the door analog of cup_approach_relative_pose, but resolved against the live grasp frame instead of a detected cup. Its orientation is the base grasp orientation, which door_grasp_yaw_ratio then yaws through the swing. Required to run open_door.
door_grasp_yaw_ratio	float	No	How far the gripper yaws about world Z as the door swings: the grasp orientation (and the approach/retract standoff) is turned by ratio x theta — a pure function of the door angle, so open_door and close_door are exact inverses and door_approach_relative_pose is authored for the shut door. -1 (default) counter-rotates, keeping the wrist on the near side of the ball as the handle swings away; 0 holds the orientation fixed in world; 1 keeps the tool square to the panel. The sign is a reachability choice — the handle is a sphere, so the grasp does not constrain wrist roll. Replanning a failed 75° sweep offline put 1 out of IK solutions by θ=47° and 0 out by θ=75°. 0 and negatives are honored, so unset is distinct from zero.
keepalive	object	No	Enables the keep-alive purge loop, which holds the machine's 1 CUP button periodically so the machine never falls out of brew temperature. Requires has_separate_brew_buttons. Omit to disable. Fields: auto_start and end ("HH:MM" local, the half-open window [auto_start, end); auto_start must mirror the time programmed into the machine's own Auto Start setting), timezone (IANA name, required — an empty value is rejected rather than silently meaning UTC), days (lowercase three-letter names, default Mon–Fri), after_min (idle minutes before a purge is due, default 40), check_interval_min (tick period, default 5), hold_sec (button dwell, default 1 — this sets how much water reaches the drip tray). after_min + 2*check_interval_min must be under 55, since the machine sleeps after roughly 60 idle minutes. See "Keeping the machine at brew temperature" below.
can_serve_decaf	bool	No	Enables the decaf and decaf_lungo drinks, which grind from the decaf grinder instead of the regular one. Orders for those drinks are rejected when this is false. Default false.
can_serve_iced	bool	No	Enables the iced_coffee drink. When true, the machine brews a lungo (lungo_brew_time_sec, and the lungo button under has_separate_brew_buttons); after brewing, the arm vision-detects a glass off the top shelf, dispenses ice into it via ice_board_name/ice_pin_name, sets the glass in a staging area, then pours the espresso over the ice. Both finished items — the empty espresso cup and the iced glass — are then placed in the serving area at the next round-robin slots (two slots are consumed per order). The glass is always vision-detected, so iced coffee requires ice_board_name, ice_pin_name, the glass_* vision fields below, and the iced claws poses below. A serving-area (or serving-area_origin) Box geometry must exist in the framesystem; this is checked at runtime, not at config time. Default false.
ice_board_name	string	When can_serve_iced is enabled	Name of a rdk:component:board whose GPIO pin triggers the ice machine.
ice_pin_name	string	When can_serve_iced is enabled	Board pin held HIGH to dispense ice. Required — there is no default pin.
ice_dispense_sec	float	No	How long the ice pin is held HIGH per drink, in seconds. Defaults to 5. With ice_vision_enabled this is only the fallback used when the camera fails.
ice_vision_enabled	bool	No	Watch the glass while ice falls and close the pin when the ice surface passes ice_stop_row_px, instead of dispensing for a fixed ice_dispense_sec. Default false: the shipping stop row was measured on one machine at one glass seating, so confirm yours with check_ice_level before turning this on.
ice_stop_row_px	int	No	Image row the ice surface must reach for the glass to count as full. The camera rides the gripper, so a row is a fixed height above the jaws. Defaults to 595.
ice_contrast_window	int	No	Rows averaged on each side of a candidate row when measuring the brightness step. The ice surface is a gradual edge, so a narrow window measures only part of it. Defaults to 48.
ice_min_contrast	float	No	Smallest brightness step counted as an ice surface. Below it the frame reports no ice. Defaults to 25.
ice_roi_x0, ice_roi_x1	int	No	Left and right edges of the column strip whose row brightness is averaged, inset from the glass walls. Default 700 and 910.
ice_roi_y1	int	No	Bottom of the scan — the lowest part of the glass the camera can see before the ice machine's ledge occludes it. Defaults to 670. There is no ice_roi_y0: the top of the scan is derived as ice_stop_row_px - ice_contrast_window, which is what keeps the glass rim out of the measurement.
ice_dispense_max_sec	float	No	Absolute ceiling on a watched dispense, measured from the pin opening. On reaching it the glass is served as it is and ice_dispense_timeouts is bumped; the order is not failed. Defaults to 60 — an observed fill took 17-20s, so a much lower value times out every drink. This is the backstop for a hopper that never delivers; ice_after_first_seen_max_sec is the one that bounds overflow.
ice_after_first_seen_max_sec	float	No	Second ceiling, measured from the first confirmed sighting rather than from the pin opening. Defaults to 30. It exists because ice_dispense_max_sec has to clear a whole fill, which leaves it ~10s above a full glass on a run whose surface is never confirmed past the stop row — and that overflow is ice, not a light drink. Ends the dispense the same way the absolute ceiling does. Must be more than ice_check_interval_sec, or the dispense ends on the first poll after ice appears.
ice_dispense_min_sec	float	No	How long the pin is held open before any reading counts. Defaults to 2.
ice_check_interval_sec	float	No	How often the glass is measured during a watched dispense. Defaults to 0.5.
ice_brightness_thresh	float	No	Row-brightness cutoff (0-255) for the brightness shadow: a second read of every dispense frame by absolute brightness, logged beside the contrast step and never acted on. Unset (the default) the shadow is off, and it deliberately has no default value — an absolute cutoff is the one number that does not survive a change in lighting, so read it off the machine with check_ice_level, which prints the range of row brightness it sees. See "Brightness shadow" below.
ice_bright_run	int	No	Consecutive rows that must clear ice_brightness_thresh for the shadow to call it a surface. One bright row is a glint off the glass; ice holds its brightness over many rows. Defaults to 8.
pour_vel_degs_per_sec	float	No	Max joint velocity (degrees/sec) for the pour tilt and return-upright pivots — the espresso pour and the milk pour alike. Overrides the general slow-movement velocity so the pour isn't dragged out (default: 60). Lower it if the espresso splashes over the ice.
pour_acc_degs_per_sec2	float	No	Max joint acceleration (degrees/sec²) for the pour pivots. The tilt is a short move that's usually acceleration-limited, so this — not velocity — is what actually snaps it faster. Default 0 leaves the arm's own acceleration in place; raise it to speed the tilt, watching for splash and joint stress.
glass_vision_service_name	string	When can_serve_iced is enabled	Name of a rdk:service:vision segmenter that returns glass detections via GetObjectPointClouds. Glass pickup mirrors cup pickup but with its own vision service and observe poses (tuned for the taller iced-coffee glass); it shares the cup camera (src_camera_name).
glass_observe_pose_switcher_name	string	When can_serve_iced is enabled	Switcher holding the glass observation vantages (swept one at a time, same as the cup observe switch). Must include a pose named glass_observe (home/recovery), and all poses must move the cam frame.
glass_approach_relative_pose	object	When can_serve_iced is enabled	6-DoF gripper offset composed onto the detected glass centroid for the pre-grab pose (same shape as cup_approach_relative_pose), tuned for the taller glass.
glass_grab_relative_pose	object	When can_serve_iced is enabled	6-DoF gripper offset for the final glass grab pose.
glass_dimensions	object	When can_serve_iced is enabled	Known glass size. Same shape and behavior as cup_dimensions ({ "diameter_mm", "height_mm" }, both > 0), applied to the glass held-item geometry; its height_mm likewise drives resting-surface seating (below). Only the iced flow picks up a glass, so it is required — and only checked — when can_serve_iced is set.
can_serve_iced_latte	bool	No	Enables the iced_latte drink: the whole iced-coffee sequence, plus a fridge trip for milk between placing the empty espresso cup and picking the glass back up. The arm opens the fridge door, vision-detects the milk bottle inside, pours it into the staged glass, sets the bottle back down where it found it, and shuts the door. Requires can_serve_iced (the latte is served in the iced glass, over ice), door_approach_relative_pose (the milk is behind the fridge door), the milk_* fields below and the iced-latte claws poses below. Default false.
milk_vision_service_name	string	When can_serve_iced_latte is enabled	Name of a rdk:service:vision segmenter that returns milk-bottle detections via GetObjectPointClouds. Milk pickup mirrors cup and glass pickup with its own vision service and observe poses; it shares the cup camera (src_camera_name).
milk_observe_pose_switcher_name	string	When can_serve_iced_latte is enabled	Switcher holding the milk observation vantages, looking into the open fridge. Must include a pose named milk_observe (home/recovery), and all poses must move the cam frame. Same sweep semantics as the cup observe switch.
milk_approach_relative_pose	object	When can_serve_iced_latte is enabled	6-DoF gripper offset composed onto the detected bottle centroid for the pre-grab standoff (same shape as cup_approach_relative_pose). It is also what the return aims at, so it must be a clean straight-up standoff over the bottle's spot on the shelf.
milk_grab_relative_pose	object	When can_serve_iced_latte is enabled	6-DoF gripper offset for the final bottle grab pose — and, replayed onto the recorded pickup centroid, the pose the bottle is set back down at.
milk_bottle_dimensions	object	When can_serve_iced_latte is enabled	Known bottle size. Same shape and behavior as cup_dimensions ({ "diameter_mm", "height_mm" }, both > 0), applied to the held-bottle geometry; its height_mm likewise drives resting-surface seating (below). Required — and only checked — when can_serve_iced_latte is set.
milk_pour_sec	float	No	How long the tilted bottle is held over the glass, in seconds. This — not the tilt angle — is the milk dose, so tune it on the machine against the bottle and glass in use. Defaults to 4.
serving_approach_relative_pose	object	Yes	6-DoF gripper offset composed onto the serving-area slot anchor for the pre-release approach pose (same shape as cup_approach_relative_pose). Used for both the hot cup and the iced glass.
serving_grab_relative_pose	object	Yes	6-DoF gripper offset composed onto the serving-area slot anchor for the release pose. Same shape as serving_approach_relative_pose; shared by cup and glass placement.
fake_mode	bool	No	Test-machine knob. When true, AllowedCollision entries that reference gripper sub-geometries (e.g. gripper:claws, which only exist on the real ufactory gripper) are skipped, so motion plans validate against fake hardware. Leave unset on the real bot. Default false.

Glass and milk-bottle pickup reuse cup_pickup_max_attempts (an item-agnostic operational knob); there are no glass- or milk-specific versions.

No-spill carry. Every free traverse of a filled container is planned with a 15° orientation constraint (noSpillOrientationToleranceDegs in coffee/motion.go) instead of free-planning straight to the goal. RDK checks the constraint along the whole path as a tube of that width around the direct slerp from the carry's start orientation to its goal, so one start-to-goal segment holds the container near level for the entire traverse — no intermediate waypoints are needed. Both endpoints are upright container poses, so the slerp itself stays level and 15° leaves the drink well inside a full cup's static spill angle. The constraint ignores theta (spin about the container's own vertical axis): a container spills when tipped, not when spun, so only tilt is bounded. The path itself is unconstrained: the planner is free to route around the machine however it likes, subject only to collision avoidance and the orientation tube. This covers the serving-area placement of a filled container (the hot cup and the iced glass), carrying the ice-filled glass to the staging area, carrying the espresso cup to the pour position, and both carries of the open milk bottle (fridge to the pour position, and back to the fridge). The espresso cup's own trip to the shelf happens after it has been poured over the ice, so it is empty by then and free-plans straight to the slot — there is nothing left in it to slosh. Manual place_held assumes filled. The goal commands the held-item (container) frame rather than the gripper — the goal pose is converted onto it — because the orientation bound is measured in the commanded frame's own axes, and the held-item frame's +Z is the container's vertical axis (see Held-item geometry below). Tune the tolerance on hardware.

Held-item geometry. A picked-up cup/glass/milk bottle is always tracked: its geometry — a box of the configured cup_dimensions / glass_dimensions / milk_bottle_dimensions centered on the grasp centroid — is attached to the gripper frame in the cached frame system as a held-item frame, so motion planning routes around the held item until it is set down (and is restored on each re-grab — the brewed cup from under the machine, the staged glass). The frame is attached rotated onto the container's axes: its +Z is the container's vertical axis, world-upright at the moment of the grab and tilting with the container thereafter as the wrist moves — that is the frame the no-spill carry commands and measures tilt against. The rotation does not move the geometry (a frame's geometry attaches at the frame's parent, not the frame itself), so collision behavior is unchanged by it. When the brewed cup is re-grabbed but nothing was cached to restore — e.g. a manually-stepped serving that never ran the vision cup pickup — its held-item box is modeled from cup_dimensions and the grip point's current world pose, so the pour and shelf placement still track the cup. The gripper-overlap collision pairs are allowed automatically on every move while an item is held; contact phases near a modeled surface (under the machine, the serving-area shelf) allow the held item against that surface too. The held-item frame is dropped when the frame system is rebuilt (reset_world, rewind, proceed).

Resting-surface seating. Pickup resolves each detection's grasp Z from the surface the container stands on rather than the raw detected centroid Z (which depth noise pushes above or below the true base), using the configured container height (cup_dimensions / glass_dimensions). It finds the highest static Box in the framesystem whose world footprint lies directly beneath the detection and whose top face is below the detected centroid, then seats the container's base 1 mm above that top — so the grasp centroid becomes surfaceTop + 1 mm + height/2, keeping the detected X/Y. "Static" means world-anchored (it moves rigidly with the world frame), so the moving arm, gripper, camera, and any held item are never mistaken for a surface; non-box and rotated geometries are bounded by their world axis-aligned extent. No framesystem changes or extra config are required — the resting surface is auto-detected. When no surface is found beneath a detection, pickup uses the raw detected Z unchanged.

Serving-area placement. Every finished cup (and, for iced coffee, the iced glass) is placed on a dedicated served-drinks shelf. Slots are tiled along the shelf's long axis (120 mm spacing, 60 mm margin from each end) on the midline of the shelf top — as many slots as the shelf length allows; the placement anchor is set so the held container's bottom rests on the shelf top: half the tracked container's height above the surface (so a taller iced glass is not driven into the shelf the way a fixed offset did), falling back to a fixed 30 mm on the rare paths where no held-item geometry is tracked. The anchor is composed with serving_grab_relative_pose (and serving_approach_relative_pose for the approach) to derive the actual grip-point pose, mirroring how the pickup composes its offsets onto the detected cup centroid. Slots are filled sequentially (round-robin): a process-local counter advances one slot per placement and wraps back to the first slot when it reaches the end, on the assumption that by the time it wraps the earliest-placed cup has been picked up. If the arm cannot plan a path to a slot (approach or descent), that slot is skipped and the next one is tried, continuing around the ring until one is reachable (the order fails only if every slot is unreachable). There is no vision-based occupancy check — placement is fully decoupled from pickup observation. The counter resets to the first slot on module restart/reconfigure. Requires a serving-area (or serving-area_origin) Box geometry in the framesystem; this is checked at runtime, not at config time. An optional serving-area-shield Box obstacle may be added to the framesystem to enclose the standing-cup zone above the shelf: it stays a hard obstacle during the lateral carry (so the arm steers clear of cups already on the shelf) but is allowed to be passed through by the gripper, claws, and held container on the linearly constrained descent into a slot and the retreat back out. Size it with clearance above the cups so the approach pose stays outside it; when the frame is absent the allowances are inert and placement behaves as before.

Iced coffee — required poses on the claws pose switcher (claws_pose_switcher_name):

When can_serve_iced is enabled, the claws switch must additionally hold these poses (all moving the frame named by the switch's component_name — the grip-point frame). Calibrate them physically on the machine via viam robot part motion get-pose/set-pose. The glass itself is vision-detected (see the glass-observe switch below), so there are no static glass-pickup poses.

Pose name	Description
ice_machine_approach	Staged in front of the ice chute.
ice_machine_dispense	Glass held under the chute while the ice pin pulses. If you re-teach this pose, re-check ice_stop_row_px: with ice_vision_enabled the ice level is measured in raw image rows at this pose, and moving the glass in frame moves what those rows mean. check_ice_level prints both the surface row and the glass rim.
staging_approach	Above the staging area where the glass rests during the pour.
staging	Down in the staging area; the glass is set here to free the gripper for the pour, then re-grabbed and placed in the serving area.
pour_approach	Espresso cup held upright above the staged glass.
pour	Espresso cup tilted to pour over the ice.

Iced latte — required poses on the claws pose switcher (claws_pose_switcher_name):

When can_serve_iced_latte is enabled, the claws switch must hold these two poses on top of the iced-coffee ones. Nothing is authored for the fridge itself: the bottle is vision-detected (see the milk-observe switch below) and set back down at the spot it was detected at, and the door is tracked through its hinge arc rather than driven to poses.

Pose name	Description
milk_pour_approach	Milk bottle held upright above the staged glass. The bottle is taller and heavier than the espresso cup, so this is its own pose rather than a reuse of pour_approach.
milk_pour	Milk bottle tilted to pour into the glass. The tilt only has to clear the rim — how much milk lands is milk_pour_sec, the dwell at this pose.

Cup pickup — required poses on the camera-observe pose switcher (camera_observe_pose_switcher_name):

Cup pickup is always vision-guided, so the dedicated camera-observe switch must hold one or more observation poses, all moving the camera frame cam (the switch's component_name). The switch must include a pose named cup_observe.

Pose name	Type	Description
cup_observe	Absolute world pose	Required. The primary view of the cup workspace and the home/recovery pose the arm returns to between grab attempts.
additional poses	Absolute world pose	Optional extra vantages tried in turn, to recover cups occluded from the primary view or reachable only from a different angle. A pose is visited when earlier poses found no cup or when the cups they saw were all unreachable; the sweep stops as soon as a cup is grabbed. An unreachable pose logs a warning and is skipped.

Dynamic glass pickup — required poses on the glass-observe pose switcher (glass_observe_pose_switcher_name):

When can_serve_iced is enabled, the dedicated glass-observe switch must hold one or more observation poses, all moving the cam frame. The switch must include a pose named glass_observe. Same sweep semantics as the cup observe switch.

Pose name	Type	Description
glass_observe	Absolute world pose	Required. The primary view of the glass storage area and the home/recovery pose between grab attempts.
additional poses	Absolute world pose	Optional extra vantages tried only when earlier poses found no glass.

Dynamic milk pickup — required poses on the milk-observe pose switcher (milk_observe_pose_switcher_name):

When can_serve_iced_latte is enabled, the dedicated milk-observe switch must hold one or more observation poses, all moving the cam frame. The switch must include a pose named milk_observe. Same sweep semantics as the cup observe switch — but calibrate these vantages with the fridge door open, since that is the only state the arm ever observes the milk in.

Pose name	Type	Description
milk_observe	Absolute world pose	Required. The primary view into the open fridge and the home/recovery pose between grab attempts.
additional poses	Absolute world pose	Optional extra vantages tried only when earlier poses found no bottle.

DoCommand

prepare_order - Prepare a drink order with optional speech greetings. Supports "espresso" and "lungo"; "decaf"/"decaf_lungo" when can_serve_decaf is set, "iced_coffee" when can_serve_iced is set, and "iced_latte" when can_serve_iced_latte is set.

{
  "prepare_order": {
    "drink": "espresso",
    "customer_name": "Alice",
    "modified_customer_name": "Alise",
    "customer_email": "alice@example.com",
    "initial_greeting": "optional custom greeting",
    "completion_statement": "optional custom completion message",
    "count": 3,
    "fulfillment": "pickup"
  }
}

Only drink is required. If initial_greeting is omitted, a random greeting is generated. If customer_name is provided, it personalizes the greeting and completion messages. customer_name must be the name the customer actually gave: it is the identity key every per-customer aggregation groups on (leaderboards, the daily digest's top customers). modified_customer_name is optional and exists for one reason — the kiosk deliberately misspells the name it shows and says, with a fresh misspelling each order, so that string can never identify a repeat customer. A caller that misspells sends the altered name here and the real one in customer_name; everything spoken and everything get_queue renders uses this field, falling back to customer_name when it is absent, which is what callers that never misspell (voice, operator) should do. If customer_email is provided and customer_detector_name is configured, the completed drink is credited to that customer's order history (see "the usual"). fulfillment is either "pickup" (default) or "delivery"; it is carried on each order through get_queue. Pickup orders keep the usual drink-ready announcement at cup handoff; delivery orders instead announce "…ready for delivery!" and send the delivery machine a delivery_request (see send_delivery_message). Delivery orders require customer_email — the delivery bot identifies the recipient by email, so an anonymous delivery is rejected at enqueue; pickup stays open to anonymous walk-ups. Orders are added to a queue and processed sequentially.

count is an optional positive integer (default 1) that enqueues N identical orders in one call — each gets its own UUID. The cap is max_batch_size (default 10). When count > 1, the response also includes order_ids: [...] (one per enqueued order) and count; existing order_id and queue_position keys still refer to the first order so existing callers keep working. To keep audio sane, the per-order "Order received…" line is replaced with a single consolidated batch announcement at submission time; the per-cup drink-ready announcement at cup handoff still fires once per order as each cup completes.

execute_action - Run a single coffee-making action by name, for manual step-by-step operation. An unknown name returns the full list of available actions in the error. Available actions:

• Brew cycle: grind_coffee, grind_decaf, tamp_ground, lock_portafilter, unlock_portafilter, release_filter, grab_filter, brew_coffee, plus the button actions for the configured machine (turn_coffee_button_on / turn_coffee_button_off, or press_espresso_button / press_lungo_button / brew_lungo under has_separate_brew_buttons), set_cup_for_coffee, give_full_cup_to_customer (place the finished cup in the serving area), clean_portafilter, place_held (place the currently held vessel in the serving area), keepalive_purge (one group-head purge on demand — available on the has_separate_brew_buttons machine whether or not keepalive is configured, so the purge_* poses can be verified before the loop is switched on).
• Iced coffee (require can_serve_iced): fetch_glass, pulse_ice_pin, move_to_ice_dispense (hold the glass under the chute without opening the pin), dispense_ice, stage_glass, grab_brewed_cup, pour_espresso, grab_staged_glass, serve_iced_coffee (the full iced sequence end-to-end), check_ice_level (measure the glass in the current camera frame and log it; no arm motion, no pin).
• Iced latte (require can_serve_iced_latte): fetch_milk (vision-grab the bottle from the already open fridge), pour_milk (pour the held bottle into the staged glass), return_milk (set the bottle back down where fetch_milk picked it up), add_milk (the whole fridge trip: open door → fetch → pour → return → close door), serve_iced_latte (the full iced-plus-milk sequence end-to-end). fetch_milk, pour_milk and return_milk are meant for stepping the sequence one move at a time while calibrating; each assumes the state the one before it leaves behind, and return_milk fails if no fetch_milk recorded a pickup position.
• Fridge door (requires door_approach_relative_pose): open_door (grip the handle and swing the door open — see below).

{"execute_action": "grind_coffee"}

with_glass: true swaps the portafilter for a glass in the frame system for that one call — the filter subtree comes out, a box of glass_dimensions at the glass_grab_relative_pose centroid goes on the gripper — for stepping the iced sequence with a glass loaded into the jaws by hand. Without it the frame system models a portafilter that is not there and no glass that is, and the arm plans a path that drags the glass across the table, or cannot reach ice_machine_dispense at all. The swap is undone when the action ends, so pass the flag on every call of the sequence, not just the first.

Accepted by move_to_ice_dispense, dispense_ice, stage_glass, pulse_ice_pin and check_ice_level; anything else is refused, and the error names those five. The pickups (fetch_glass, grab_staged_glass, grab_brewed_cup) refuse it because they start with empty jaws: a stand-in there is a phantom sitting exactly where the real glass is about to be grasped, and every candidate fails to plan. Reach empty jaws with lock_portafilter then release_filter instead.

glass_dimensions / glass_grab_relative_pose are needed only for that empty-jaws case — when the gripper already models a held item, only the filter is dropped. A filter locked into the machine is modeled in the bayonet rather than on the claws, so it is left alone.

{"execute_action": "move_to_ice_dispense", "with_glass": true}

annotate: true makes check_ice_level, dispense_ice and pulse_ice_pin save an annotated JPEG of each frame they measure — scan band, stop row, both methods' readings — into save_motion_requests_dir. An order's dispense saves one without being asked; the flag only adds the hand-run actions, which are a tuning loop fired as fast as it can be typed and would otherwise bury the orders. Does nothing without save_motion_requests_dir, or on a dispense without ice_vision_enabled.

{"execute_action": "check_ice_level", "annotate": true}

cancel - Stop whatever is running, and nothing else. It cancels the shared sequence context so the run aborts at its next step boundary, calls Stop on the arm so the in-flight trajectory halts where it stands instead of playing out to its authored pose, and pauses the queue. No arm motion is planned, no gripper is opened, no state flag is cleared, and the cached frame system is left untouched: the portafilter stays wherever it was, a held cup stays in the jaws, pending orders stay queued.

{"cancel": true}

Returns {"status": "cancelled", "cancelled": true, "queue": "paused"} — cancelled is false when nothing was running, and queue reports the real pause state.

To get the arm back to a clean starting state afterwards, run rewind, then proceed to resume the queue.

Pause after a fault. An order that fails on its own — a genuine fault, not a cancel or reset_world — always pauses the queue exactly as cancel does. The next order would otherwise start from an unknown physical state, grinding a second dose onto used grounds or planning against a stale world, and not everything a fault leaves behind (a knocked-over cup, a half-finished pour) is something the service can see. An error log names any mid-cycle state still recorded — the portafilter locked in the group head or holding grounds, the filter frame locked to world, an item in the gripper, a glass staged, or the fridge door modeled open — and get_queue reports is_paused: true; recover the same way as after a cancel (rewind, shut the fridge by hand if it is open, then proceed). A failed manually-stepped execute_action never pauses the queue.

cancel_order - Drop one order out of the backlog, by ID. It is clear_queue narrowed to a single order, and it spares the drink on the arm for the same reason: the machine keeps making whatever it is making, the queue is not paused, no state flag is touched, and the orders behind the cancelled one move up a place. This is the customer-facing "actually, never mind" — cancel is the operator's stop button.

{"cancel_order": "3f8c1e2a-…"}

Returns {"status": "cancelled", "order_id": "3f8c1e2a-…", "customer_name": "Alice", "drink": "lungo", "remaining": 2}, where remaining is the depth after the removal on the same count get_queue reports — backlog plus the order on the arm. The order is announced as cancelled when conversational is on.

It errors rather than guessing when the order can't be dropped:

• The order on the arm — it is in the queue's current slot, not the backlog, so a cancel aimed at it is refused rather than quietly missing. Use cancel to stop the machine instead.
• An order that has already been made — it is in the completed buffer get_queue shows for ~15s.
• An unknown ID — it was cancelled already, or never existed.

get_queue marks every order with a cancellable boolean saying which of these it is, so a UI can offer the action only where it will be accepted. Note this is not the same as "first in the list": the front of the backlog is cancellable precisely because nothing has started on it.

rewind - Drive the arm back to the state a brew cycle starts from: nothing in the gripper, no grounds in the portafilter, the filter home in the claws. It stops any running sequence first, so it is safe to send at any time without a preceding cancel.

In order: drop a cup or glass still in the jaws (open → detach the held-item geometry → close; a gripper closed on the thin filter handle reads as closed, so the portafilter is never dropped), then run whichever recovery the recorded portafilter state calls for:

• Portafilter locked in the machine (after release_filter, before grab_filter): grab → unlock → clean → home.
• Portafilter in the claws with grounds in it (after grinding, before cleaning): clean → home.
• Neither: no arm motion.

The cached frame system is rebuilt at the end, discarding any mid-cycle mutation such as a filter frame reparented to world by lock_portafilter. The queue stays paused with its pending orders intact; send proceed to resume. If recovery motion fails, the frame system is left untouched and the state flags stay set, so a second rewind retries from where the first stopped.

⚠️ A cancel that fired mid-lock_portafilter — between the arm entering the machine and the gripper opening — leaves the bayonet partially engaged, and rewind may try to route the arm away from it. There is no safe automated recovery for that window: free the filter by hand first.

{"rewind": true}

Returns {"status": "rewound", "cancelled": false, "recovered": true, "queue": "paused"} — recovered reports whether recovery motion actually ran.

get_queue - Get the current order queue status.

{"get_queue": true}

Returns:

{
  "count": 2,
  "orders": [
    {"id": "3f8c1e2a-…", "drink": "espresso", "customer_name": "Alice", "fulfillment": "pickup",
     "enqueued_at": "2026-09-17T09:12:03Z", "raw_step": "Brewing", "step_history": [],
     "completed_at": "", "cancellable": false},
    {"id": "9b21d740-…", "drink": "lungo", "customer_name": "Bob", "fulfillment": "pickup",
     "enqueued_at": "2026-09-17T09:12:40Z", "raw_step": "", "step_history": [],
     "completed_at": "", "cancellable": true}
  ],
  "is_paused": false,
  "is_busy": true
}

count is how many drinks still have to be made — the backlog plus the one on the arm. Orders that have finished stay in the list with completed_at set for ~15s so a UI can render a "Ready!" card without diffing polls, but they don't count toward the depth. cancellable says whether cancel_order would accept this order.

proceed - Re-sync the recorded world with the real one, and resume queue processing after a pause — from a cancel, or from a fault that left the machine mid-cycle (see "Pause after a fault" above).

The frame system is rebuilt from the framesystem service on every proceed, discarding whatever an interrupted order left mid-cycle — a filter frame reparented to world by lock_portafilter, a held-item geometry, a staged-glass obstacle. This is unconditional because the next order's opportunistic refresh deliberately declines to rebuild while a held item or locked filter is recorded, and a paused queue is not the only way those mutations are stranded: a failed manually-stepped execute_action leaves them behind with the queue still running. If the rebuild fails, nothing resumes and a paused queue stays paused.

The recorded fridge-door angle is cleared too, so the rebuilt world has a shut door like every other configured obstacle. proceed is the operator saying the machine has been put right by hand, and that includes the fridge. Rebuilding a model still cannot shut a real door — which is why a rebuild never clears the angle on its own, and every other one re-applies it — so this is an assertion you are making, not a check the service can run.

⚠️ Shut the fridge before sending proceed. If the door is really still open, the model now claims it is closed and the next plan will route the arm straight through the panel. A proceed that forgets an angle says so: a warning in the logs, and a fridge_door_cleared_degs field in the response (absent when the door was already shut).

The pause is only released if there is one — proceed on a running queue rebuilds the world and reports resumed: false. Releasing the pause is clearing the queue's paused flag, done by proceed itself: any cancel pauses the queue, including one that interrupts an execute_action or a keepalive purge with no order in flight, and such a pause has no order-queue goroutine waiting to hear about it. Two proceeds racing therefore resume once, and the second reports resumed: false. proceed refuses outright while a sequence is still running (including one that is unwinding from a cancel): the frame system cannot be swapped under a goroutine that is planning with it.

Because the rebuild forgets the modeled contents of the gripper without opening the gripper, proceed is the wrong command when the jaws really are holding something — a portafilter or cup a manually-stepped execute_action picked up. Use cancel then rewind, which physically lets go and homes the arm, and only then proceed. A proceed that forgets a held item logs a warning saying so.

{"proceed": true}

Returns {"status": "resumed", "resumed": true, "frame_system_reset": true}, or {"status": "reset", "resumed": false, "frame_system_reset": true} when the queue was not paused. A fridge_door_cleared_degs field is added when the rebuild forgot a door angle.

clear_queue - Drop the backlog of orders still waiting to be made. The order currently being brewed keeps running and stays in the queue, and recently-completed orders keep showing as ready. Use cancel to stop an order mid-brew, or reset_world to cancel and wipe the queue entirely.

{"clear_queue": true}

Returns {"status": "cleared", "removed": 2, "kept_current": false}. When an order was being brewed, kept_current is true and kept_current_order_id names the order that was spared.

cleanup_pending_clips - Attempt a video save for any remaining pending-clip records under data_dir, removing each record only once its save succeeds; a failed save (including one that exceeds the 60s save timeout) keeps the record for the next run. Catches clips whose live save was interrupted (process died during the post-roll wait) or failed (e.g. cam storage unavailable). Records younger than one full clip window plus a segment-flush margin are skipped, so an in-progress order is not double-saved. If records exist but cam_storage_mux_name is unset, the command returns an error and leaves them in place so they can be recovered once the mux is configured again. Intended to be invoked via a Viam scheduled job.

{"cleanup_pending_clips": true}

Returns {"saved": 1, "failed": 0, "skipped": 0}.

send_delivery_message - Run a DoCommand on the peer machine's service named by delivery_handler_name and return its response — the manual test hook for the delivery channel. The value is forwarded to the peer verbatim, so it must be a non-empty object using the command vocabulary the peer's own service understands. Errors when delivery_handler_name is unset, the value isn't an object, or the peer is unreachable (10s timeout).

{"send_delivery_message": {"delivery_request": {"order_id": "test", "order_timestamp": "2026-07-16T15:04:05Z", "cup_type": "cup", "customer_email": "alice@example.com", "pickup_position": 1}}}

Returns {"sent": true, "peer_response": {...}} where peer_response is whatever the peer's service returned.

In normal operation this fires automatically: when a fulfillment: "delivery" order's drink lands in the serving area, the coffee service sends the peer a delivery_request with the shape above — order_id/order_timestamp (RFC3339 enqueue time) from the order, customer_email (required for delivery orders, so always non-empty here), cup_type — the container label, "glass" for iced drinks and "cup" for everything else (same labels the cup-pickup pipeline uses) — and pickup_position the 0-based serving-area slot the drink was placed in. The send is deliberately synchronous: the service waits (up to 10s) for the bot's {"received": true} acknowledgment before the drink-ready announcement, so an unconfirmed handoff is logged rather than assumed. Failures never fail the order — the drink is already in the serving area. The channel is otherwise one-way: the coffee machine observes its own serving slots by camera rather than waiting for delivery progress reports.

send_daily_summary - Post a Slack digest of the orders from the last 24 hours. Normally fired on a schedule by viam-server's job manager (see "Daily order summary in Slack" below); calling it by hand is how you test the digest off-schedule. Requires both slack_notifier_name and order_sensor_name.

{"send_daily_summary": true}

The command takes no options — the window is a rolling 24 hours ending now, and timestamps render in the host's timezone. Returns {"sent": true, "orders": N}.

send_weekly_chores - Post this week's chore assignments to Slack. Normally fired every Monday by viam-server's job manager (see "Weekly chore wheel in Slack" below). Requires chore_wheel and slack_notifier_name.

{"send_weekly_chores": true}

Pass an object with a date to read the wheel for a different week — how you preview next Monday, or check what last Monday should have said — without waiting for it:

{"send_weekly_chores": {"date": "2026-10-05"}}

Returns {"sent": true, "week": N, "<chore>": "<person>", ...}.

reset_world - Recover the service to a clean idle state from anywhere. In order: cancels any running sequence (waiting for it to actually stop), clears the queue (pending + recently completed), rebuilds the cached frame system from the framesystem service (discarding mid-cycle mutations like a portafilter frame reparented to world by lock_portafilter), forgets that the fridge door is standing open, and releases the queue pause left by a cancel or a fault. Safe to call from any state — each step is skipped when not applicable. The queue clear and rebuild run holding the arm, so if another sequence (a manual action, a keep-alive purge) claims it first, reset_world fails without changing anything past the cancel — send it again. Does not move the arm — if you want to re-home, run execute_action afterward.

⚠️ reset_world asserts that the physical world matches the configured frame system. Like proceed, it clears the recorded fridge-door angle, so shut the door by hand before running it — otherwise the model believes the panel is closed while it stands open, and the next plan will route the arm straight through it.

{"reset_world": true}

Returns {"status": "reset", "cancelled": true, "cleared": 2, "unpaused": true} — fields reflect which steps actually fired.

run_cup_flow - Exercise the full cup-handling path without brewing, count times. Each iteration sweeps the camera-observe poses grabbing the first reachable cup across them (closest-first, continuing past a pose whose cups are all unreachable), sets it under the machine, retrieves it, and places it on the next sequential served-shelf slot (round-robin). Intended for tuning the observe-pose sweep and shelf placement on hardware.

Assumes the portafilter has been physically removed from the claws — the flow never touches portafilter state. Honors cancel. The value is the iteration count (>= 1); true runs a single iteration.

{"run_cup_flow": 5}

Returns {"status": "complete", "iterations": 5}.

execute_action: open_door - Grip the fridge handle and pull the door open along its hinge arc, then release and retract, leaving the door open. The door is a static obstacle (fridge-door) whose root frame origin sits on the hinge; open_door sweeps the door angle in software (door_open_angle_degs, default 90°, in door_pivot_degrees_per_step increments, default 10°), re-placing the door obstacle at each step so the grasp frame (door_grasp_frame_name, default fridge-handle-ball) and the door panel track the real swing and collision-checking stays honest. The gripper aims its grip-point frame at the grasp frame's center, with the orientation from door_approach_relative_pose; the pre-grasp standoff is that same relative pose's translation offset from the ball center (resolved against the live ball frame, like cup_approach_relative_pose against a detected cup). Neither approach nor grasp is a separately-authored switch pose. The traverse to the standoff plans freely; the standoff-to-ball insertion and the post-release exit are linear (defaultApproachConstraint), the same split dynamic cup pickup uses — the gripper/grasp-frame pair is exempt from collision checking during the pull, so an unconstrained insertion could arc in through the handle bar and still plan clean. The jaws are opened at the standoff (not before — an open gripper has a wider collision silhouette than a closed one) and closed once at the ball center. Through the swing the gripper tracks the ball's point exactly and yaws the tool about world Z by door_grasp_yaw_ratio x the door angle (default -1, i.e. the tool turns against the swing, which keeps the wrist reachable as the handle travels). The sweep is planned without a linear constraint, so the waypoints are the only thing telling the planner how the tool should be pointed along the arc — hence the explicit orientation. The retract standoff is yawed by the sweep's total turn for the same reason, so the exit backs off the handle rather than into the now-open panel. Contact between the gripper and the grasp frame is allowed during the pull. After the swing it releases and retracts to the same relative-pose standoff resolved against the ball's open position, leaving the door open. Requires only door_approach_relative_pose to be set — no poses are authored on the switch for this action. Gated like every action (one sequence at a time) and honors cancel; the frame system is rebuilt on exit so the door mutation never leaks.

{"execute_action": "open_door"}

execute_action: close_door - The reverse of open_door: grip the open door's handle and push the panel back shut, then release and retract. Same grasp derivation, door_grasp_yaw_ratio orientation tracking, per-step obstacle re-placement, and cancellation — only the target angle differs (0° instead of door_open_angle_degs). The yaw follows the travel direction, so a close unwinds exactly what the open wound up.

Where the modeled door lives between actions. Both actions sweep from wherever the door currently stands to their target, and the service remembers the angle. The frame system rebuilds the door at its authored shut transform, so that recorded angle is re-applied after every rebuild — opening the fridge does not close it, and the panel must stay modeled where it physically is or later plans will route the arm through it. Only angles the arm actually reached are recorded, so an aborted sweep leaves the model at the door's real position rather than the intended one. The record is cleared only by the two commands in which an operator asserts the world is as configured — proceed and reset_world — so shut the door by hand before sending either.

{"execute_action": "close_door"}

Returns {"status": "complete", "action": "open_door"}.

Iced latte: the milk step

An iced_latte is an iced_coffee with one extra stretch spliced into the serving step. Everything up to and including "place the empty espresso cup in the serving area" is identical; then, with the gripper free and the iced glass still standing in the staging area:

1. Open the fridge — the open_door sweep, so the modeled panel tracks the real one and every plan that follows routes around it.
2. Fetch the bottle — the same vision pickup cups and glasses use, against milk_vision_service_name and the vantages on milk_observe_pose_switcher_name. Detections are seated on the shelf beneath them (resting-surface seating), ranked closest-first, and grabbed via milk_approach_relative_pose / milk_grab_relative_pose. The world centroid it was grasped at is recorded. If no bottle is seen — or none can be reached — the arm asks the customer to put the milk back on its shelf (or nudge it) and retries, exactly like a missing cup.
3. Pour — carry the bottle to milk_pour_approach and tilt to milk_pour as a fixed-point pivot, the same pour the espresso gets. The dwell at the tilted pose (milk_pour_sec) is the milk dose; the staged glass stays a hard obstacle throughout.
4. Put the bottle back — not at an authored pose, but at the centroid step 2 recorded, with the same two offsets composed onto it. Where the milk stands in the fridge changes every time somebody puts it away, so the return is the grasp replayed rather than a spot to calibrate.
5. Shut the fridge — the close_door sweep.

Then the sequence rejoins the iced flow: re-grab the staged glass and place it in the serving area. As with iced coffee, two serving slots are consumed per order (the empty espresso cup, then the latte).

The door is opened once and closed once, so the fridge stands open for the pour — a few seconds of open door in exchange for halving the door sweeps, which are the slowest and most failure-prone part of the trip.

When a milk step fails. The arm does not try to shut the door itself: it may still be holding the bottle, and a sweep needs the gripper for the handle. So the door is left where it stands, the model keeps recording that angle, and the error says so. Take the bottle out of the gripper if it is holding one and rewind to recover the arm — rewind rebuilds the frame system with the door still held open, which is correct, because it has not been shut. Then shut the door by hand and send proceed, which is what declares it shut again.

action - Control the gripper. Supported values: "open_gripper", "close_gripper".

{"action": "open_gripper"}

Returns {"status": "opened"} or {"status": "closed", "grabbed": true}.

Keeping the machine at brew temperature

The Breville BES920 drops into POWER SAVE after one hour idle and powers off completely after four, and the one-hour sleep cannot be disabled in its settings. A brew started on a sleeping machine is refused with three beeps, so the arm serves an empty cup and records it as a success. Two things prevent that, and both are needed:

1. Program the machine's own Auto Start. MENU → AUTO START → ON → a time ~15 minutes before people arrive. This handles the cold morning in hardware, and the same time goes in keepalive.auto_start. Note Auto Start has no day-of-week setting, so it also fires at weekends; the machine's own Auto Off shuts it down again after four hours.
2. Configure keepalive. During the window, the arm holds the 1 CUP button for about a second whenever nothing has run water through the machine for after_min. This is Breville's documented group-head purge, and it resets the machine's idle timer so it never leaves brew temperature. The filter pose switcher needs purge_approach and purge_press for this.

Because a purge is the one arm motion nobody requested, it announces itself through speech_service_name and waits 5 seconds before moving, regardless of conversational — it's a safety notice rather than status narration. The arm returns to home when the purge finishes.

The arm never presses POWER — per the manual, pressing POWER while the machine is in POWER SAVE turns it off. The consequence is that this cannot recover a machine that is genuinely powered down: if Auto Start does not fire, or someone switches the machine off, every order that day will brew cold and be recorded as a success. Detecting that needs a machine-state sensor, which is not part of this feature.

Water from each purge goes to the drip tray and is counted in the drip_tray_brews usage-sensor field, so empty the tray on the counter rather than on brew count alone.

Daily order summary in Slack

Once slack_notifier_name and order_sensor_name are both configured, send_daily_summary posts a Block Kit digest of the last 24 hours of orders to the same channel the failure alerts go to. A quiet 24 hours still posts a short "No orders in the last 24 hours" line, which is what keeps a quiet channel distinguishable from a broken digest.

The message reads:

:coffee: Orders in the last 24 hours

:clock3: 17m20s brewing  ·  :white_check_mark: 6/8 succeeded  ·  :coffee: 8 orders

Succeeded                 6
Faulted                   1
Cancelled by operator     1
───────────────────────────
Total                     8

:trophy: Alice — 3  ·  :sleeping: 1 decaf  ·  :fire: 12 in a row

:coffee: *Drinks*
• espresso — 4 (avg 2m12s)
• iced_latte — 2 (avg 4m10s)

:x: *Faults by step*
• Locking portafilter — 1

:clock3: Sun 5:30 PM – Mon 5:30 PM EDT · order a coffee

:trophy: names whoever ordered most. Walk-ups who skip the name screen are left out of it entirely — pooled under one "anonymous" label they would win most days and say nothing, so the leaderboard counts only named customers and consequently does not sum to the order total. Everyone tied at the top is named rather than one of them picked arbitrarily, since a two-order tie is the normal case on a quiet day. Every attempt counts toward a customer's tally whether or not the machine managed it: they asked for a drink, and the failure is the machine's record rather than theirs.

The headline line carries the three numbers worth having in a notification preview, each with an emoji the sections below reuse for the same idea — :clock3: for time, :coffee: for drinks, :x: for faults. The outcome counts are a fixed-width table in a code fence: Slack has no table block that renders reliably across clients, and a code fence is the only place it honours column alignment. Emoji do not render inside one, which is why they sit on the line above rather than in the rows. Every digest links the ordering app in its footer.

The window is a rolling 24 hours ending when the digest runs, not a calendar day. Run the job once a day and consecutive digests tile exactly: every order is reported once, evening orders included, and none is counted twice. A calendar-day window would instead have stopped at the moment the digest fired and silently dropped anything brewed after it.

When usage_sensor_name is also configured the digest adds a current streak — the machine's run of consecutive successful orders, read live from the sensor's successful_consecutive_orders counter. It is deliberately not a windowed figure: any fault or operator cancel resets it whenever it happens, so it describes the machine right now and can span days. The field is omitted entirely when no usage sensor is wired in or the read fails, rather than shown as 0, which would read as a streak that had just broken.

Brew times are reported per drink, in the drinks breakdown (• iced_latte — 7 (avg 4m12s)), not as one number across all of them. An iced latte's fridge trip runs minutes longer than an espresso, so a single mean would track the day's drink mix rather than the machine, and a slow day would be indistinguishable from a latte-heavy one. A drink whose every attempt failed shows its count with no timing rather than 0s. The grid still carries total brewing — a sum stays meaningful across mixed drinks, since it measures how long the machine was working.

The numbers do not come from anything the service keeps in memory. They are read back out of the cloud tabular store that the order sensor syncs into, using QueryTabularDataForResource from the RDK's module package, so a module restart or reconfigure inside the window loses nothing. This requires data capture to be enabled and syncing on the order-sensor component — without it the digest is honestly empty rather than wrong.

Scheduling lives in the machine config, not in this module. Add a jobs entry so viam-server's job manager calls the command on a cron; changing the hour is then a config edit rather than a module rebuild and redeploy:

"jobs": [
  {
    "name": "daily-order-summary",
    "schedule": "CRON_TZ=America/New_York 30 17 * * *",
    "resource": "coffee",
    "method": "DoCommand",
    "command": { "send_daily_summary": true }
  }
]

⚠️ CRON_TZ= is the only timezone that matters here, and it is not optional. The job manager builds its scheduler without a location, so a bare "30 17 * * *" fires at 17:30 in whatever timezone the host is set to — four hours off from New York, and silently so. Omit it only if the host's own timezone is already the one you want. The digest prints the window it covered in its footer (Sun 5:30 PM – Mon 5:30 PM EDT), which is where a wrong firing time shows up.

Fire it daily. The 24-hour window only tiles against a daily schedule. Restricting the cron to weekdays with 1-5 leaves a gap — Monday's digest reaches back to Sunday evening, so everything brewed Friday evening through Sunday afternoon is never reported by any run. Keep * * * even if the machine only gets used on weekdays; a quiet weekend costs two "No orders" lines.

method must be DoCommand — the job manager has a fast path for it that calls the service directly instead of going through gRPC reflection.

One consequence worth knowing: a job that fails is only a log line plus an entry in the job's history; nothing is posted to Slack, so a broken digest looks the same as a channel nobody used. The "No orders in the last 24 hours" heartbeat is the cheap check against that.

Prerequisite: the query authenticates from the VIAM_API_KEY / VIAM_API_KEY_ID environment variables, which viam-server only injects into modules when the machine config carries an api-key auth handler. Without one, the digest fails at call time with an auth error while everything else about the service keeps working.

Weekly chore wheel in Slack

#weekly-chore-wheel-in-slack

The machine has upkeep that isn't the arm's job — the ice maker, the table, the claws — and send_weekly_chores posts who has which of it this week. It is the paper chore wheel: names on an inner disc, chores on an outer ring, turned one notch every Monday.

"chore_wheel": {
  "people": ["Vijay", "Nicolas P", "Julie", "Daniel", "Cheuk", "Ale"],
  "chores": [
    "Cleaning the ice maker",
    "Cleaning the table",
    "Tightening the claws",
    "Cleaning the cleaner brushes",
    "Cleaning the milk bottle"
  ]
}

The message reads:

🎡 Chore wheel — week of Sep 21

• Cleaning the ice maker — Nicolas P
• Cleaning the table — Julie
• Tightening the claws — Daniel
• Cleaning the cleaner brushes — Cheuk
• Cleaning the milk bottle — Ale
• free week — Vijay

Any number of chores works. Fewer chores than people leaves free weeks — six people and five chores means one person is off each week. More chores than people sends the wheel round again, so some people draw two that week. Either way every chore is assigned every week.

The rotation is a function of the calendar, not of state. Person i draws slot (i − week) mod n and every n-th slot after it, where the slots are the chores padded with free weeks to a whole number of turns, and week counts Mondays since a fixed epoch. So over one cycle — one week per person — everyone does every chore exactly once and takes the same number of free weeks, and there is nothing to persist: a module restart, a redeploy, or running the command twice in one morning all read the same wheel. The cost is that the order is predictable to anyone who works it out; the benefit is that nobody can draw the ice maker three weeks running.

The roster order is the wheel. people is the order around the disc and the order in the message. Reordering it, or inserting someone in the middle, reshuffles who has what this week — append new people at the end. Removing someone shortens the cycle and likewise reshuffles.

Scheduling lives in the machine config, exactly as for the daily digest:

"jobs": [
  {
    "name": "weekly-chore-wheel",
    "schedule": "CRON_TZ=America/New_York 0 9 * * 1",
    "resource": "coffee",
    "method": "DoCommand",
    "command": { "send_weekly_chores": true }
  }
]

The CRON_TZ= caveat from the digest applies here too, with one wrinkle of its own: the week number comes from the calendar date in the timezone the command runs in. A job firing at 9am New York and a hand-run at 9am London on the same Monday agree; a hand-run late Sunday evening in New York is still last week. Fire it on Mondays and none of this matters.

Pose reference

Which poses the service can reach, and which frame each moves. The frame is the switch's component_name; the table records what those switches are expected to carry.

filter frame — the pose_switcher_name switch

Pose	Used for	Required when
grinder_approach, grinder_activate	grind	always
decaf_grinder_approach, decaf_grinder_activate	decaf grind	can_serve_decaf
tamper_approach, tamper_activate	tamp	always
coffee_approach, coffee_in, coffee_locked_final	lock the portafilter	always
coffee_shake, coffee_shake_left	dislodge a stuck puck while unlocking (the two lean opposite ways)	portafilter_shake_sec > 0
close_to_cleaning, approach_to_cleaning_scrapper, cleaning_scrapper_active, approach_to_cleaning_brush, cleaning_brush_active	clean	always — rewind recovery cleans too
purge_approach, purge_press	hold the 1 CUP button to keep the machine at brew temperature	keepalive is configured
home	end of cycle	always

grip-point frame — the claws_pose_switcher_name switch

Pose	Used for	Required when
filter_released, coffee_locked_final	hand the portafilter off and re-grab it	always
coffee_button_approach, coffee_button_on, coffee_button_off	hold the toggle down through the brew	has_separate_brew_buttons: false
espresso_button_approach/_press, lungo_button_approach/_press	poke the per-shot buttons	has_separate_brew_buttons: true
cup_under_machine_approach, cup_ready_for_coffee	place and retrieve the cup	always
ice_machine_approach, ice_machine_dispense, staging_approach, staging, pour_approach, pour	iced coffee	can_serve_iced
milk_pour_approach, milk_pour	pour milk into the staged glass	can_serve_iced_latte

The milk bottle needs no authored pickup or return poses: it is vision-detected inside the fridge and set back down at the centroid it was grasped at.

cam frame — the observe switches

Pose	Switch	Required when
cup_observe	camera_observe_pose_switcher_name	always
glass_observe	glass_observe_pose_switcher_name	can_serve_iced
milk_observe	milk_observe_pose_switcher_name	can_serve_iced_latte

Both observe switches sweep every pose they carry, so additional vantages alongside these are used even though only these are required.

Note that coffee_locked_final exists on both the filter and claws switches as two genuinely different poses in two different frames.

---

Model: viam:beanjamin:dial-control-motion

API: rdk:service:generic

Translates Stream Deck dial inputs into relative arm motions. Each dial tick contributes a step (mm for translations, degrees for rotations) along the chosen axis. The service tracks the absolute dial position between calls to determine direction (handling rollover at the dial range boundaries) and accumulates pending motion in a per-axis bucket. A background drain loop flushes accumulated motion to the arm at drain_interval_ms, applying a per-axis acceleration multiplier — single detents stay at 1× for fine control, while rapid spinning amplifies motion non-linearly.

Configuration

{
  "arm_name": "my-arm",
  "dial_move_x_mm": 5,
  "dial_move_y_mm": 5,
  "dial_move_z_mm": 5,
  "dial_move_orientation_mm": 5,
  "dial_move_rx_deg": 2,
  "dial_move_ry_deg": 2,
  "dial_move_rz_deg": 2,
  "dial_max_position": 100,
  "drain_interval_ms": 20,
  "accel_threshold_count": 1,
  "accel_max_multiplier": 10,
  "accel_exponent": 1.5,
  "accel_smoothing_alpha": 0.4
}

Name	Type	Required	Default	Description
arm_name	string	Yes	—	Name of the arm component to move.
dial_move_x_mm	float	No	1	Base millimeters per dial detent on the X axis.
dial_move_y_mm	float	No	1	Base millimeters per dial detent on the Y axis.
dial_move_z_mm	float	No	1	Base millimeters per dial detent on the Z axis.
dial_move_orientation_mm	float	No	1	Base millimeters per dial detent along the tool's orientation vector.
dial_move_rx_deg	float	No	1	Base degrees per dial detent rotating around the body's local X.
dial_move_ry_deg	float	No	1	Base degrees per dial detent rotating around the body's local Y.
dial_move_rz_deg	float	No	1	Base degrees per dial detent rotating around the body's local Z.
dial_max_position	float	No	100	Maximum dial position value, used for rollover detection.
drain_interval_ms	int	No	20 (50 Hz)	Flush cadence in milliseconds. Detents arriving within a window are summed before being applied.
accel_threshold_count	float	No	1	Translation: smoothed-detent count at which multiplier reaches 1×. Below this it's pinned to 1×. Default of 1 ramps from the first detent.
accel_max_multiplier	float	No	10	Translation: upper bound on the acceleration multiplier at high spin rates.
accel_exponent	float	No	1.5	Translation: curve shape, 1 linear, 2 quadratic. Multiplier = clamp((smoothed/threshold)^exponent, 1, max).
accel_smoothing_alpha	float	No	0.4	Translation: EWMA factor in (0, 1] across drain windows. 1 = no smoothing (instant); smaller = smoother / laggier.
accel_rotation_threshold_count	float	No	translation	Rotation override for accel_threshold_count. Falls back to the translation value if unset.
accel_rotation_max_multiplier	float	No	translation	Rotation override for accel_max_multiplier. Falls back to the translation value if unset.
accel_rotation_exponent	float	No	translation	Rotation override for accel_exponent. Falls back to the translation value if unset.
accel_rotation_smoothing_alpha	float	No	translation	Rotation override for accel_smoothing_alpha. Falls back to the translation value if unset.

DoCommand

dial_move_x / dial_move_y / dial_move_z - Enqueue a translation along the named axis from a Stream Deck dial value. The first call for a given axis calibrates the dial position and does not move the arm.

{"dial_move_x": 50}

Returns {"status": "queued", "axis": "x", "step": 5.0} or {"status": "dial_initialized", "axis": "x", "position": 50} on first call.

dial_move_orientation - Enqueue a translation along the current tool orientation vector.

{"dial_move_orientation": 50}

dial_move_rx / dial_move_ry / dial_move_rz - Enqueue a rotation around the named world axis. Step magnitude is in degrees per detent.

{"dial_move_rx": 50}

toggle_axis_mode - Flip the dial-mode for X/Y/Z dials between translation and rotation. Bind this to a Stream Deck button to repurpose the dials live. While in rotation mode, dial_move_x is routed to rx (and similarly for y/z); dial_move_orientation is unaffected.

{"toggle_axis_mode": true}

Returns {"status": "toggled", "axis_mode": "rotation"}.

set_axis_mode - Set the mode explicitly (idempotent). Value must be "translation" or "rotation".

{"set_axis_mode": "rotation"}

Returns {"status": "set", "axis_mode": "rotation"}.

get_axis_mode - Read the current mode without changing it.

{"get_axis_mode": true}

Returns {"axis_mode": "translation"}.

Removed: dial_move_speed no longer exists. The new acceleration model (accel_threshold_count / accel_max_multiplier / accel_exponent) replaces it. Stream Deck profiles bound to dial_move_speed will receive an error and need to be remapped.

---

Model: viam:beanjamin:maintenance-sensor

API: rdk:component:sensor

Reports whether the system is safe for maintenance. Returns is_safe: true only when the arm is not moving, no order is running, and the queue is empty. Useful for gating maintenance workflows or triggering alerts.

Configuration

{
  "coffee_service_name": "coffee",
  "arm_name": "my-arm"
}

Name	Type	Required	Description
coffee_service_name	string	Yes	Name of the viam:beanjamin:coffee service to query for queue/running state.
arm_name	string	Yes	Name of the arm component to check for physical movement.

Readings

Returns a single reading:

{"is_safe": true}

is_safe is false when any of the following are true:

• The arm is physically moving
• An order is currently running
• There are orders in the queue

---

Model: viam:beanjamin:order-sensor

API: rdk:component:sensor

Receives a summary of each order attempt from the viam:beanjamin:coffee service. Configure the coffee service with order_sensor_name set to this component’s name, and add this sensor under the coffee resource’s depends_on.

Each reading is returned at most once from Readings. When there is no queued reading, Readings returns data.ErrNoCaptureToStore (and a nil readings map), which Data Management treats as “nothing to store” until the next order completes.

Configuration

{}

No attributes. Wire the sensor through the coffee service as described above.

Readings

With nothing queued, Readings returns ErrNoCaptureToStore and no readings map (clients should use data.IsNoCaptureToStoreError in Go).

After each order attempt completes (success, failure, or panic), the next Readings call returns something like:

{
  "order_id": "<uuid>",
  "drink": "espresso",
  "customer_name": "Alice",
  "modified_customer_name": "Alise",
  "order_ok": true,
  "operator_cancelled": false,
  "error_message": "",
  "failed_step": "",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "decaf": false,
  "start_time": "2026-04-01T12:00:00.000000000Z",
  "end_time": "2026-04-01T12:02:05.000000000Z",
  "duration_ms": 125000
}

start_time and end_time are UTC RFC3339Nano timestamps: wall clock from when queue processing begins for that order through when the attempt finishes (greeting, drink prep, completion speech). duration_ms matches end_time − start_time. On failure, order_ok is false and error_message is set; panics use a panic: ... message. When successful, error_message is an empty string.

The remaining fields exist to support observability (per-step error rates and failure investigation):

• customer_name — the name the customer gave, and the key to group on for per-customer aggregation. Never group on modified_customer_name.
• modified_customer_name — the misspelling this particular order was shown and told, recorded so a support question ("my cup said Alise") can be traced back to an order. It is re-rolled per order, so grouping on it gives one customer a separate bucket per drink. Empty when the caller never misspelled. Readings written before this field existed have a misspelling in customer_name itself and cannot be regrouped.
• failed_step — the step label the order errored at (e.g. "Brewing", "Grinding"), matching the setStep labels surfaced through get_queue. Empty on success. Count readings by failed_step to see where orders die.
• operator_cancelled — true when the failure was an operator cancel (a context.Canceled interruption), not a genuine fault. Exclude these from step error-rate metrics so intentional cancellations don't inflate failure counts. failed_step is still populated (it marks where the cancel interrupted).
• trace_id — the OpenTelemetry trace ID for the order. Use it to jump from a failed reading to the order's full distributed trace (every motion plan and step span). Empty if no trace context was present.
• decaf — whether the order took the decaf grinder branch, so you can tell why a given step ran (or didn't) without cross-referencing the coffee service config. Derived from the drink.

A per-step error rate is then count(failed_step == X AND NOT operator_cancelled) / count(all orders).

---

Model: viam:beanjamin:customer-detector

API: rdk:service:generic

Identifies return customers using facial recognition. Wraps the viam:vision:face-identification vision service to register customer faces (associated with a name and email) and later identify them when they return.

Prerequisites

• A configured camera component.
• The viam:vision:face-identification module added as a vision service, with its picture_directory pointing to <data_dir>/known_faces.

Configuration

{
  "camera_name": "<string>",
  "vision_service_name": "<string>",
  "data_dir": "<string>",
  "confidence_threshold": <float>,
  "min_face_area_fraction": <float>
}

Name	Type	Required	Description
camera_name	string	Yes	Name of the camera component used to capture customer photos.
vision_service_name	string	Yes	Name of the face-identification vision service dependency.
data_dir	string	Yes	Directory for storing known face images and customer records. Must match the vision service's picture_directory parent (i.e. the vision service's picture_directory should be <data_dir>/known_faces).
confidence_threshold	float	No	Minimum confidence score to consider a face match. Defaults to 0.5.
min_face_area_fraction	float	No	Minimum fraction of the (center-cropped) image area a detected face bounding box must cover to be considered for identification. Defaults to 0.08 (face spans ~28% of the frame linearly).

Example Configuration

{
  "camera_name": "customer-cam",
  "vision_service_name": "face-detector",
  "data_dir": "/data/customers",
  "confidence_threshold": 0.6,
  "min_face_area_fraction": 0.08
}

The face-identification vision service should be configured with picture_directory set to /data/customers/known_faces (matching the data_dir above). Both modules must share this path so the customer-detector can write face images that the vision service reads.

DoCommand

register_customer — Capture a single photo from the camera, save it as a known face, and associate it with the customer's name and email. Call this multiple times during a registration session to capture different angles (front, left, right, etc.). Does not trigger embedding recomputation — call finish_registration when done.

{
  "register_customer": {
    "name": "Alice Smith",
    "email": "alice@example.com"
  }
}

Returns:

{
  "registered": "alice@example.com",
  "name": "Alice Smith",
  "image_path": "/data/customers/known_faces/alice@example.com/face_1.jpeg"
}

finish_registration — Call after capturing all face images for a customer. Triggers the vision service to recompute its embeddings so the new faces become recognisable.

{"finish_registration": "alice@example.com"}

Returns:

{"email": "alice@example.com", "name": "Alice Smith", "face_images": 5}

identify_customer — Capture a photo and attempt to match the face against registered customers.

{"identify_customer": true}

Returns (match found):

{
  "identified": true,
  "name": "Alice Smith",
  "email": "alice@example.com",
  "confidence": 0.87,
  "is_registered": true
}

Returns (no match):

{
  "identified": false,
  "message": "no known customer detected",
  "num_detections": 0
}

list_customers — List all registered customer emails.

{"list_customers": true}

Returns:

{"customers": ["alice@example.com", "bob@example.com"], "count": 2}

remove_customer — Remove a customer and their face images.

{"remove_customer": "alice@example.com"}

Returns:

{"removed": "alice@example.com"}

record_order — Append a completed drink to a customer's order history (the data behind "the usual"). The coffee service calls this automatically after a successful brew when customer_detector_name is configured and the order carried a customer_email; you can also call it directly. An unknown email is a no-op (not an error), so it's safe to call for anonymous walk-ups.

{"record_order": {"email": "alice@example.com", "drink": "espresso"}}

Returns:

{"recorded": true, "email": "alice@example.com", "drink": "espresso"}

get_usual — Return a customer's usual drink, derived from their recorded order history. Returns {"has_usual": false} when the customer is unknown or has no history.

{"get_usual": "alice@example.com"}

Returns:

{"has_usual": true, "drink": "espresso", "count": 7}

get_info — Return static service info. Currently {"camera_name": <short name>} for the camera the detector is wired to.

{"get_info": true}

Status()

Status() reports the customer currently in front of the camera and their usual, so a poller (notably viam:conversation-bundle:voice-command's command_status) can greet them by name and offer their usual. It runs a best-effort identification and folds in get_usual; results are cached briefly so per-turn polling doesn't re-run the vision model on every call.

When a registered customer is recognized:

{
  "recognized": true,
  "name": "Alice Smith",
  "email": "alice@example.com",
  "confidence": 0.87,
  "usual_drink": "espresso",
  "usual_count": 7
}

Otherwise: {"recognized": false}.

Storage

Customer records (name, email, image directory, order history) are persisted to <data_dir>/customers.json. Order history is capped at the most recent 50 entries per customer. Face images are stored under <data_dir>/known_faces/<email>/ — one subdirectory per customer, which is the directory structure the face-identification vision service expects. Registering the same customer multiple times adds additional face samples, improving recognition accuracy.

---

Development

When iterating on poses, we recommend using the built-in viam CLI motion commands to query and test arm positions on a running machine.

Note: --organization , --location, and --machine will be infered from the part ID

Print motion service status

viam robot part motion print-status \
  --organization <org> \
  --location <location> \
  --machine <machine> \
  --part <part>

Get the current pose of a component

viam robot part motion get-pose \
  --organization <org> \
  --location <location> \
  --machine <machine> \
  --part <part> \
  --component <component-name>

Move a component to a pose

viam robot part motion set-pose \
  --organization <org> \
  --location <location> \
  --machine <machine> \
  --part <part> \
  --component <component-name> \
  -x <mm> -y <mm> -z <mm> \
  --ox <float> --oy <float> --oz <float> --theta <degrees>

Note: Only the pose values specified will be modified. Example if you only set -x 100, it will move the component by just changing the X value of its current pose

Once you've found the right poses, add them to your multi-poses-execution-switch configuration.

The web app's calibration view at ?view=calibrate lists which poses belong to which frame on a given machine, and which of them are set by hand rather than derived from another pose. It also polls the live filter / grip-point / cam positions off the running machine and offers each as a copyable pose, so the jog-read-paste loop doesn't need the CLI. The pose list comes from a manifest generated from the machines' live app config — regenerate it with make web-app-manifest whenever a pose is added, removed, renamed, or re-baselined.

Motion planning timeout

Every plan the coffee service makes — direct move, pivot, circular, no-spill carry, and each step of the fridge-door sweep — is capped at 15 seconds (motionPlanTimeout in coffee/motion.go), as is each multi-poses-execution-switch SetPosition move. RDK's own default is 300s, which makes a plan that will never succeed indistinguishable from one still searching, and holds the brew cycle open long past the point where the drink is worth serving.

A plan that overruns fails with motion planning failed (<label>, after <duration>), and every successful plan logs planned <label> in <duration>, so the logs say which it was. A timeout is almost always an unreachable goal rather than a planner that needed longer; if a legitimately hard plan starts hitting the cap, raise the constant rather than working around it per call site.

List recent orders

order_sensor_name writes one reading per order attempt, which makes the recent brew history queryable without opening the data page:

make orders LIMIT=20

#  When (UTC)        Result     Drink       Customer  Dur    Failed step     Order ID
-  ----------------  ---------  ----------  --------  -----  --------------  ------------------------------------
1  2026-09-04 13:16  OK         espresso    —         202s                   aee279d8-fc5f-4d2b-934c-50c8c0037108
2  2026-09-04 13:20  FAILED     espresso    —         122s   Serving         2917a819-3f9b-4955-8ac1-1624482c6067
3  2026-09-04 13:35  CANCELLED  espresso    —         2.1s   Grinding        3ea3deb5-5f12-4621-944e-8fabc94b876c
4  2026-09-04 13:49  OK         espresso    —         199s                   47b90796-1d45-4ab8-bf84-7f2886dfacc8
5  2026-09-04 16:49  OK         espresso    Adam      199s                   53f5ef7a-202a-4da3-87de-bbecf3a283fd

3 OK · 1 failed · 1 operator-cancelled

Rows are in brew order, oldest first, so # reads as the sequence the machine actually ran. An order that isn't order_ok is split into FAILED (a genuine fault) and CANCELLED (an operator stopped it). A — customer is an order placed outside the kiosk, which has no name attached. Sub-second durations are printed in milliseconds rather than rounded to 0s, since the instant failures are usually the interesting ones.

Copy an order ID straight into fetch-order:

make orders LIMIT=50 ORDERS_FLAGS="--errors"
make fetch-order ORDER=<the id from that table>

Flag	Description
--limit	How many of the most recent orders to show. Defaults to 20.
--errors	Print each unsuccessful order's error_message under its row.
--newest-first	List newest first instead of in brew order. Drops the # column, which would otherwise count backwards.
--part-id	Only show orders from one machine part. Defaults to every part in the org.
--org-id	Viam organization to query. Defaults to $VIAM_ORG_ID, else the viam-dev org the beanjamin machines live in.
--viam	Path to the viam CLI binary. Defaults to viam on PATH.

This reads the sensor's tabular data through viam data query tabular mql. It has to be MQL, not SQL — viam data query tabular sql cannot resolve the nested data.readings.* paths the readings live under.

Fetch one order's data

Every plan the module makes while save_motion_requests_dir is set is synced to the data page tagged with its order ID, step, motion type, and planning outcome (see the save_motion_requests_dir row above). To pull one order's plans down for offline debugging:

make fetch-order ORDER=5fb95a4c-83f8-4e66-862d-52cbca842ed5

That shells out to viam data export binary filter --tags <order-id> (so a logged-in viam CLI is required), then flattens the export's tag= directory tree into a single ./<order-id>/ directory, renaming each file to <index>-<timestamp>-<step>-<motion>-<outcome>.json. One order is roughly 70 plans / 80 MB, and the directories are gitignored:

5fb95a4c-83f8-4e66-862d-52cbca842ed5/
  001-20260903_141523.123-locking_portafilter-move-success.json
  002-20260903_141527.880-locking_portafilter-carry-success.json
  003-20260903_141602.410-grinding-circular-success.json
  004-20260903_141640.005-brewing-move-failure.json

The timestamp is the machine's local clock, not UTC, so these names do not line up directly with the UTC times in the order-events sensor readings. The leading index makes alphabetical order execution order, so the directory reads top-to-bottom as the order's motion history.

Each file round-trips through RDK's ReadRequestAndResponseFromFile, which means two concatenated JSON documents per file — the request (frame_system, goals, start_state, obstacles_in_world_frame, constraints, planner_options) followed by the response (path, trajectory), absent when planning failed. Plain jq . fails on them with "Extra data"; use jq -s to read the pair as an array.

Add WITH_VIDEO=1 for the camera clips, pass other flags through FETCH_FLAGS, or call the CLI directly:

make fetch-order ORDER=<order-id> WITH_VIDEO=1
go run ./cmd/cli fetch-order --help
go run ./cmd/cli fetch-order --out /tmp --with-video <order-id>

Flag	Description
--out	Parent directory to create <order-id>/ in. Defaults to the working directory.
--from	Reorganize an export destination you already downloaded instead of downloading again. Copies rather than moves, leaving your export tree intact.
--with-video	Also download the order's three camera clips (~130 MB of mp4). They share the order's tag but aren't plan requests, so the export asks only for application/json unless this is set. Clips keep the names the video store gave them (which embed a {"order_id": ...} blob) and sort after the numbered plans.
--viam	Path to the viam CLI binary. Defaults to viam on PATH.
--timeout	Seconds to allow the export. 0 (the default) leaves the viam CLI's own default in place.

Note that viam data export binary exits 0 having downloaded nothing when the account can't read the machine's location, so a missing order and a missing permission look alike from the outside. The command turns that into a no data found for order <id> error rather than reporting success, and leaves no directory behind. A truncated order ID looks identical from the outside, so check it is a full UUID (8-4-4-4-12) first.

Brightness shadow

The shipping measurement finds the ice surface as a brightness step — dark empty glass above, bright ice below — because a step survives a lighting change where an absolute cutoff does not. The cost is a dead band: the step needs a full ice_contrast_window of rows either side of a candidate, so it can only name rows ice_stop_row_px to ice_roi_y1 - window (595-622 at the defaults). A surface above the stop row is unnameable, which is why the dispense stops on the surface disappearing rather than on a row.

An absolute cutoff has no dead band, but it is exactly what a lighting change breaks. Set ice_brightness_thresh and it runs as a shadow: the same frames, read both ways, with only the contrast step deciding anything. Each watched dispense logs its first sighting, when its own stop test (row <= ice_stop_row_px) would have fired, and a closing comparison; differences in kind bump ice_shadow_disagreements, timing gaps do not. Nothing about the pin changes, so it is safe to leave on.

Pick the threshold first: check_ice_level prints the range of row brightness it sees, and the cutoff has to sit above an empty glass's brightest row and below ice's dimmest. 132 is the midpoint on the committed fixtures under one lighting condition — a seed, not a value.

Annotated frames

Every dispense inside an order saves the drawn frame under tag=<order-id>/tag=ice_dispense/tag=ice_<outcome>/; a hand-run action saves one only with annotate: true, and without an order ID to nest under. Outcome is stopped, timeout, surface_cap, vision_fallback, cancelled or check. Under a data-synced capture dir the data manager tags the upload from those segments, so frames filter on the data page by order and by how the run ended, like the motion-plan requests. vision_fallback and cancelled save the last frame that measured cleanly, captioned with its age.

Rows grow downward, so a surface drawn above the stop row is a full glass — which is why the step reports nothing there and the shadow reports a row. A log line says the stop row was reached; only the picture says whether it still matches how that glass was gripped.

To tune a stop row: glass in the jaws, move_to_ice_dispense with with_glass: true, then check_ice_level with annotate: true as often as you like. No ice spent, a drawn frame each time.