Skip to content

Cloud-hypervisor does not have necessary capabilities when installed from urunc-deploy #1052

Description

@cmainas

When using urunc-deploy to install all supported monitors in a Kubernetes cluster, cloud-hypervisor i s also installed. However, a boot will fail due to cloud-hypervisor not being able to set the MTU on the tap device.

6: ioctl (35106) failed: Operation not permitted (os error 1)

35106 is 0x8922 = SIOCSIFMTU, which requires CAP_NET_ADMIN. The installed binary does not have any file capabilities, so the --net device cannot be created and the VM aborts.

Steps to reproduce

  1. Install urunc on a node with urunc-deploy.
  2. Deploy a pod over urunc with an image using cloud-hypervisor as the monitor (e.g. harbor.nbfc.io/nubificus/urunc/nginx-cloud-hypervisor-linux-raw:latest)

The pod reports Running with one restart and is unreachable:

NAME        READY   STATUS    RESTARTS      AGE
urunc-chv   1/1     Running   1 (15s ago)   20s

First boot (kubectl logs --previous), after ~0.015s:

cloud-hypervisor: 0.015323s: <main> ERROR:.../cloud-hypervisor/src/lib.rs:23 -- Fatal error:
  VmBoot(VmBoot(DeviceManager(CreateVirtioNet(OpenTap(TapSetMtu(
    IoctlError(35106, Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" })))))))
Error: Cloud Hypervisor exited with the following chain of errors:
  0: Error booting VM
  1: The VM could not boot
  2: Error from device manager
  3: Cannot create virtio-net device
  4: Failed to open taps
  5: Setting MTU failed
  6: ioctl (35106) failed: Operation not permitted (os error 1)
  7: Operation not permitted (os error 1)

The surviving process after the restart has neither --net nor an ip= on the guest cmdline:

/opt/urunc/bin/cloud-hypervisor --memory size=268M,shared=on --cpus boot=1 \
  --kernel /cntrRootfs/.boot/kernel --console off --serial tty --seccomp false \
  --initramfs /urunit.conf --fs tag=fs0,socket=/tmp/vhostqemu \
  --cmdline panic=-1 console=ttyS0 root=fs0 rw rootfstype=virtiofs \
            retain_initrd URUNIT_CONFIG=/sys/firmware/initrd init=/urunit -- <cmd>

Workaround

Granting the capability on the node fixes it completely:

sudo setcap cap_net_admin+ep /opt/urunc/bin/cloud-hypervisor

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    K8s/ToolsRelated to container/cloud native tools, orchestratorsbugSomething isn't working

    Type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions