When using urunc-deploy to install all supported monitors in a Kubernetes cluster, cloud-hypervisor i s also installed. However, a boot will fail due to cloud-hypervisor not being able to set the MTU on the tap device.
6: ioctl (35106) failed: Operation not permitted (os error 1)
35106 is 0x8922 = SIOCSIFMTU, which requires CAP_NET_ADMIN. The installed binary does not have any file capabilities, so the --net device cannot be created and the VM aborts.
Steps to reproduce
- Install urunc on a node with
urunc-deploy.
- Deploy a pod over
urunc with an image using cloud-hypervisor as the monitor (e.g. harbor.nbfc.io/nubificus/urunc/nginx-cloud-hypervisor-linux-raw:latest)
The pod reports Running with one restart and is unreachable:
NAME READY STATUS RESTARTS AGE
urunc-chv 1/1 Running 1 (15s ago) 20s
First boot (kubectl logs --previous), after ~0.015s:
cloud-hypervisor: 0.015323s: <main> ERROR:.../cloud-hypervisor/src/lib.rs:23 -- Fatal error:
VmBoot(VmBoot(DeviceManager(CreateVirtioNet(OpenTap(TapSetMtu(
IoctlError(35106, Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" })))))))
Error: Cloud Hypervisor exited with the following chain of errors:
0: Error booting VM
1: The VM could not boot
2: Error from device manager
3: Cannot create virtio-net device
4: Failed to open taps
5: Setting MTU failed
6: ioctl (35106) failed: Operation not permitted (os error 1)
7: Operation not permitted (os error 1)
The surviving process after the restart has neither --net nor an ip= on the guest cmdline:
/opt/urunc/bin/cloud-hypervisor --memory size=268M,shared=on --cpus boot=1 \
--kernel /cntrRootfs/.boot/kernel --console off --serial tty --seccomp false \
--initramfs /urunit.conf --fs tag=fs0,socket=/tmp/vhostqemu \
--cmdline panic=-1 console=ttyS0 root=fs0 rw rootfstype=virtiofs \
retain_initrd URUNIT_CONFIG=/sys/firmware/initrd init=/urunit -- <cmd>
Workaround
Granting the capability on the node fixes it completely:
sudo setcap cap_net_admin+ep /opt/urunc/bin/cloud-hypervisor
When using
urunc-deployto install all supported monitors in a Kubernetes cluster, cloud-hypervisor i s also installed. However, a boot will fail due to cloud-hypervisor not being able to set the MTU on the tap device.35106is0x8922=SIOCSIFMTU, which requiresCAP_NET_ADMIN. The installed binary does not have any file capabilities, so the--netdevice cannot be created and the VM aborts.Steps to reproduce
urunc-deploy.uruncwith an image using cloud-hypervisor as the monitor (e.g. harbor.nbfc.io/nubificus/urunc/nginx-cloud-hypervisor-linux-raw:latest)The pod reports
Runningwith one restart and is unreachable:First boot (
kubectl logs --previous), after ~0.015s:The surviving process after the restart has neither
--netnor anip=on the guest cmdline:Workaround
Granting the capability on the node fixes it completely: