config-linux.md, Allowed Device list says of linux.resources.devices:
The runtime MUST apply entries in the listed order.
A reader naturally takes that to mean the array in config.json is the whole list and that the last entry to match a device decides the outcome, so a caller can allow broadly and then deny something specific. I would like to know whether that is what the specification intends, because at least one reference implementation adds rules of its own after the configured ones, and if that is the intended behaviour then the specification does not currently describe it.
runc appends its own allow rules to the end of the list, after everything from config.json, in CreateCgroupConfig:
// Append the default allowed devices to the end of the list.
for _, device := range defaultDevs {
c.Resources.Devices = append(c.Resources.Devices, &device.Rule)
}This is not a defect nobody noticed. The comment on the set being appended says so plainly, and explains why it has not been changed:
This behaviour is at the very least "questionable" (if not outright wrong) according to the runtime-spec.
Yes, we have to include certain devices other than the ones the user specifies, but several devices listed here are not part of the spec (including "mknod for any device"?!). In addition, these rules are appended to the user-provided set which means that users cannot disable this behaviour.
... unfortunately I'm too scared to change this now because who knows how many people depend on this (incorrect and arguably insecure) behaviour.
The complete set appended, read from that file at the commit linked above, is eleven entries, all of them allows: c *:* m and b *:* m, then c 1:3 (/dev/null), c 1:8 (/dev/random), c 1:7 (/dev/full), c 5:0 (/dev/tty), c 1:5 (/dev/zero), c 1:9 (/dev/urandom), c 136:* (pts), c 5:2 (/dev/ptmx) and c 10:200 (/dev/net/tun).
The practical consequence is that a bundle cannot deny any of those devices, nor deny mknod for anything, however it orders its own entries, because the appended rules always come last and therefore always win. A configuration that denies /dev/full is silently ineffective. So the ordering guarantee as written is not one a caller can rely on, and the "allowed device list" is not wholly under the caller's control.
Either reading seems defensible and I do not think it is my call which is right:
- The specification means what it says, the array in
config.json is the complete list, and a runtime that appends entries a caller cannot override diverges from it. In that case nothing in the specification needs to change, but the divergence is worth being explicit about, since implementers are clearly reading the current text and then doing something else for compatibility reasons.
- Runtimes are expected to supply rules of their own, for the devices they are required to supply among others. In that case the specification should say so, and say where those rules sit relative to the configured ones, and whether a configured entry is able to override them. Requiring the runtime's own rules to be applied first, so that a later configured entry can still override them, would keep the ordering guarantee meaningful while preserving the reason the rules exist.
There is a conformance-testing angle that prompted this. In opencontainers/runtime-tools#816 I added a check that verifies the device list by attempting access from inside the container rather than by reading it back out of devices.list, which is needed because the unified hierarchy exposes no such file. Choosing the device numbers for that test meant working around the appended set: a fixture that denied one of the devices listed above would fail against runc for reasons that have nothing to do with the property being tested. Having to design around it is what suggested the guarantee is not usable as stated.
To be clear about scope, this is a question about what the specification should say rather than a bug report against runc, and I have not filed anything there. If the appended behaviour is what implementations should be doing, the specification is the right place to describe it.
config-linux.md, Allowed Device list says of
linux.resources.devices:A reader naturally takes that to mean the array in
config.jsonis the whole list and that the last entry to match a device decides the outcome, so a caller can allow broadly and then deny something specific. I would like to know whether that is what the specification intends, because at least one reference implementation adds rules of its own after the configured ones, and if that is the intended behaviour then the specification does not currently describe it.runc appends its own allow rules to the end of the list, after everything from
config.json, inCreateCgroupConfig:This is not a defect nobody noticed. The comment on the set being appended says so plainly, and explains why it has not been changed:
The complete set appended, read from that file at the commit linked above, is eleven entries, all of them allows:
c *:* mandb *:* m, thenc 1:3(/dev/null),c 1:8(/dev/random),c 1:7(/dev/full),c 5:0(/dev/tty),c 1:5(/dev/zero),c 1:9(/dev/urandom),c 136:*(pts),c 5:2(/dev/ptmx) andc 10:200(/dev/net/tun).The practical consequence is that a bundle cannot deny any of those devices, nor deny
mknodfor anything, however it orders its own entries, because the appended rules always come last and therefore always win. A configuration that denies/dev/fullis silently ineffective. So the ordering guarantee as written is not one a caller can rely on, and the "allowed device list" is not wholly under the caller's control.Either reading seems defensible and I do not think it is my call which is right:
config.jsonis the complete list, and a runtime that appends entries a caller cannot override diverges from it. In that case nothing in the specification needs to change, but the divergence is worth being explicit about, since implementers are clearly reading the current text and then doing something else for compatibility reasons.There is a conformance-testing angle that prompted this. In opencontainers/runtime-tools#816 I added a check that verifies the device list by attempting access from inside the container rather than by reading it back out of
devices.list, which is needed because the unified hierarchy exposes no such file. Choosing the device numbers for that test meant working around the appended set: a fixture that denied one of the devices listed above would fail against runc for reasons that have nothing to do with the property being tested. Having to design around it is what suggested the guarantee is not usable as stated.To be clear about scope, this is a question about what the specification should say rather than a bug report against runc, and I have not filed anything there. If the appended behaviour is what implementations should be doing, the specification is the right place to describe it.