[zeeshanlakhani]

This adds multicast replication to sidecar-lite, emulating DPD's sidecar.p4 on dendrite @ zl/mcast-swadm, following RFD 488.

We manage 2 bitmaps per underlay group (external and underlay), for which replication is driven by p4rs replicate (dep @ p4#240). Ingress zeroes out at most one of these bitmaps determined by the geneve multicast option tag (tag 0 zeroes out underlay; tag 1 zeroes out external), and merges both.

Tag 0 (External) also decaps upon ingress. With no tag given, as on external NAT ingress packets, both bitmaps remain in place.

Only geneve-encapsulated multicast commits to replication. Multicast packets that miss ingress NAT drop on the empty bitmap, matching DPD's drop_mcastv{4,6}_no_group semantics. Packet copies that remain encapsulated lose a hop on the outer IPv6 header, as modelled in MulticastRouter6.

External multicast is NAT-encapsulated into the underlay via nat_v{4,6_mcast(_untagged), reusing forward_to_sled. Tagged and untagged variants exist as separate tables since x4c cannot key directly on header validity. The encap piece now zeroes out the outer src, traffic class, flow label, and inner eth src, like DPD's add_encap_headers.

Egress decap checks ff04::/64 and tag=2, matching DPD's egress mcast_tag_check, with per-port decap from a RID-keyed bitmap (tbl_decap_ports) and optional VLAN. tag=2 decapsulation also rejects inner unicast. tag=0 copies (already decapped in ingress) use the table for the VLAN. Dst MAC derivation (RFC 1112, RFC 2464) and per-port src MAC rewriting run on every packet copy.

Includes:

• The parser rejects ff01 addresses outright, as well as ff02 destined packets with a hop limit of 0 (ff02 does still go to the scrimlet).
• Multicast packets with a TTL/hop_limit <= 1 get dropped in ingress after the fwd_from_scrimlet return, letting scrimlet transmissions pass through.
• Both decapsulation paths drop inner TTL <= 1. IPv4 checksums use RFC 1624 (eqn. 3): a checksum of 0xfeff turns into 0x0000 after decrementing the TTL (XOR used here due to ~ not being parsable by x4c).
• scadm gets add/remove/dump for replication (v6), source filters, mcast NAT, decap ports, src MAC, and asic-id-to-port (for logical mapping).
• Unicast modifications:
  • The router drops TTL/hop_limit <= 1 and decrements remaining packets on routes to nonzero ports, updating the IPv4 checksum.
    • Port 0 (the scrimlet) skips both the TTL check and decrement.
    • DPD's Router4/6 still decrement service-bound packets.
  • The NAT encap zeroing described above is shared with unicast NAT.
  • Packets with a parsed Geneve header skip ingress NAT, including unicast.

References

This work depends on the P4(rs) work in #240 and softnpu's zl/multicast.

[rcgoodfellow]

Our disposition on VLANs to date is that we'll do tagging to participate in segmented upstream networks e.g., the vlan variants of the forwarding actions on routing tables. But we've not done anything in terms of ingress isolation/segmentation with VLANs. What is driving keying ingress NAT tables for mcast on vlan ids?

[zeeshanlakhani]

Yep, this adds ingress VLAN filtering for multicast. It came from oxidecomputer/dendrite#195, which fixed oxidecomputer/dendrite#107 (a packet arriving on one VLAN got NAT-encapped into a group configured for another). In the thread, @FelixMcFelix suggested using a single exact VLAN entry per group for customer ingress, and that's what merged. Geneve packets bypass NatIngress and are decapsulated later without returning to it, which means the lookup sees the incoming (customer's) packet's original VLAN state.

In dpd, an external group's optional vlan_id feeds the NAT ingress key and egress route, and gets propagated to the underlay group's external decap bitmap. This linked setting controls ingress admission and VLAN tagging on decapsulated external packet copies.

I carried this over here for emulation parity with dendrite's sidecar (mcast_nat_v4_wrong_vlan_miss covers this miss).

[rcgoodfellow]

a packet arriving on one VLAN got NAT-encapped into a group configured for another

This implies we are attempting to support overlapping address spaces? Supporting overlapping address spaces is not something we support, independent of address type. I think would be a pretty big lift in control plane to get there. It seems to me that the destination group address alone needs to be sufficient to determine where a packet needs to go within the rack on ingress.

[zeeshanlakhani]

I didn't mean to imply overlapping spaces actually. More so, the focus is on VLAN translation. Dpd rejects duplicate group IPs, and the VLAN match is about controlling admission. The destination group address already decides where it goes in the rack.

As mentioned, the check came from oxidecomputer/dendrite#195, fixing oxidecomputer/dendrite#107, which recorded a legit, failing test. The intent was to prevent cross-VLAN translation; #107 captured an encapsulated packet leaving an external port.

If ingress should accept the packet regardless of VLAN, I'm down to update dendrite off main and here as well. We'd need to enforce the external-to-external replication restriction separately, while also allowing delivery to internal receivers. Thoughts?

[zeeshanlakhani]

There are a few alternative directions I take this too @rcgoodfellow, but I thought to just bring this up first as a question. @FelixMcFelix may have thoughts here too?

[rcgoodfellow]

If ingress should accept the packet regardless of VLAN, I'm down to update dendrite off main and here as well.

This is what we do for unicast. We are a router, and it's not generally expected that VLANs will be preserved across a routed boundary. There is no concept of VLAN isolation within the rack, so there are no VLAN-based boundaries we can meaningfully enforce on ingress.

I understand there was a failing test before, but it's not clear to me that we want multicast ingress tied to VLANs in the way that the test was specifying.

[zeeshanlakhani]

I gotcha.

[zeeshanlakhani]

I'll update here and with a PR off dendrite/main too.

[zeeshanlakhani]

@rcgoodfellow updated on this side. The dendrite fix will go in the mcast stack of PRs.