Hi Jegadish, The physical ConnectX-6 Dx card is on @zabaj1's server - that is where firmware `22.39.8002` and the standard in-tree kernel `mlx5_core` driver were running. On my testbed, I verified the sysfs and devlink call sequence.
The firmware difference explains why it didn't trigger on your test machine. In the `22.39.x` tree, Mellanox tightened command mailbox serialization and queue teardown during e-switch mode transitions. On older `22.36.1010`, queue teardown is faster, so the race window is much smaller. The race also depends on VF count. With 4, 8, or more VFs, spawning and immediately unbinding that many netdevs quickly saturates the firmware mailbox. Also, disabling autoprobe before creating VFs is the documented sequence in kernel PCI docs and NVIDIA DOCA switchdev guides: 1. `echo 0 > /sys/class/net/<PF>/device/sriov_drivers_autoprobe` 2. `echo <N> > /sys/class/net/<PF>/device/sriov_numvfs` 3. `devlink dev eswitch set <DEV> mode switchdev` 4. Rebind VFs. Spawning VFs with autoprobe=0 stops the kernel from attaching drivers in legacy mode only to unbind them a millisecond later. It avoids the race completely on any firmware version and any VF count. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2166228 Title: Mellanox NIC is unable to enter switchdev mode To manage notifications about this bug go to: https://bugs.launchpad.net/netplan/+bug/2166228/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
