On 5/11/21 8:17 PM, Klaus Jensen wrote: > Hi all, > > I need some help with grok'ing qdev busses. Stefan, Michael - David > suggested on IRC that I CC'ed you guys since you might have solved a > similar issue with virtio devices. I've tried to study how that works, > but I'm not exactly sure how to apply it to the issue I'm having.
The experts on this topic are Peter/Markus/Eduardo. > Currently, to support multiple namespaces on the emulated nvme device, > one can do something like this: > > -device nvme,id=nvme-ctrl-0,serial=foo,... > -device nvme-ns,id=nvme-ns-0,bus=nvme-ctrl-0,... > -device nvme-ns,id-nvme-ns-1,bus=nvme-ctrl-0,... > > The nvme device creates an 'nvme-bus' and the nvme-ns devices has > dc->bus_type = TYPE_NVME_BUS. This all works very well and provides a > nice overview in `info qtree`: > > bus: main-system-bus > type System > ... > dev: q35-pcihost, id "" > .. > bus: pcie.0 > type PCIE > .. > dev: nvme, id "nvme-ctrl-0" > .. > bus: nvme-ctrl-0 > type nvme-bus > dev: nvme-ns, id "nvme-ns-0" > .. > dev: nvme-ns, id "nvme-ns-1" > .. > > > Nice and qdevy. > > We have since introduced support for NVM Subsystems through an > nvme-subsys device. The nvme-subsys device is just a TYPE_DEVICE and > does not show in `info qtree` (I wonder if this should actually just > have been an -object?). Anyway. The nvme device has a 'subsys' link > parameter and we use this to manage the namespaces across the subsystem > that may contain several nvme devices (controllers). The problem is that > this doesnt work too well with unplugging since if the nvme device is > `device_del`'ed, the nvme-ns devices on the nvme-bus are unrealized > which is not what we want. We really want the namespaces to linger, > preferably on an nvme-bus of the nvme-subsys device so they can be > attached to other nvme devices that may show up (or already exist) in > the subsystem. IIUC, while we can have unattached drives, we can't (by design) have qdev unattached to qbus. Not sure this is a good suggestion (bad design IMO) but you could add a fake nvme qbus to hold the lingering nvme devices... > The core problem I'm having is that I can't seem to create an nvme-bus > from the nvme-subsys device and make it available to the nvme-ns device > on the command line: > > -device nvme-subsys,id=nvme-subsys-0,... > -device nvme-ns,bus=nvme-subsys-0 > > The above results in 'No 'nvme-bus' bus found for device 'nvme-ns', even > though I do `qbus_create_inplace()` just like the nvme device. However, > I *can* reparent the nvme-ns device in its realize() method, so if I > instead define it like so: > > -device nvme-subsys,id=nvme-subsys-0,... > -device nvme,id=nvme-ctrl-0,subsys=nvme-subsys-0 > -device nvme-ns,bus=nvme-ctrl-0 > > I can then call `qdev_set_parent_bus()` and set the parent bus to the > bus creates in the nvme-subsys device. This solves the problem since the > namespaces are not "garbage collected" when the nvme device is removed, > but it just feels wrong you know? Also, if possible, I'd of course > really like to retain the nice entries in `info qtree`. > > > Thanks, > Klaus