Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
On 6/17/2015 at 06:27 PM, in message 55814b9f.6070...@eu.citrix.com, George Dunlap george.dun...@eu.citrix.com wrote: On 06/17/2015 04:59 AM, Juergen Gross wrote: On 06/16/2015 06:34 PM, George Dunlap wrote: On Tue, Jun 16, 2015 at 4:59 PM, Ian Jackson ian.jack...@eu.citrix.com wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]): Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. That does seem to be the case: What seems to be the case -- that it contains the global bus, or not? Looking at the output of udevadm monitor --property for sdc1 (on another plug): DEVPATH=/devices/pci:00/:00:1d.7/usb1/1-1/1-1:1.0/host22/target22:0:0 /22:0:0:0/block/sdc/sdc1 ID_PATH=pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0 ID_PATH_TAG=pci-_00_1d_7-usb-0_1_1_0-scsi-0_0_0_0 I don't know where that ID_PATH comes from. It looks like that's constructed in udev: http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-path_i d.c See handle_usb() (line 542) in particular. If I'm reading it right, what it basically does it take the [bus]-[port], and replace it with usb-0:[port]. (IOW, the '0' is hard-coded.) Also, if I'm reading it right, this won't work properly for Juergen's system that has two USB devices at the same pci address -- they'll both end up resolving to [pciaddr]-usb-0:[whatever]. Juergen -- can you give this a try? Run udevadm info on the two USB busses that share the same PCI slot, and see if the ID_PATH is the same for both? This was the correct question. :-) The ID_PATH is the same, but: It turns out that the two busses are for one physical hcd. One is the bus for USB 3.0, the other one for USB 2.0. I guess the bus is just selected by the capability of the plugged in device. So the [port] in usb-0:[port] is unique across the two logical busses. Right -- I think I had come to the conclusion that would probably be the case later yesterday evening. Glad to have it confirmed. So in addition to our other identifiers, stealing udev's naming scheme should work, and would be useful. The next challenge would be how to implement it effectively. It's already udev's job to make sure that a string is unique -- as much as possible we want to leverage their efforts, rather than re-implementing our own. One thing that Ian suggested was that we could add a udev rule that would create links from the ID_PATH of the usb device into the sysfs device node. (Both seem to be available in the udev rule environment.) That would allow us to easily translate the ID_PATH into the other things we might want (bus-port, bus.addr, c) and vice versa. But I think all that will certainly not be done by the feature freeze. The core functionality of Chunyan's series, wrt the pvusb functionality is complete; as with my HVM series before, it's mainly the interface that is being discussed. There are two options here: Chunyan could try to implement the interface we've been discussing here (assuming that she doesn't have any objections to what we've described); No, I don't have any objections. I can do that. - Chunyan or, I could take Chunyan's series, try to implement what we've been talking about, and then add in the HVM functionality as well. What does everyone think? -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
On 06/17/2015 04:59 AM, Juergen Gross wrote: On 06/16/2015 06:34 PM, George Dunlap wrote: On Tue, Jun 16, 2015 at 4:59 PM, Ian Jackson ian.jack...@eu.citrix.com wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]): Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. That does seem to be the case: What seems to be the case -- that it contains the global bus, or not? Looking at the output of udevadm monitor --property for sdc1 (on another plug): DEVPATH=/devices/pci:00/:00:1d.7/usb1/1-1/1-1:1.0/host22/target22:0:0/22:0:0:0/block/sdc/sdc1 ID_PATH=pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0 ID_PATH_TAG=pci-_00_1d_7-usb-0_1_1_0-scsi-0_0_0_0 I don't know where that ID_PATH comes from. It looks like that's constructed in udev: http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-path_id.c See handle_usb() (line 542) in particular. If I'm reading it right, what it basically does it take the [bus]-[port], and replace it with usb-0:[port]. (IOW, the '0' is hard-coded.) Also, if I'm reading it right, this won't work properly for Juergen's system that has two USB devices at the same pci address -- they'll both end up resolving to [pciaddr]-usb-0:[whatever]. Juergen -- can you give this a try? Run udevadm info on the two USB busses that share the same PCI slot, and see if the ID_PATH is the same for both? This was the correct question. :-) The ID_PATH is the same, but: It turns out that the two busses are for one physical hcd. One is the bus for USB 3.0, the other one for USB 2.0. I guess the bus is just selected by the capability of the plugged in device. So the [port] in usb-0:[port] is unique across the two logical busses. Right -- I think I had come to the conclusion that would probably be the case later yesterday evening. Glad to have it confirmed. So in addition to our other identifiers, stealing udev's naming scheme should work, and would be useful. The next challenge would be how to implement it effectively. It's already udev's job to make sure that a string is unique -- as much as possible we want to leverage their efforts, rather than re-implementing our own. One thing that Ian suggested was that we could add a udev rule that would create links from the ID_PATH of the usb device into the sysfs device node. (Both seem to be available in the udev rule environment.) That would allow us to easily translate the ID_PATH into the other things we might want (bus-port, bus.addr, c) and vice versa. But I think all that will certainly not be done by the feature freeze. The core functionality of Chunyan's series, wrt the pvusb functionality is complete; as with my HVM series before, it's mainly the interface that is being discussed. There are two options here: Chunyan could try to implement the interface we've been discussing here (assuming that she doesn't have any objections to what we've described); or, I could take Chunyan's series, try to implement what we've been talking about, and then add in the HVM functionality as well. What does everyone think? -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API): Remember that the path you gave in your previous e-mail isn't the path for the *usb device*, it's the path for the *block device*. It contains a PCI address, but it looks like it also contains part of the USB topology. Are you sure that's actually a stable interface, or does it just happen that on your hardware the discovery always happens in the same order? The block device is (in path terms) underneath the usb device, obviously. Not all of that path is relevant to identifying the USB device. On my system /sys/bus/usb/devices/2-3.3 is a link to /sys/devices/pci:00/:00:1d.7/usb2/2-3/2-3.3/. This contains the pci bus address, but it also contains the bus number, which we've just said may be unstable across reboots. You mean the 2 in `usb2' ? I think that bus number is the bus number within the controller, not globally. I suppose it might be possible to specify buspci,port -- the pci address of the root bus, and the topology from there. In theory I guess that should be stable? Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. In any case, at the moment you're essentially inventing from whole cloth a new way of specifying USB devices that (as far as I know) isn't supported by any other program that uses USB. If you can't specify the device by hardware path, you can't specify it deterministically. And as you can see it _is_ supported by other programs that use USB. mount can use it! I think the hardware path to the controller, at least, should be treated as an opaque OS-specific string. It might have a different format on BSD. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
On 06/16/2015 02:37 PM, Ian Jackson wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API): Remember that the path you gave in your previous e-mail isn't the path for the *usb device*, it's the path for the *block device*. It contains a PCI address, but it looks like it also contains part of the USB topology. Are you sure that's actually a stable interface, or does it just happen that on your hardware the discovery always happens in the same order? The block device is (in path terms) underneath the usb device, obviously. Not all of that path is relevant to identifying the USB device. On my system /sys/bus/usb/devices/2-3.3 is a link to /sys/devices/pci:00/:00:1d.7/usb2/2-3/2-3.3/. This contains the pci bus address, but it also contains the bus number, which we've just said may be unstable across reboots. You mean the 2 in `usb2' ? I think that bus number is the bus number within the controller, not globally. On the contrary: $ ls -l usb* lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb1 - ../../../devices/pci:00/:00:1a.7/usb1/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb2 - ../../../devices/pci:00/:00:1d.7/usb2/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb3 - ../../../devices/pci:00/:00:1a.0/usb3/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb4 - ../../../devices/pci:00/:00:1a.1/usb4/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb5 - ../../../devices/pci:00/:00:1a.2/usb5/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb6 - ../../../devices/pci:00/:00:1d.0/usb6/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb7 - ../../../devices/pci:00/:00:1d.1/usb7/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb8 - ../../../devices/pci:00/:00:1d.2/usb8/ $ ls /sys//devices/pci:00/:00:1d.7/ | grep usb usb2/ In other words, the global bus enumeration leaks its way into the device path; which means at very least the sysfs device path is potentially unstable across reboots even if you include the pci controller it's on. I suppose it might be possible to specify buspci,port -- the pci address of the root bus, and the topology from there. In theory I guess that should be stable? Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. In any case, at the moment you're essentially inventing from whole cloth a new way of specifying USB devices that (as far as I know) isn't supported by any other program that uses USB. If you can't specify the device by hardware path, you can't specify it deterministically. And as you can see it _is_ supported by other programs that use USB. mount can use it! Mount doesn't know anything about USB devices; all it knows are block devices. You might as well say that 'ls' knows about USB devices because it can read files on a filesystem that resides on a USB stick. I'm talking about programs that explicitly manipulate USB devices -- in particular, those that may pass them through to a guest or seize them, like libvirt, qemu, virtualbox, c. I'm not opposed to doing something new if it really is better. But when you find that a dozen other projects before you have done things one way, you should at least take care before you decide to do something radically different, to make sure that it's because you actually have something better, and not because you've just missed something. I think the hardware path to the controller, at least, should be treated as an opaque OS-specific string. It might have a different format on BSD. If we can make an actual path that's stable across reboots, that would certainly be a good thing. But if it's not stable across reboots, it has no advantage over the bus-port[.port.port] interface. -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
On Tue, Jun 16, 2015 at 4:59 PM, Ian Jackson ian.jack...@eu.citrix.com wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]): Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. That does seem to be the case: What seems to be the case -- that it contains the global bus, or not? Looking at the output of udevadm monitor --property for sdc1 (on another plug): DEVPATH=/devices/pci:00/:00:1d.7/usb1/1-1/1-1:1.0/host22/target22:0:0/22:0:0:0/block/sdc/sdc1 ID_PATH=pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0 ID_PATH_TAG=pci-_00_1d_7-usb-0_1_1_0-scsi-0_0_0_0 I don't know where that ID_PATH comes from. It looks like that's constructed in udev: http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-path_id.c See handle_usb() (line 542) in particular. If I'm reading it right, what it basically does it take the [bus]-[port], and replace it with usb-0:[port]. (IOW, the '0' is hard-coded.) Also, if I'm reading it right, this won't work properly for Juergen's system that has two USB devices at the same pci address -- they'll both end up resolving to [pciaddr]-usb-0:[whatever]. Juergen -- can you give this a try? Run udevadm info on the two USB busses that share the same PCI slot, and see if the ID_PATH is the same for both? -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
Tuesday, June 16, 2015, 4:41:52 PM, you wrote: On 06/16/2015 02:37 PM, Ian Jackson wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API): Remember that the path you gave in your previous e-mail isn't the path for the *usb device*, it's the path for the *block device*. It contains a PCI address, but it looks like it also contains part of the USB topology. Are you sure that's actually a stable interface, or does it just happen that on your hardware the discovery always happens in the same order? The block device is (in path terms) underneath the usb device, obviously. Not all of that path is relevant to identifying the USB device. On my system /sys/bus/usb/devices/2-3.3 is a link to /sys/devices/pci:00/:00:1d.7/usb2/2-3/2-3.3/. This contains the pci bus address, but it also contains the bus number, which we've just said may be unstable across reboots. You mean the 2 in `usb2' ? I think that bus number is the bus number within the controller, not globally. On the contrary: $ ls -l usb* lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb1 - ../../../devices/pci:00/:00:1a.7/usb1/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb2 - ../../../devices/pci:00/:00:1d.7/usb2/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb3 - ../../../devices/pci:00/:00:1a.0/usb3/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb4 - ../../../devices/pci:00/:00:1a.1/usb4/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb5 - ../../../devices/pci:00/:00:1a.2/usb5/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb6 - ../../../devices/pci:00/:00:1d.0/usb6/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb7 - ../../../devices/pci:00/:00:1d.1/usb7/ lrwxrwxrwx 1 root root 0 Jun 15 10:14 usb8 - ../../../devices/pci:00/:00:1d.2/usb8/ $ ls /sys//devices/pci:00/:00:1d.7/ | grep usb usb2/ In other words, the global bus enumeration leaks its way into the device path; which means at very least the sysfs device path is potentially unstable across reboots even if you include the pci controller it's on. I suppose it might be possible to specify buspci,port -- the pci address of the root bus, and the topology from there. In theory I guess that should be stable? Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. In any case, at the moment you're essentially inventing from whole cloth a new way of specifying USB devices that (as far as I know) isn't supported by any other program that uses USB. If you can't specify the device by hardware path, you can't specify it deterministically. And as you can see it _is_ supported by other programs that use USB. mount can use it! Mount doesn't know anything about USB devices; all it knows are block devices. You might as well say that 'ls' knows about USB devices because it can read files on a filesystem that resides on a USB stick. I'm talking about programs that explicitly manipulate USB devices -- in particular, those that may pass them through to a guest or seize them, like libvirt, qemu, virtualbox, c. I'm not opposed to doing something new if it really is better. But when you find that a dozen other projects before you have done things one way, you should at least take care before you decide to do something radically different, to make sure that it's because you actually have something better, and not because you've just missed something. I think the hardware path to the controller, at least, should be treated as an opaque OS-specific string. It might have a different format on BSD. If we can make an actual path that's stable across reboots, that would certainly be a good thing. But if it's not stable across reboots, it has no advantage over the bus-port[.port.port] interface. -George Would a symlink to a sysfs or device node entry also be allowed ? That way you can leverage the power of tools like udev or other device managers and script the linkage based on a multitude of device properties (f.e. serial numbers, manufacturer id, device id, etc.). Which can make it persistent across reboots or
Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]
On 06/16/2015 06:34 PM, George Dunlap wrote: On Tue, Jun 16, 2015 at 4:59 PM, Ian Jackson ian.jack...@eu.citrix.com wrote: George Dunlap writes (Re: [Xen-devel] [PATCH V4 3/7] libxl: add pvusb API [and 1 more messages]): Yes. The whole point of paths like this is that they are stable if the physical topology doesn't change. So on my netbook /dev/disk/by-path/pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0-part1 always refers to the 1st MBR partition on logical device 0 on the USB storage device plugged into the USB port physically on the front left of the computer. That would be great if it were true, but I'm not convinced that the above path doesn't include a globally-enumerated bus number, which might change across reboots if it's enumerated in a different order. It may be that the format above is *not* based on the sysfs path of the device; that the '0' immediately after the USB means the first (and perhaps only) controller at this PCI address. In which case, yes, having a string like this might be a unique identifier for a particular port that would be stable across reboots. That needs some investigation. That does seem to be the case: What seems to be the case -- that it contains the global bus, or not? Looking at the output of udevadm monitor --property for sdc1 (on another plug): DEVPATH=/devices/pci:00/:00:1d.7/usb1/1-1/1-1:1.0/host22/target22:0:0/22:0:0:0/block/sdc/sdc1 ID_PATH=pci-:00:1d.7-usb-0:1:1.0-scsi-0:0:0:0 ID_PATH_TAG=pci-_00_1d_7-usb-0_1_1_0-scsi-0_0_0_0 I don't know where that ID_PATH comes from. It looks like that's constructed in udev: http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-path_id.c See handle_usb() (line 542) in particular. If I'm reading it right, what it basically does it take the [bus]-[port], and replace it with usb-0:[port]. (IOW, the '0' is hard-coded.) Also, if I'm reading it right, this won't work properly for Juergen's system that has two USB devices at the same pci address -- they'll both end up resolving to [pciaddr]-usb-0:[whatever]. Juergen -- can you give this a try? Run udevadm info on the two USB busses that share the same PCI slot, and see if the ID_PATH is the same for both? This was the correct question. :-) The ID_PATH is the same, but: It turns out that the two busses are for one physical hcd. One is the bus for USB 3.0, the other one for USB 2.0. I guess the bus is just selected by the capability of the plugged in device. So the [port] in usb-0:[port] is unique across the two logical busses. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
