[libvirt] [PATCH v2] nodedev: Increase the netlink socket buffer size to the one used by udev
From: "ning.bo" When a number of SRIOV VFs (up to 128 on Intel XL710) is created: for i in `seq 0 1`; do echo 63 > /sys/class/net//device/sriov_numvfs done libvirtd will then report "udev_monitor_receive_device returned NULL" error because the netlink socket buffer is not big enough (using GDB on libudev confirmed this with ENOBUFFS) and thus some udev events were dropped. This results in some devices being missing in the nodedev-list output. This patch overrides the system's rmem_max limit but for that, we need to make sure we've got root privileges. https://bugzilla.redhat.com/show_bug.cgi?id=1450960 Signed-off-by: ning.bo Signed-off-by: Erik Skultety --- Additionally, we might want to check for the errno, providing a specific message if such issue occurs again in a further non-specified point in time and return the generic, yet cryptic one for all other cases. src/node_device/node_device_udev.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/src/node_device/node_device_udev.c b/src/node_device/node_device_udev.c index 4ecb0b18f..0b3717397 100644 --- a/src/node_device/node_device_udev.c +++ b/src/node_device/node_device_udev.c @@ -1728,6 +1728,13 @@ static int nodeStateInitialize(bool privileged, udev_monitor_enable_receiving(priv->udev_monitor); +/* mimic udevd's behaviour and override the systems rmem_max limit in case + * there's a significant number of device 'add' events + */ +if (geteuid() == 0) +udev_monitor_set_receive_buffer_size(priv->udev_monitor, + 128 * 1024 * 1024); + /* We register the monitor with the event callback so we are * notified by udev of device changes before we enumerate existing * devices because libvirt will simply recreate the device if we -- 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2] nodedev: Increase the netlink socket buffer size to the one used by udev
On Fri, May 19, 2017 at 03:16:26PM +0200, Erik Skultety wrote: > From: "ning.bo" > Ping? > When a number of SRIOV VFs (up to 128 on Intel XL710) is created: > for i in `seq 0 1`; do > echo 63 > /sys/class/net//device/sriov_numvfs > done > > libvirtd will then report "udev_monitor_receive_device returned NULL" > error because the netlink socket buffer is not big enough (using GDB on > libudev confirmed this with ENOBUFFS) and thus some udev events were > dropped. This results in some devices being missing in the nodedev-list > output. This patch overrides the system's rmem_max limit but for that, > we need to make sure we've got root privileges. > > https://bugzilla.redhat.com/show_bug.cgi?id=1450960 > > Signed-off-by: ning.bo > Signed-off-by: Erik Skultety > --- > Additionally, we might want to check for the errno, providing a specific > message if such issue occurs again in a further non-specified point in time > and > return the generic, yet cryptic one for all other cases. > > src/node_device/node_device_udev.c | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git a/src/node_device/node_device_udev.c > b/src/node_device/node_device_udev.c > index 4ecb0b18f..0b3717397 100644 > --- a/src/node_device/node_device_udev.c > +++ b/src/node_device/node_device_udev.c > @@ -1728,6 +1728,13 @@ static int nodeStateInitialize(bool privileged, > > udev_monitor_enable_receiving(priv->udev_monitor); > > +/* mimic udevd's behaviour and override the systems rmem_max limit in > case > + * there's a significant number of device 'add' events > + */ > +if (geteuid() == 0) > +udev_monitor_set_receive_buffer_size(priv->udev_monitor, > + 128 * 1024 * 1024); > + > /* We register the monitor with the event callback so we are > * notified by udev of device changes before we enumerate existing > * devices because libvirt will simply recreate the device if we > -- > 2.13.0 > > -- > libvir-list mailing list > libvir-list@redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2] nodedev: Increase the netlink socket buffer size to the one used by udev
On 05/19/2017 03:16 PM, Erik Skultety wrote: > From: "ning.bo" > > When a number of SRIOV VFs (up to 128 on Intel XL710) is created: > for i in `seq 0 1`; do > echo 63 > /sys/class/net//device/sriov_numvfs > done > > libvirtd will then report "udev_monitor_receive_device returned NULL" > error because the netlink socket buffer is not big enough (using GDB on > libudev confirmed this with ENOBUFFS) and thus some udev events were > dropped. This results in some devices being missing in the nodedev-list > output. This patch overrides the system's rmem_max limit but for that, > we need to make sure we've got root privileges. > > https://bugzilla.redhat.com/show_bug.cgi?id=1450960 > > Signed-off-by: ning.bo > Signed-off-by: Erik Skultety > --- > Additionally, we might want to check for the errno, providing a specific > message if such issue occurs again in a further non-specified point in time > and > return the generic, yet cryptic one for all other cases. > > src/node_device/node_device_udev.c | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git a/src/node_device/node_device_udev.c > b/src/node_device/node_device_udev.c > index 4ecb0b18f..0b3717397 100644 > --- a/src/node_device/node_device_udev.c > +++ b/src/node_device/node_device_udev.c > @@ -1728,6 +1728,13 @@ static int nodeStateInitialize(bool privileged, > > udev_monitor_enable_receiving(priv->udev_monitor); > > +/* mimic udevd's behaviour and override the systems rmem_max limit in > case > + * there's a significant number of device 'add' events > + */ > +if (geteuid() == 0) > +udev_monitor_set_receive_buffer_size(priv->udev_monitor, > + 128 * 1024 * 1024); > + > /* We register the monitor with the event callback so we are > * notified by udev of device changes before we enumerate existing > * devices because libvirt will simply recreate the device if we ACK and safe for freeze. Although on my system it works even without the check for euid. Michal -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2] nodedev: Increase the netlink socket buffer size to the one used by udev
On Mon, May 29, 2017 at 02:16:17PM +0200, Michal Privoznik wrote: > On 05/19/2017 03:16 PM, Erik Skultety wrote: > > From: "ning.bo" > > > > When a number of SRIOV VFs (up to 128 on Intel XL710) is created: > > for i in `seq 0 1`; do > > echo 63 > /sys/class/net//device/sriov_numvfs > > done > > > > libvirtd will then report "udev_monitor_receive_device returned NULL" > > error because the netlink socket buffer is not big enough (using GDB on > > libudev confirmed this with ENOBUFFS) and thus some udev events were > > dropped. This results in some devices being missing in the nodedev-list > > output. This patch overrides the system's rmem_max limit but for that, > > we need to make sure we've got root privileges. > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1450960 > > > > Signed-off-by: ning.bo > > Signed-off-by: Erik Skultety > > --- > > Additionally, we might want to check for the errno, providing a specific > > message if such issue occurs again in a further non-specified point in time > > and > > return the generic, yet cryptic one for all other cases. > > > > src/node_device/node_device_udev.c | 7 +++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/src/node_device/node_device_udev.c > > b/src/node_device/node_device_udev.c > > index 4ecb0b18f..0b3717397 100644 > > --- a/src/node_device/node_device_udev.c > > +++ b/src/node_device/node_device_udev.c > > @@ -1728,6 +1728,13 @@ static int nodeStateInitialize(bool privileged, > > > > udev_monitor_enable_receiving(priv->udev_monitor); > > > > +/* mimic udevd's behaviour and override the systems rmem_max limit in > > case > > + * there's a significant number of device 'add' events > > + */ > > +if (geteuid() == 0) > > +udev_monitor_set_receive_buffer_size(priv->udev_monitor, > > + 128 * 1024 * 1024); > > + > > /* We register the monitor with the event callback so we are > > * notified by udev of device changes before we enumerate existing > > * devices because libvirt will simply recreate the device if we > > ACK and safe for freeze. Although on my system it works even without the > check for euid. Are you sure? Because the session daemon should not have the CAP_NET_ADMIN capability. Though it's true that since we don't care about the return value the check is unnecessary I give you that. Still, the check is harmless and we know that it will fail for non-privileged user, so I'm going to keep it here. Thanks for review, Erik > > Michal > > -- > libvir-list mailing list > libvir-list@redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2] nodedev: Increase the netlink socket buffer size to the one used by udev
On Fri, May 19, 2017 at 03:16:26PM +0200, Erik Skultety wrote: > From: "ning.bo" > > When a number of SRIOV VFs (up to 128 on Intel XL710) is created: > for i in `seq 0 1`; do > echo 63 > /sys/class/net//device/sriov_numvfs > done > > libvirtd will then report "udev_monitor_receive_device returned NULL" > error because the netlink socket buffer is not big enough (using GDB on > libudev confirmed this with ENOBUFFS) and thus some udev events were > dropped. This results in some devices being missing in the nodedev-list > output. This patch overrides the system's rmem_max limit but for that, > we need to make sure we've got root privileges. > > https://bugzilla.redhat.com/show_bug.cgi?id=1450960 > > Signed-off-by: ning.bo > Signed-off-by: Erik Skultety > --- > Additionally, we might want to check for the errno, providing a specific > message if such issue occurs again in a further non-specified point in time > and > return the generic, yet cryptic one for all other cases. > > src/node_device/node_device_udev.c | 7 +++ > 1 file changed, 7 insertions(+) This has broken the build on older systems as udev doesn't have this method. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list