Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-21 Thread Martin Wilck
On Mon, 2021-10-18 at 10:04 -0500, David Teigland wrote: > > I began trying to use RUN several > months ago and I think I gave up trying to find a way to pass values > from > the RUN program back into the udev rule (possibly by writing values > to a > temp file and then doing IMPORT{file}). 

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-21 Thread Martin Wilck
On Wed, 2021-10-20 at 09:50 -0500, David Teigland wrote: > > I was just providing some background history after you and Peter both > mentioned the idea of using RUN instead of IMPORT.  That is, I gave > up > trying to use RUN many months ago because it wouldn't work, while > IMPORT > actually

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-20 Thread David Teigland
On Wed, Oct 20, 2021 at 02:54:54PM +, Martin Wilck wrote: > I see. The problem with IMPORT (like RUN, actually) is that it's > required to finish quickly, which might not be the case if we encounter > resource shortage / contention as observed by Heming. If that happens, > the programs started

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-20 Thread David Teigland
On Wed, Oct 20, 2021 at 02:40:25PM +, Martin Wilck wrote: > > I began trying to use RUN several months ago and I think I gave up > > trying to find a way to pass values from the RUN program back into the > > udev rule (possibly by writing values to a temp file and then doing > > IMPORT{file}). 

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-19 Thread David Teigland
On Mon, Oct 18, 2021 at 11:51:27PM +0200, Zdenek Kabelac wrote: > The more generic solution with auto activation should likely try to 'active' > as much found complete VGs as it can at any given moment in time. > ATM lvm2 suffers when it's being running massively parallel - this has not > been

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-18 Thread heming.z...@suse.com
On 10/18/21 23:04, David Teigland wrote: On Mon, Oct 18, 2021 at 06:24:49AM +, Martin Wilck wrote: I'd like to second Peter here, "RUN" is in general less fragile than "IMPORT{PROGRAM}". You should use IMPORT{PROGRAM}" if and only if - the invoked program can work with incomplete udev

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-18 Thread Zdenek Kabelac
Dne 18. 10. 21 v 17:04 David Teigland napsal(a): On Mon, Oct 18, 2021 at 06:24:49AM +, Martin Wilck wrote: I'd like to second Peter here, "RUN" is in general less fragile than "IMPORT{PROGRAM}". You should use IMPORT{PROGRAM}" if and only if - the invoked program can work with incomplete

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-18 Thread David Teigland
On Mon, Oct 18, 2021 at 06:24:49AM +, Martin Wilck wrote: > I'd like to second Peter here, "RUN" is in general less fragile than > "IMPORT{PROGRAM}". You should use IMPORT{PROGRAM}" if and only if > > - the invoked program can work with incomplete udev state of a device >(the progrem

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-18 Thread Martin Wilck
On Thu, 2021-09-30 at 10:55 -0500, David Teigland wrote: > On Wed, Sep 29, 2021 at 11:39:52PM +0200, Peter Rajnoha wrote: > > > > >   - For event-based activation, we need to be sure that we use > > "RUN" > >     rule, not any of "IMPORT{program}" or "PROGRAM" rule. The > > difference > >     is

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-01 Thread Martin Wilck
On Thu, 2021-09-30 at 09:41 -0500, Benjamin Marzinski wrote: > On Thu, Sep 30, 2021 at 07:51:08AM +, Martin Wilck wrote: > > > For multipathd, we don't even need to care when all the block devices > have been processed.  We only need to care about devices that are > currently multipathed. If

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-01 Thread Martin Wilck
On Thu, 2021-09-30 at 23:32 +0800, heming.z...@suse.com wrote: > > I just want to say that some of the issues might simply be > > regressions/issues with systemd/udev that could be fixed. We as > > providers of block device abstractions where we need to handle, > > sometimes, thousands of devices,

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-01 Thread Peter Rajnoha
On Fri 01 Oct 2021 07:41, Martin Wilck wrote: > On Thu, 2021-09-30 at 23:32 +0800, heming.z...@suse.com wrote: > > > I just want to say that some of the issues might simply be > > > regressions/issues with systemd/udev that could be fixed. We as > > > providers of block device abstractions where

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-10-01 Thread Peter Rajnoha
On Thu 30 Sep 2021 10:55, David Teigland wrote: > On Wed, Sep 29, 2021 at 11:39:52PM +0200, Peter Rajnoha wrote: > > Hmm, thinking about this, I've just realized one more important and related > > thing here I didn't realize before - the LVM regex filters! These may > > contain > > symlink names

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Martin Wilck
On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: > On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: > > On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > > > I have pondered this quite a bit, but I can't say I have a > > > concrete > > > plan. > > > > > > To avoid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Martin Wilck
On Thu, 2021-09-30 at 16:07 +0800, heming.z...@suse.com wrote: > On 9/30/21 3:51 PM, Martin Wilck wrote: > > > Another performance story: > The legacy lvm2 (2.02.xx) with lvmetad daemon, the event-activation > mode > is very likely timeout on a large scale PVs. > When customer met this issue, we

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Martin Wilck
On Wed, 2021-09-29 at 23:39 +0200, Peter Rajnoha wrote: > On Mon 27 Sep 2021 10:38, David Teigland wrote: > > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > > - We could use the new lvm-activate-* services to replace the > > > > activation > > > > generator when lvm.conf

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Martin Wilck
On Wed, 2021-09-29 at 23:53 +0200, Peter Rajnoha wrote: > On Tue 28 Sep 2021 06:34, Martin Wilck wrote: > > > > You said it should wait for multipathd, which in turn waits for > > udev > > settle. And indeed it makes some sense. After all: the idea was to > > avoid locking issues or general

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Benjamin Marzinski
On Thu, Sep 30, 2021 at 07:51:08AM +, Martin Wilck wrote: > On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: > > On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: > > > On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > > > > I have pondered this quite a bit, but I can't

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread heming.z...@suse.com
On 9/30/21 7:41 PM, Peter Rajnoha wrote: On 9/30/21 10:07, heming.z...@suse.com wrote: On 9/30/21 3:51 PM, Martin Wilck wrote: On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread David Teigland
On Thu, Sep 30, 2021 at 01:29:07PM +0200, Peter Rajnoha wrote: > If we see this udev settle as the key point, then I think we should probably > concentrate on enhancing systemd/udev to provide this functionality (and > primarily the udevadm trigger functionality and waiting for related >

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread David Teigland
On Wed, Sep 29, 2021 at 11:39:52PM +0200, Peter Rajnoha wrote: > Hmm, thinking about this, I've just realized one more important and related > thing here I didn't realize before - the LVM regex filters! These may contain > symlink names as one can find them in /dev. But for those symlinks, we need

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread David Teigland
On Thu, Sep 30, 2021 at 07:22:29AM +, Martin Wilck wrote: > On Wed, 2021-09-29 at 23:39 +0200, Peter Rajnoha wrote: > > For event-based activation, I'd expect it to really behave in event- > > based manner, that is, to respond to events as soon as they come and not > > wait for all the other

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread heming.z...@suse.com
On 9/30/21 3:51 PM, Martin Wilck wrote: On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: I have pondered this quite a bit, but I can't say I have a concrete plan. To avoid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Peter Rajnoha
On 9/30/21 10:07, heming.z...@suse.com wrote: On 9/30/21 3:51 PM, Martin Wilck wrote: On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: I have pondered this quite a bit,

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-30 Thread Peter Rajnoha
On 9/30/21 09:51, Martin Wilck wrote: On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote: On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: I have pondered this quite a bit, but I can't say I have a concrete plan. To avoid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Peter Rajnoha
On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote: > On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > > I have pondered this quite a bit, but I can't say I have a concrete > > plan. > > > > To avoid depending on "udev settle", multipathd needs to partially > > revert to

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Peter Rajnoha
On Tue 28 Sep 2021 06:34, Martin Wilck wrote: > Hello David and Peter, > > On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote: > > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > > - We could use the new lvm-activate-* services to replace the > > > > activation > > > >

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Peter Rajnoha
On Mon 27 Sep 2021 10:38, David Teigland wrote: > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > - We could use the new lvm-activate-* services to replace the activation > > > generator when lvm.conf event_activation=0. This would be done by simply > > > not creating the

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Martin Wilck
On Tue, 2021-09-28 at 12:42 -0500, Benjamin Marzinski wrote: > On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > > On Tue, 2021-09-28 at 09:42 -0500, David Teigland wrote: > > > > > > I have pondered this quite a bit, but I can't say I have a concrete > > plan. > > > > To avoid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Martin Wilck
On Tue, 2021-09-28 at 17:16 +0200, Martin Wilck wrote: > On Tue, 2021-09-28 at 09:42 -0500, David Teigland wrote: > > > > > Firstly, with the new devices file, only the actual md/mpath device > > will > > be in the devices file, the components will not be, so lvm will > > never > > attempt to

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Martin Wilck
Hello David and Peter, On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote: > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > - We could use the new lvm-activate-* services to replace the > > > activation > > > generator when lvm.conf event_activation=0.  This would be

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Martin Wilck
On Tue, 2021-09-28 at 09:42 -0500, David Teigland wrote: > On Tue, Sep 28, 2021 at 06:34:06AM +, Martin Wilck wrote: > > Hello David and Peter, > > > > On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote: > > > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > > > - We

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Benjamin Marzinski
On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > On Tue, 2021-09-28 at 09:42 -0500, David Teigland wrote: > > On Tue, Sep 28, 2021 at 06:34:06AM +, Martin Wilck wrote: > > > Hello David and Peter, > > > > > > On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote: > > > > On

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-29 Thread Benjamin Marzinski
On Tue, Sep 28, 2021 at 10:56:09AM -0500, David Teigland wrote: > On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > > Hm. This would mean that the switch to event-based PV detection could > > happen before "udev settle" ends. A coldplug storm of uevents could > > create 1000s of PVs

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-28 Thread David Teigland
On Tue, Sep 28, 2021 at 03:16:08PM +, Martin Wilck wrote: > Hm. This would mean that the switch to event-based PV detection could > happen before "udev settle" ends. A coldplug storm of uevents could > create 1000s of PVs in a blink after event-based detection was enabled. > Wouldn't that

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-28 Thread David Teigland
On Tue, Sep 28, 2021 at 06:34:06AM +, Martin Wilck wrote: > Hello David and Peter, > > On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote: > > On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > > > - We could use the new lvm-activate-* services to replace the > > > >

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-27 Thread David Teigland
On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote: > > - We could use the new lvm-activate-* services to replace the activation > > generator when lvm.conf event_activation=0. This would be done by simply > > not creating the event-activation-on file when event_activation=0. > >

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-27 Thread Peter Rajnoha
Hi! the patches and logic looks promising, there's just one thing I'm worried about... On Thu 09 Sep 2021 14:44, David Teigland wrote: > I've implemented a solution like this and would like any thoughts, > improvements, or testing to verify it can help: >

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-09-13 Thread Martin Wilck
On Thu, 2021-09-09 at 14:44 -0500, David Teigland wrote: > On Tue, Jun 08, 2021 at 01:23:33PM +, Martin Wilck wrote: > > On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: > > > On Mon 07 Jun 2021 16:48, David Teigland wrote: > > > > > > > > If there are say 1000 PVs already present on

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-08 Thread Tom Yan
I wonder if using a shell wrapper to detect whether pvscan actually exits way earlier than the unit does is a more accurate approach. On Sat, 3 Jul 2021 at 19:50, heming.z...@suse.com wrote: > > On 7/3/21 6:02 AM, David Teigland wrote: > > On Fri, Jul 02, 2021 at 09:22:03PM +, Martin Wilck

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-07 Thread Martin Wilck
On Fr, 2021-07-02 at 16:09 -0500, David Teigland wrote: > On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: > > dev_cache_scan //order: O(n^2) > >  + _insert_dirs //O(n) > >  | if obtain_device_list_from_udev() true > >  |   _insert_udev_dir //O(n) > >  | > >  +

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-03 Thread heming.z...@suse.com
On 7/3/21 6:02 AM, David Teigland wrote: On Fri, Jul 02, 2021 at 09:22:03PM +, Martin Wilck wrote: On Fr, 2021-07-02 at 16:09 -0500, David Teigland wrote: On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: dev_cache_scan //order: O(n^2)  + _insert_dirs //O(n)  | if

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-02 Thread David Teigland
On Fri, Jul 02, 2021 at 09:22:03PM +, Martin Wilck wrote: > On Fr, 2021-07-02 at 16:09 -0500, David Teigland wrote: > > On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: > > > dev_cache_scan //order: O(n^2) > > >  + _insert_dirs //O(n) > > >  | if

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-02 Thread Tom Yan
On Sat, 3 Jul 2021 at 05:17, David Teigland wrote: > > On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: > > dev_cache_scan //order: O(n^2) > > + _insert_dirs //O(n) > > | if obtain_device_list_from_udev() true > > | _insert_udev_dir //O(n) > > | > > +

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-07-02 Thread David Teigland
On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: > dev_cache_scan //order: O(n^2) > + _insert_dirs //O(n) > | if obtain_device_list_from_udev() true > | _insert_udev_dir //O(n) > | > + dev_cache_index_devs //O(n) I've been running some experiments and trying some

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-17 Thread David Teigland
On Thu, Jun 17, 2021 at 11:46:55AM +0800, heming.z...@suse.com wrote: > the booting time out problem cause by two side: > 1> libudev take much time > 2> thousands of lvm2-pvscan@.service running on the same time > > the new patch or existing code (with obtain_device_list_from_udev=0) can > avoid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-17 Thread heming.z...@suse.com
On 6/17/21 12:38 AM, David Teigland wrote: On Thu, Jun 17, 2021 at 12:18:47AM +0800, heming.z...@suse.com wrote: I don't know if I missed something, the result didn't show any progress. the result of "devices/obtain_device_list_from_udev=0" even got regression: from 23.3 => 39.8 the lvm2

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-16 Thread heming.z...@suse.com
On 6/16/21 1:03 AM, David Teigland wrote: On Tue, Jun 08, 2021 at 10:39:37AM -0500, David Teigland wrote: I think it would be an improvement to: . Make obtain_device_list_from_udev only control how we get the device list. Then we can easily default to 0 and readdir /dev if it's better. .

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-16 Thread David Teigland
On Thu, Jun 17, 2021 at 12:18:47AM +0800, heming.z...@suse.com wrote: > I don't know if I missed something, the result didn't show any progress. > the result of "devices/obtain_device_list_from_udev=0" even got regression: > from 23.3 => 39.8 > > the lvm2 version with dev-dct-device-info-1

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-15 Thread Zdenek Kabelac
Dne 15. 06. 21 v 19:03 David Teigland napsal(a): On Tue, Jun 08, 2021 at 10:39:37AM -0500, David Teigland wrote: I think it would be an improvement to: . Make obtain_device_list_from_udev only control how we get the device list. Then we can easily default to 0 and readdir /dev if it's

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-15 Thread David Teigland
On Tue, Jun 08, 2021 at 10:39:37AM -0500, David Teigland wrote: > I think it would be an improvement to: > > . Make obtain_device_list_from_udev only control how we get the device > list. Then we can easily default to 0 and readdir /dev if it's better. > > . Use both native md/mpath detection

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-09 Thread Heming Zhao
Either my mailbox or lvm mail list is broken, I can't see my last two mails appear in the mail list. This mail I will mention another issue about lvm2-pvscan@.service. both event activation and direct activation have same issue: the shutdown take much time. the code logic in pvscan_cache_cmd()

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-09 Thread heming.z...@suse.com
On 6/9/21 12:18 AM, heming.z...@suse.com wrote: On 6/8/21 5:30 AM, David Teigland wrote: On Mon, Jun 07, 2021 at 10:27:20AM +, Martin Wilck wrote: Most importantly, this was about LVM2 scanning of physical volumes. The number of udev workers has very little influence on PV scanning,

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread heming.z...@suse.com
On 6/8/21 4:26 PM, Martin Wilck wrote: On Mo, 2021-06-07 at 16:30 -0500, David Teigland wrote: On Mon, Jun 07, 2021 at 10:27:20AM +, Martin Wilck wrote: Most importantly, this was about LVM2 scanning of physical volumes. The number of udev workers has very little influence on PV scanning,

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Di, 2021-06-08 at 15:56 +0200, Peter Rajnoha wrote: > > The issue is that we're relying now on udev db records that contain > info about mpath and MD components - without this, the detection (and > hence filtering) could fail in certain cases. So if go without > checking > udev db, that'll be

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Di, 2021-06-08 at 18:02 +0200, Zdenek Kabelac wrote: > > > > > A third related improvement that could follow is to add stronger > > > native > > > mpath detection, in which lvm uses uses /etc/multipath/wwids, > > > directly or > > > through a multipath library, to identify mpath components. 

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Di, 2021-06-08 at 10:39 -0500, David Teigland wrote: > > . Use both native md/mpath detection *and* udev info when it's > readily >   available (don't wait for it), instead of limiting ourselves to one >   source of info.  If either source indicates an md/mpath component, >   then we consider

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread heming.z...@suse.com
On 6/8/21 5:30 AM, David Teigland wrote: On Mon, Jun 07, 2021 at 10:27:20AM +, Martin Wilck wrote: Most importantly, this was about LVM2 scanning of physical volumes. The number of udev workers has very little influence on PV scanning, because the udev rules only activate systemd service.

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: > On Mon 07 Jun 2021 16:48, David Teigland wrote: > > > > If there are say 1000 PVs already present on the system, there > > could be > > real savings in having one lvm command process all 1000, and then > > switch > > over to processing

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Mo, 2021-06-07 at 16:30 -0500, David Teigland wrote: > On Mon, Jun 07, 2021 at 10:27:20AM +, Martin Wilck wrote: > > Most importantly, this was about LVM2 scanning of physical volumes. > > The > > number of udev workers has very little influence on PV scanning, > > because the udev rules

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread David Teigland
On Tue, Jun 08, 2021 at 03:47:39PM +, Martin Wilck wrote: > Hm. You can boot with "multipath=off" which udev would take into > account. What would you do in that case? Native mpath detection would > probably not figure it out. libmpathvalid will still know? > Please don't. Use libmpathvalid

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Zdenek Kabelac
Dne 08. 06. 21 v 17:47 Martin Wilck napsal(a): On Di, 2021-06-08 at 10:39 -0500, David Teigland wrote: . Use both native md/mpath detection *and* udev info when it's readily   available (don't wait for it), instead of limiting ourselves to one   source of info.  If either source indicates an

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread David Teigland
On Tue, Jun 08, 2021 at 08:26:01AM +, Martin Wilck wrote: > IIUC, 2) is the effect of _pvscan_aa_quick(). 3) is surprising; > apparently libudev's device detection causes a factor 3 slowdown. > While 40s is not bad, you can see that event based activation still > performs far worse than

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Peter Rajnoha
On Tue 08 Jun 2021 14:48, Martin Wilck wrote: > On Di, 2021-06-08 at 15:56 +0200, Peter Rajnoha wrote: > > > > The issue is that we're relying now on udev db records that contain > > info about mpath and MD components - without this, the detection (and > > hence filtering) could fail in certain

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Zdenek Kabelac
Dne 08. 06. 21 v 15:56 Peter Rajnoha napsal(a): On Tue 08 Jun 2021 15:46, Zdenek Kabelac wrote: Dne 08. 06. 21 v 15:41 Peter Rajnoha napsal(a): On Tue 08 Jun 2021 13:23, Martin Wilck wrote: On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: On Mon 07 Jun 2021 16:48, David Teigland wrote:

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Peter Rajnoha
On Tue 08 Jun 2021 15:46, Zdenek Kabelac wrote: > Dne 08. 06. 21 v 15:41 Peter Rajnoha napsal(a): > > On Tue 08 Jun 2021 13:23, Martin Wilck wrote: > > > On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: > > > > On Mon 07 Jun 2021 16:48, David Teigland wrote: > > > > > If there are say 1000

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Zdenek Kabelac
Dne 08. 06. 21 v 15:41 Peter Rajnoha napsal(a): On Tue 08 Jun 2021 13:23, Martin Wilck wrote: On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: On Mon 07 Jun 2021 16:48, David Teigland wrote: If there are say 1000 PVs already present on the system, there could be real savings in having

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Peter Rajnoha
On Tue 08 Jun 2021 13:23, Martin Wilck wrote: > On Di, 2021-06-08 at 14:29 +0200, Peter Rajnoha wrote: > > On Mon 07 Jun 2021 16:48, David Teigland wrote: > > > > > > If there are say 1000 PVs already present on the system, there > > > could be > > > real savings in having one lvm command process

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Peter Rajnoha
On Mon 07 Jun 2021 16:48, David Teigland wrote: > On Mon, Jun 07, 2021 at 03:48:30PM +, Martin Wilck wrote: > > On So, 2021-06-06 at 14:15 +0800, heming.z...@suse.com wrote: > > > > > > 1. During boot phase, lvm2 automatically swithes to direct activation > > > mode > > > ("event_activation =

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On Mo, 2021-06-07 at 23:30 +0800, heming.z...@suse.com wrote: > On 6/7/21 6:27 PM, Martin Wilck wrote: > > On So, 2021-06-06 at 11:35 -0500, Roger Heflin wrote: > > > This might be a simpler way to control the number of threads at > > > the > > > same time. > > > > > > On large machines (cpu

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On So, 2021-06-06 at 11:35 -0500, Roger Heflin wrote: > This might be a simpler way to control the number of threads at the > same time. > > On large machines (cpu wise, memory wise and disk wise).   I have > only seen lvm timeout when udev_children is set to default.   The > default seems to be

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Martin Wilck
On So, 2021-06-06 at 14:15 +0800, heming.z...@suse.com wrote: > > 1. During boot phase, lvm2 automatically swithes to direct activation > mode > ("event_activation = 0"). After booted, switch back to the event > activation mode. > > Booting phase is a speical stage. *During boot*, we could

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-08 Thread Roger Heflin
The case we had is large physical machines with around 1000 disks. We did not see the issue on the smaller cpu/disked physicals and/or vm's. It seemed like both high cpu counts and high disk counts were needed, but in our environment both of those are usually together.The smallest machines

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-07 Thread David Teigland
On Mon, Jun 07, 2021 at 03:48:30PM +, Martin Wilck wrote: > On So, 2021-06-06 at 14:15 +0800, heming.z...@suse.com wrote: > > > > 1. During boot phase, lvm2 automatically swithes to direct activation > > mode > > ("event_activation = 0"). After booted, switch back to the event > > activation

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-07 Thread David Teigland
On Mon, Jun 07, 2021 at 10:27:20AM +, Martin Wilck wrote: > Most importantly, this was about LVM2 scanning of physical volumes. The > number of udev workers has very little influence on PV scanning, > because the udev rules only activate systemd service. The actual > scanning takes place in

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-07 Thread David Teigland
Hi Heming, Thanks for the analysis and tying things together for us so clearly, and I like the ideas you've outlined. On Sun, Jun 06, 2021 at 02:15:23PM +0800, heming.z...@suse.com wrote: > I send this mail for a well known performance issue: > when system is attached huge numbers of devices.

Re: [linux-lvm] Discussion: performance issue on event activation mode

2021-06-07 Thread Zdenek Kabelac
Dne 07. 06. 21 v 17:48 Martin Wilck napsal(a): On So, 2021-06-06 at 14:15 +0800, heming.z...@suse.com wrote: 1. During boot phase, lvm2 automatically swithes to direct activation mode ("event_activation = 0"). After booted, switch back to the event activation mode. Booting phase is a speical

[linux-lvm] Discussion: performance issue on event activation mode

2021-06-06 Thread heming.z...@suse.com
Hello David & Zdenek, I send this mail for a well known performance issue: when system is attached huge numbers of devices. (ie. 1000+ disks), the lvm2-pvscan@.service costs too much time and systemd is very easy to time out, and enter emergency shell in the end. This performance topic had