Re: [linux-lvm] lvmlockd: about the limitation on lvresizing the LV active on multiple nodes
Hello David, Happy new year! On 01/03/2018 01:10 AM, David Teigland wrote: * resizing an LV that is active in the shared mode on multiple hosts It seems a big limitation to use lvmlockd in cluster: Only in the case where the LV is active on multiple hosts at once, i.e. a cluster fs, which is less common than a local fs. In the general case, it's not safe to assume that an LV can be modified by one node while it's being used by others, even when all of them hold shared locks on the LV. You'd want to prevent that in general. Exceptions exist, but whether an exception is ok will likely depend on what the specific change is, what application is using the LV, whether that application can tolerate such a change. One (perhaps the only?) valid exception I know about is extending an LV while it's being used under a cluster fs (any cluster fs?) The only concrete scenario I can think of is also cluster fs, like OCFS2, tunefs.ocfs2 can enlarge the FS to use all the device space online. (In reference to your later email, this is not related to lock queueing, but rather to basic ex/sh lock incompatibility, and when/how to allow exceptions to that.) I thought the procedures to allow lvresize is like below if the LV is used by cluster FS: Assume the LV is active with "sh" lock on multiple nodes (node1 and node2), and we lvextend on node1: - node1: the "sh" lock on r1 (the LV resource) needs to up convert: "sh" -> "ex"; - node2: on behalf of the BAST, the "sh" lock on r1needs to down convert: "sh" -> "nl", which means the LV should be suspended; - node1: on receiving AST (get "ex" lock), lvresize is allowed; After the completion of lvresize, the original lock state should be restored on every node, meanwhile the latest metadata can be refreshed, maybe like below: - node1: restore the original lock mode, "ex" -> "sh", the metadata version will be increased, so that request to update metadata can be sent to other nodes; - node2: on receiving request, "nl" -> "sh", then to refresh the metadata from disk; The simplest approach I can think of to allow lvextend under a cluster fs would be a procedure like: If there is a simple approach, I think it maybe worth a try. 1. one one node: lvextend --lockopt skip -L+1G VG/LV That option doesn't exist, but illustrates the point that some new option could be used to skip the incompatible LV locking in lvmlockd. Hmm, is it safe to just skip the locking while the LV is active on other node? Is there somewhere in the code to avoid concurrent lvm command to execute at the same time? 2. on each node: lvchange --refresh VG/LV This updates dm on each node with the new device size. 3. gfs2_grow VG/LV or equivalent At this point the fs on any node can begin accessing the new space. It would be great. Regards, Eric ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
On 01/02/2018 10:43 AM, Alasdair G Kergon wrote: The key concept to grasp is that LVM works with devices (i.e. major number + minor number pairs) rather than filesystem paths and it makes a single "use or ignore" decision for each device (not for each path). That seems even more misleading than the original documentation, since the filters are applied to each path, and not to the device. I understand what you mean, now, but I don't think that explaining the system that way is likely to help users understand. I'll send a patch later which might be more clear. Should I send it to this list? Martian (or Marian? Signature didn't match the From header on an earlier message) said that they'd also send a patch, but I'm not sure where it will be sent. BTW, simply removing the "a" filter solved my problem. My vm_*_data LVs are no longer processed, but the md device that backs the primary VG is, so everything works as I expect it to. Thanks, again. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
Also there is no special handling for |.*| - a match against it gets treated as a positive deliberate match, just the same way as a match against |/dev/sda2|. Alasdair ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
The key concept to grasp is that LVM works with devices (i.e. major number + minor number pairs) rather than filesystem paths and it makes a single "use or ignore" decision for each device (not for each path). One device might be reachable through more than one filesystem path that lvm has been configured to check and, if so, all the paths are checked against the filter independently and the results feed into that single decision according to the rules described. Alasdair ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
On 01/02/2018 04:35 PM, Gordon Messmer wrote:
On 01/02/2018 03:03 AM, Marian Csontos wrote:
Filters accept any device if any of it's "names" (all symbolic links)
is matched by an a pattern ("a|.*/|" in your case) and matches no
previous r pattern
I don't follow you. The name being processed in that log *does* match a
previous r pattern.
I do not think that log shows what is processed. LVM displays device
names according to different option - preferred_names it is.
If lvmetad is in use, which by default it is, run `pvscan - --cache
$DEV_MAJOR:$DEV_MINOR` - that should be more helpful WRT which devices
are scanned and accepted as Physical Volumes.
for example anything in /dev/mapper/ is accepted.
^ THIS! In your case the /dev/mapper/vm_* link IS accepted by the
"a|.*/|" in global_filter. And /dev/disk/by-*/* too. And maybe others...
Yes, I'd considered that might be an issue, but that's not the block
device name that the logs indicate is being used. A path that I've
specifically rejected is being processed. If a second path to the block
device might be processed, then I can see the need to make additional
changes, but I can't solve that problem without understanding the basic
logic of the filter system.
The documentation in lvm.conf says "The first regex in the list to match
the path is used, producing the 'a' or 'r' result for the device." but
that doesn't seem to be correct.
IMHO the documentation is confusing if not straight incorrect. I hate
that and IMHO we should fix how the filtering works, but that would be
an incompatible change, and those are not loved. :-(
How it actually works is: if ANY path is accepted, the device is. In
upstream the lvm.conf also says (emphasis mine):
"When multiple path names exist for a block device, if *any path* name
matches an 'a' pattern before an 'r' pattern, then the device is
accepted. If all the path names match an 'r' pattern first, then the
device is rejected."
May be we should replace the "device" in the last sentence by "path" so
it would say:
"The first regex in the list to match the path is used, producing the
'a' or 'r' result for *that path*."
And:
"When multiple path names exist for a block device"
could be safely discarded - as it is true most of if not all the time
(or replaced by "Keep in mind devices nowadays are linked from many
places in the /dev tree, and")
So we would have the following:
"If any path name matches an 'a' pattern before an 'r' pattern, then the
device is accepted."
I am not a native speaker, but to me that is consistent with current
behavior which is unlikely to change. I will send a patch to see if it
is opposed by anyone...
-- Martian
___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] lvmlockd: about the limitation on lvresizing the LV active on multiple nodes
> * resizing an LV that is active in the shared mode on multiple hosts > > It seems a big limitation to use lvmlockd in cluster: Only in the case where the LV is active on multiple hosts at once, i.e. a cluster fs, which is less common than a local fs. In the general case, it's not safe to assume that an LV can be modified by one node while it's being used by others, even when all of them hold shared locks on the LV. You'd want to prevent that in general. Exceptions exist, but whether an exception is ok will likely depend on what the specific change is, what application is using the LV, whether that application can tolerate such a change. One (perhaps the only?) valid exception I know about is extending an LV while it's being used under a cluster fs (any cluster fs?) (In reference to your later email, this is not related to lock queueing, but rather to basic ex/sh lock incompatibility, and when/how to allow exceptions to that.) The simplest approach I can think of to allow lvextend under a cluster fs would be a procedure like: 1. one one node: lvextend --lockopt skip -L+1G VG/LV That option doesn't exist, but illustrates the point that some new option could be used to skip the incompatible LV locking in lvmlockd. 2. on each node: lvchange --refresh VG/LV This updates dm on each node with the new device size. 3. gfs2_grow VG/LV or equivalent At this point the fs on any node can begin accessing the new space. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] lvmlockd manpage: prevent concurrent activation of logical volumes?
On Thu, Dec 28, 2017 at 04:30:09PM +0800, Eric Ren wrote: > Hi David, > > I's afraid the statement below in description section of lvmlockd manpage: > > " > · prevent concurrent activation of logical volumes > " > > is easy for normal user to mistake it as: wow, lvmlockd doesn't support > active-active LV on multiple nodes? Hi, I see what you mean. What that point was trying to summarize is "prevent conflicting activation of LVs". I may just make that one word change. Thanks ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
On 01/02/2018 03:03 AM, Marian Csontos wrote:
Filters accept any device if any of it's "names" (all symbolic links)
is matched by an a pattern ("a|.*/|" in your case) and matches no
previous r pattern
I don't follow you. The name being processed in that log *does* match a
previous r pattern.
for example anything in /dev/mapper/ is accepted.
Yes, I'd considered that might be an issue, but that's not the block
device name that the logs indicate is being used. A path that I've
specifically rejected is being processed. If a second path to the block
device might be processed, then I can see the need to make additional
changes, but I can't solve that problem without understanding the basic
logic of the filter system.
The documentation in lvm.conf says "The first regex in the list to match
the path is used, producing the 'a' or 'r' result for the device." but
that doesn't seem to be correct.
___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] unable to exclude LVs using global_filter
On 01/02/2018 07:47 AM, Gordon Messmer wrote:
I'd like to avoid scanning LVs, and I thought I'd set global_filter
appropriately. The following logs show global_filter set to reject
^/dev/VolGroup/vm_, but the system is scanning
/dev/VolGroup/vm_wiki_data. Any hints as to what I'm doing wrong?
Dec 06 13:08:37 kvm-test.private lvm[532]: Setting devices/global_filter
to global_filter = [ "r|^/dev/VolGroup/vm_|", "r|^/dev/dm-|", "a|.*/|" ]
Hi Gordon, remove the last component in the filter: 'a|.*/|'.
Filters accept any device if any of it's "names" (all symbolic links) is
matched by an a pattern ("a|.*/|" in your case) and matches no previous
r pattern - for example anything in /dev/mapper/ is accepted.
When used as a blacklist filter(s) must contain only "r" paterns.
BTW it is usually better to use white list: list "a" patterns for all
disks you wish to accept and add "r|.*|" at the end.
-- Martian
___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] lvmlockd: about the limitation on lvresizing the LV active on multiple nodes
Hi David, I see the comments on res_process(): """ /* * Go through queued actions, and make lock/unlock calls on the resource * based on the actions and the existing lock state. * * All lock operations sent to the lock manager are non-blocking. * This is because sanlock does not support lock queueing. * Eventually we could enhance this to take advantage of lock * queueing when available (i.e. for the dlm). """ Is it the reason why lvmlockd has limitation on lvresize with "sh" lock because lvmlockd cannot up convert "sh" to "ex" to perform lvresize command? Regards, Eric On 12/28/2017 06:42 PM, Eric Ren wrote: Hi David, I see there is a limitation on lvesizing the LV active on multiple node. From `man lvmlockd`: """ limitations of lockd VGs ... * resizing an LV that is active in the shared mode on multiple hosts """ It seems a big limitation to use lvmlockd in cluster: """ c1-n1:~ # lvresize -L-1G vg1/lv1 WARNING: Reducing active logical volume to 1.00 GiB. THIS MAY DESTROY YOUR DATA (filesystem etc.) Do you really want to reduce vg1/lv1? [y/n]: y LV is already locked with incompatible mode: vg1/lv1 """ Node "c1-n1" is the last node having vg1/lv1 active on it. Can we change the lock mode from "shared" to "exclusive" to lvresize without having to deactivate the LV on the last node? It will reduce the availability if we have to deactivate LV on all nodes to resize. Is there plan to eliminate this limitation in the near future? Regards, Eric ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
