Re: [linux-lvm] vgchange acquire an exclusive VG lock for refresh
Thank you for your reply.
According to your review, I reworked a proper patch.
---
tools/vgchange.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/vgchange.c b/tools/vgchange.c
index e2d3dad02..8c1d605c6 100644
--- a/tools/vgchange.c
+++ b/tools/vgchange.c
@@ -1008,7 +1008,8 @@ int vgchange(struct cmd_context *cmd, int argc, char
**argv)
if (update)
flags |= READ_FOR_UPDATE;
- else if (arg_is_set(cmd, activate_ARG))
+ else if (arg_is_set(cmd, activate_ARG) ||
+arg_is_set(cmd, refresh_ARG))
flags |= READ_FOR_ACTIVATE;
if (!(handle = init_processing_handle(cmd, NULL))) {
--
2.33.0
在 2023/8/29 14:36, wangzhiqiang (Q) 写道:
> Patches based on the latest version:
>
> ---
> tools/vgchange.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/vgchange.c b/tools/vgchange.c
> index f07b245bd..f35ff4c0e 100644
> --- a/tools/vgchange.c
> +++ b/tools/vgchange.c
> @@ -1003,7 +1003,7 @@ int vgchange(struct cmd_context *cmd, int argc, char
> **argv)
> init_external_device_info_source(DEV_EXT_NONE);
> }
>
> - if (update)
> + if (update || arg_is_set(cmd, refresh_ARG))
> flags |= READ_FOR_UPDATE;
> else if (arg_is_set(cmd, activate_ARG))
> flags |= READ_FOR_ACTIVATE;
> --
> 2.33.0
>
> 在 2023/8/29 11:53, wangzhiqiang (Q) 写道:
>> I've got an issue while execute vgchange --refresh concurrently, process
>> vgchange and systemd-udevd
>> enters the D state.
>> root2933 0.0 1.0 54216 31344 pts/1D> --refresh
>> root2934 0.0 1.1 54216 32788 pts/1D> --refresh
>> root2935 0.0 1.0 54216 31496 pts/1D> --refresh
>> root2937 0.0 1.0 54216 31500 pts/1S> --refresh
>> root2951 0.0 0.0 0 0 ?D11:21 0:00
>> [systemd-udevd]
>> root2952 0.0 0.0 0 0 ?D11:21 0:00
>> [systemd-udevd]
>>
>> process stack as follows
>> [root@localhost ~]# cat /proc/2937/stack
>> [<0>] dm_wait_for_bios_completion+0x298/0x340 [dm_mod]
>> [<0>] dm_wait_for_completion+0xfe/0x110 [dm_mod]
>> [<0>] __dm_suspend+0xd0/0x310 [dm_mod]
>> [<0>] dm_suspend+0x133/0x180 [dm_mod]
>> [<0>] dev_suspend+0x115/0x150 [dm_mod]
>> [<0>] ctl_ioctl+0x320/0x4c0 [dm_mod]
>> [<0>] dm_ctl_ioctl+0x26/0x40 [dm_mod]
>> [<0>] __se_sys_ioctl+0xeb/0x120
>> [<0>] do_syscall_64+0x30/0x40
>> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
>> [root@localhost ~]# cat /proc/2935/stack
>> [<0>] dm_suspend+0x2b/0x180 [dm_mod]
>> [<0>] dev_suspend+0x115/0x150 [dm_mod]
>> [<0>] ctl_ioctl+0x320/0x4c0 [dm_mod]
>> [<0>] dm_ctl_ioctl+0x26/0x40 [dm_mod]
>> [<0>] __se_sys_ioctl+0xeb/0x120
>> [<0>] do_syscall_64+0x30/0x40
>> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
>> [root@localhost ~]# cat /proc/2951/stack
>> [<0>] __lock_page+0x25a/0x7a0
>> [<0>] truncate_inode_pages_range+0xc52/0x1370
>> [<0>] __blkdev_put+0x15b/0x3f0
>> [<0>] blkdev_close+0x49/0x60
>> [<0>] __fput+0x1b9/0x480
>> [<0>] task_work_run+0xad/0xf0
>> [<0>] do_exit+0x47f/0x810
>> [<0>] do_group_exit+0x77/0x150
>> [<0>] get_signal+0x2a2/0xa60
>> [<0>] arch_do_signal_or_restart+0x1c6/0x310
>> [<0>] exit_to_user_mode_loop+0xdb/0x100
>> [<0>] exit_to_user_mode_prepare+0x7e/0x90
>> [<0>] syscall_exit_to_user_mode+0x12/0x40
>> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
>>
>> I recommend get an exclusive lock when vgchange --refresh.
>>
>> >From 7ddadf2bbbe0383893ab8f5b7cb8d51ebf7a5ed4 Mon Sep 17 00:00:00 2001
>> From: wangzhiqiang
>> Date: Tue, 29 Aug 2023 10:28:15 +0800
>> Subject: [PATCH] vgchange acquire an exclusive VG lock for refresh.
>>
>> Let's run the following test cases, we've got hungtask.
>> P2 or P3 can't suspend snap and udevd also can't read
>> device info because real or cow has been suspend.
>>
>> P1(vgchange) P2(vgchange) P3(vgchange)
>>
>> suspend real
>> suspend cow
>> suspend snap
>> suspend snap
>> suspend snap
>>
>> function query()
>> {
>> while true
>> do
>> vgchange --refresh
>> done
>> }
>>
>> query 1 &> /dev/null &
>> query 2 &> /dev/null &
>> query 3 &> /dev/null &
>>
>> [ 247.250669] INFO: task vgchange:2934 blocked for more than 122 seconds.
>> [ 247.251935] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
>> this message.
>> [ 247.252619] task:vgchangestate:D stack:0 pid: 2934 ppid:
>> 2930 flags:0x0080
>> [ 247.252626] Call Trace:
>> [ 247.252633] __schedule+0x547/0xe80
>> [ 247.252640] ? io_schedule_timeout+0xb0/0xb0
>> [ 247.252647] ? _raw_spin_lock+0x7a/0xd0
>> [ 247.252652] schedule+0x6f/0x130
>> [ 247.252658] schedule_preempt_disabled+0xa/0x10
>> [ 247.252664] __mutex_lock.constprop.0+0x311/0x700
>> [ 247.252673] ?
Re: [linux-lvm] vgchange acquire an exclusive VG lock for refresh
On Tue, Aug 29, 2023 at 02:36:53PM +0800, wangzhiqiang (Q) wrote: > diff --git a/tools/vgchange.c b/tools/vgchange.c > index f07b245bd..f35ff4c0e 100644 > --- a/tools/vgchange.c > +++ b/tools/vgchange.c > @@ -1003,7 +1003,7 @@ int vgchange(struct cmd_context *cmd, int argc, char > **argv) > init_external_device_info_source(DEV_EXT_NONE); > } > > - if (update) > + if (update || arg_is_set(cmd, refresh_ARG)) > flags |= READ_FOR_UPDATE; > else if (arg_is_set(cmd, activate_ARG)) > flags |= READ_FOR_ACTIVATE; Shouldn't it be: else if (arg_is_set(cmd, activate_ARG) || arg_is_set(cmd, refresh_ARG)) flags |= READ_FOR_ACTIVATE; READ_FOR_ACTIVATE should translate to an ex lock like READ_FOR_UPDATE. Note the same fix for lvchange refresh in commit 47bcb446b83f9aa4fca74301fda109a8bf670fbb Dave ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Swapping LLVM drive
On Mon, 28 Aug 2023, Roska Postit wrote: After reading your answer more carefully I got the following idea: How do you see if I boot the system (this is a desktop computer and the old and the new drive are both NVMe SSD) from USB Linux and then just do a 'dd' for the entire drive (in block level, bit-by-bit). Then I remove the old disk out of the system. Shouldn't it boot normally now ? That would work, yes, but you don't expand /boot - which you really should do. Also, copying the entire filesystem is not only less efficient, but involves needless writes to an SSD (where they are a more limited resource than on magnetic drives). Then I will create a new partition for the all unused space (1.5GB) on new disk which I then will add to the LVM as a new Physical Volume (PV) in That is pointless when you can just expand the partition (which is trivial when it is the last one). You don't want more PVs unless they are actually physical volumes - or there is some special circumstance that prevents just expanding the partition. ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Swapping LLVM drive
On Mon, 28 Aug 2023, Phillip Susi wrote: Why would you use dd/partclone instead of just having LVM move everything to the new drive on the fly? Partition the new drive, use pvcreate to initialize the partition as a pv, vgextend to add the pv to the existing vg, pvmove to evacuate the logical volumes from the old disk, then vgreduce to remove it from the vg. 1. Unnecessary copying. 2. You lose your free backup of the system on the old drive, which should be carefully labeled and kept handy for a year. (After that, SSDs start to run the risk of data retention issues.) Don't forget you'll need to reinstall grub on the new drive for it to boot. And that is the most important reason. "Just reinstall grub" is a much larger learning curve than "dd" IMO. ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Swapping LLVM drive
Thank you very much for your time. This was very very helpful. ma 28. elok. 2023 klo 18.34 Stuart D Gathman (stu...@gathman.org) kirjoitti: > On Sun, 27 Aug 2023, Roska Postit wrote: > > > What is the most proper way to swap my 500GB SSD drive to the bigger 2TB > SSD > > drive in the following LLVM configuration ? > > > > nvme0n1259:00 465,8G 0 disk > > ├─nvme0n1p1259:10 512M 0 part /boot/efi > > ├─nvme0n1p2259:20 488M 0 part /boot > > └─nvme0n1p3259:30 464,8G 0 part > > ├─pc3--vg-root 254:00 463,8G 0 lvm / > > └─pc3--vg-swap_1 254:10 980M 0 lvm [SWAP] > > Since you are not mirroring, just add the new drive. > > If this is a laptop, and you can only have one drive, then I suggest > you mount the new drive via USB (note there are at least 2 kinds of > nvme interface and you have to get a matching USB enclosure). > > Use dd to copy the partition table (this also often contains boot code) > to the new disk on USB. > Then use dd to copy the smaller partitions (efi,boot). > Now use cfdisk to delete the 3rd partition. > Expand the boot partition to 1G (you'll thank me later). > Allocate the entire rest of the disk to p3. > Create a new vg with a different name. Allocate root and swap on > new VG the same sizes. > Take a snapshot of current root (delete swap on old drive since you > didn't leave yourself any room), and use partclone to efficiently > copy the filesystem over to new root. > > Either a) edit grub and fstab on new drive to use new vg name or > b) boot from a live media to rename old and new vg or > c) rename vg just before shutting down to remove drive - >I think LVM can operate with dup VG name, but I've never >navigated the details. > > Swap drives after powerdown. > > A modern filesystem like ext2, xfs, btrfs, etc can expand as you expand > the root LV. Leave yourself some working room in the > VG.___ > linux-lvm mailing list > linux-lvm@redhat.com > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Swapping LLVM drive
After reading your answer more carefully I got the following idea: How do you see if I boot the system (this is a desktop computer and the old and the new drive are both NVMe SSD) from USB Linux and then just do a 'dd' for the entire drive (in block level, bit-by-bit). Then I remove the old disk out of the system. Shouldn't it boot normally now ? Then I will create a new partition for the all unused space (1.5GB) on new disk which I then will add to the LVM as a new Physical Volume (PV) in pc3_vg group. Then I just need to configure the Logical Volume to use that new added storage space. If this is possible I think it would be a bit simpler way to do this. What do you think about this method ? ma 28. elok. 2023 klo 19.31 Roska Postit (roskapostit99...@gmail.com) kirjoitti: > Thank you very much for your time. This was very very helpful. > > ma 28. elok. 2023 klo 18.34 Stuart D Gathman (stu...@gathman.org) > kirjoitti: > >> On Sun, 27 Aug 2023, Roska Postit wrote: >> >> > What is the most proper way to swap my 500GB SSD drive to the bigger >> 2TB SSD >> > drive in the following LLVM configuration ? >> > >> > nvme0n1259:00 465,8G 0 disk >> > ├─nvme0n1p1259:10 512M 0 part /boot/efi >> > ├─nvme0n1p2259:20 488M 0 part /boot >> > └─nvme0n1p3259:30 464,8G 0 part >> > ├─pc3--vg-root 254:00 463,8G 0 lvm / >> > └─pc3--vg-swap_1 254:10 980M 0 lvm [SWAP] >> >> Since you are not mirroring, just add the new drive. >> >> If this is a laptop, and you can only have one drive, then I suggest >> you mount the new drive via USB (note there are at least 2 kinds of >> nvme interface and you have to get a matching USB enclosure). >> >> Use dd to copy the partition table (this also often contains boot code) >> to the new disk on USB. >> Then use dd to copy the smaller partitions (efi,boot). >> Now use cfdisk to delete the 3rd partition. >> Expand the boot partition to 1G (you'll thank me later). >> Allocate the entire rest of the disk to p3. >> Create a new vg with a different name. Allocate root and swap on >> new VG the same sizes. >> Take a snapshot of current root (delete swap on old drive since you >> didn't leave yourself any room), and use partclone to efficiently >> copy the filesystem over to new root. >> >> Either a) edit grub and fstab on new drive to use new vg name or >> b) boot from a live media to rename old and new vg or >> c) rename vg just before shutting down to remove drive - >>I think LVM can operate with dup VG name, but I've never >>navigated the details. >> >> Swap drives after powerdown. >> >> A modern filesystem like ext2, xfs, btrfs, etc can expand as you expand >> the root LV. Leave yourself some working room in the >> VG.___ >> linux-lvm mailing list >> linux-lvm@redhat.com >> https://listman.redhat.com/mailman/listinfo/linux-lvm >> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >> > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] vgchange acquire an exclusive VG lock for refresh
Patches based on the latest version:
---
tools/vgchange.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/vgchange.c b/tools/vgchange.c
index f07b245bd..f35ff4c0e 100644
--- a/tools/vgchange.c
+++ b/tools/vgchange.c
@@ -1003,7 +1003,7 @@ int vgchange(struct cmd_context *cmd, int argc, char
**argv)
init_external_device_info_source(DEV_EXT_NONE);
}
- if (update)
+ if (update || arg_is_set(cmd, refresh_ARG))
flags |= READ_FOR_UPDATE;
else if (arg_is_set(cmd, activate_ARG))
flags |= READ_FOR_ACTIVATE;
--
2.33.0
在 2023/8/29 11:53, wangzhiqiang (Q) 写道:
> I've got an issue while execute vgchange --refresh concurrently, process
> vgchange and systemd-udevd
> enters the D state.
> root2933 0.0 1.0 54216 31344 pts/1D --refresh
> root2934 0.0 1.1 54216 32788 pts/1D --refresh
> root2935 0.0 1.0 54216 31496 pts/1D --refresh
> root2937 0.0 1.0 54216 31500 pts/1S --refresh
> root2951 0.0 0.0 0 0 ?D11:21 0:00
> [systemd-udevd]
> root2952 0.0 0.0 0 0 ?D11:21 0:00
> [systemd-udevd]
>
> process stack as follows
> [root@localhost ~]# cat /proc/2937/stack
> [<0>] dm_wait_for_bios_completion+0x298/0x340 [dm_mod]
> [<0>] dm_wait_for_completion+0xfe/0x110 [dm_mod]
> [<0>] __dm_suspend+0xd0/0x310 [dm_mod]
> [<0>] dm_suspend+0x133/0x180 [dm_mod]
> [<0>] dev_suspend+0x115/0x150 [dm_mod]
> [<0>] ctl_ioctl+0x320/0x4c0 [dm_mod]
> [<0>] dm_ctl_ioctl+0x26/0x40 [dm_mod]
> [<0>] __se_sys_ioctl+0xeb/0x120
> [<0>] do_syscall_64+0x30/0x40
> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
> [root@localhost ~]# cat /proc/2935/stack
> [<0>] dm_suspend+0x2b/0x180 [dm_mod]
> [<0>] dev_suspend+0x115/0x150 [dm_mod]
> [<0>] ctl_ioctl+0x320/0x4c0 [dm_mod]
> [<0>] dm_ctl_ioctl+0x26/0x40 [dm_mod]
> [<0>] __se_sys_ioctl+0xeb/0x120
> [<0>] do_syscall_64+0x30/0x40
> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
> [root@localhost ~]# cat /proc/2951/stack
> [<0>] __lock_page+0x25a/0x7a0
> [<0>] truncate_inode_pages_range+0xc52/0x1370
> [<0>] __blkdev_put+0x15b/0x3f0
> [<0>] blkdev_close+0x49/0x60
> [<0>] __fput+0x1b9/0x480
> [<0>] task_work_run+0xad/0xf0
> [<0>] do_exit+0x47f/0x810
> [<0>] do_group_exit+0x77/0x150
> [<0>] get_signal+0x2a2/0xa60
> [<0>] arch_do_signal_or_restart+0x1c6/0x310
> [<0>] exit_to_user_mode_loop+0xdb/0x100
> [<0>] exit_to_user_mode_prepare+0x7e/0x90
> [<0>] syscall_exit_to_user_mode+0x12/0x40
> [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
>
> I recommend get an exclusive lock when vgchange --refresh.
>
>>From 7ddadf2bbbe0383893ab8f5b7cb8d51ebf7a5ed4 Mon Sep 17 00:00:00 2001
> From: wangzhiqiang
> Date: Tue, 29 Aug 2023 10:28:15 +0800
> Subject: [PATCH] vgchange acquire an exclusive VG lock for refresh.
>
> Let's run the following test cases, we've got hungtask.
> P2 or P3 can't suspend snap and udevd also can't read
> device info because real or cow has been suspend.
>
> P1(vgchange) P2(vgchange) P3(vgchange)
>
> suspend real
> suspend cow
> suspend snap
> suspend snap
> suspend snap
>
> function query()
> {
> while true
> do
> vgchange --refresh
> done
> }
>
> query 1 &> /dev/null &
> query 2 &> /dev/null &
> query 3 &> /dev/null &
>
> [ 247.250669] INFO: task vgchange:2934 blocked for more than 122 seconds.
> [ 247.251935] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
> this message.
> [ 247.252619] task:vgchangestate:D stack:0 pid: 2934 ppid: 2930
> flags:0x0080
> [ 247.252626] Call Trace:
> [ 247.252633] __schedule+0x547/0xe80
> [ 247.252640] ? io_schedule_timeout+0xb0/0xb0
> [ 247.252647] ? _raw_spin_lock+0x7a/0xd0
> [ 247.252652] schedule+0x6f/0x130
> [ 247.252658] schedule_preempt_disabled+0xa/0x10
> [ 247.252664] __mutex_lock.constprop.0+0x311/0x700
> [ 247.252673] ? kernel_text_address.part.0+0xb9/0xc0
> [ 247.252678] ? ww_mutex_unlock+0x50/0x50
> [ 247.252685] ? unwind_get_return_address+0x33/0x50
> [ 247.252692] ? cred_has_capability.isra.0+0x14b/0x220
> [ 247.252698] ? check_nnp_nosuid.isra.0+0x170/0x170
> [ 247.252704] mutex_lock+0x114/0x130
> [ 247.252710] ? __mutex_lock_slowpath+0x10/0x10
> [ 247.252716] ? rwsem_down_write_slowpath+0x830/0x830
> [ 247.252733] ? dm_get+0x40/0x70 [dm_mod]
> [ 247.252752] ? dm_get_md+0xb3/0x120 [dm_mod]
> [ 247.252770] dm_suspend+0x2b/0x180 [dm_mod]
> [ 247.252788] dev_suspend+0x115/0x150 [dm_mod]
> [ 247.252805] ctl_ioctl+0x320/0x4c0 [dm_mod]
> [ 247.252823] ? dev_set_geometry+0x290/0x290 [dm_mod]
> [ 247.252841] ? copy_params+0x2e0/0x2e0 [dm_mod]
> [ 247.252846] ? locks_dispose_list+0xa0/0xd0
> [ 247.252851] ? locks_remove_posix+0x24f/0x340
> [ 247.252856] ?
