Re: pvmove hangs

2009-02-11 Thread Sebastian Korte
 IMHO it is not arch or distro specific. It looks exactly like as
 https://bugzilla.redhat.com/show_bug.cgi?id=193330 and the LVM
 developers are keen for somebody with a reproducer.

Dan, thank you very much for the link. Your tip at comment #10 helped.
If I specify each logical volume, pvmove moves all data successfully:
pvmove -v /dev/dasdd1 -n lvhome
pvmove -v /dev/dasdd1 -n lvusr
pvmove -v /dev/dasdd1 -n lvvar
...

It seems to avoid a deadlock situation. I will provide the requested
informations on the bugtracker.

Best regards,
Sebastian

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390


pvmove hangs

2009-02-09 Thread Sebastian Korte
Hello list!

I try to move all logical volumes which reside on a minidisk
(/dev/dasdd1) on DASD MOD9 to a minidisk (/dev/dasde1) on DASD MOD54.
/ and /boot reside on separate minidisks, the rest is LVM managed.

I can reproduce the problem with a test system:
zlinux:~ # df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/dasdc1   879M  113M  767M  13% /
udev  247M  108K  247M   1% /dev
/dev/dasdb145M   13M   30M  31% /boot
/dev/mapper/system-lvhome
  492M   33M  460M   7% /home
/dev/mapper/system-lvopt
  656M   35M  622M   6% /opt
/dev/mapper/system-lvtmp
  368M   33M  336M   9% /tmp
/dev/mapper/system-lvusr
  1.2G  563M  606M  49% /usr
/dev/mapper/system-lvvar
  876M   81M  796M  10% /var
zlinux:~ # pvscan
  PV /dev/dasdd1   VG system   lvm2 [4.57 GB / 1.09 GB free]
  PV /dev/dasde1   VG system   lvm2 [4.80 GB / 4.80 GB free]

zlinux:~ # pvmove -v /dev/dasdd1
Wiping cache of LVM-capable devices
Finding volume group system
Archiving volume group system metadata (seqno 7).
Creating logical volume pvmove0
Executing: /sbin/modprobe dm-mirror
Moving 292 extents of logical volume system/lvusr
Moving 219 extents of logical volume system/lvvar
Moving 164 extents of logical volume system/lvopt
Moving 123 extents of logical volume system/lvhome
Moving 92 extents of logical volume system/lvtmp
Found volume group system
Found volume group system
Found volume group system
Found volume group system
Found volume group system
Updating volume group metadata
Creating volume group backup /etc/lvm/backup/system (seqno 8).
Found volume group system
Found volume group system
Suspending system-lvusr (253:0)
Found volume group system
Found volume group system
Suspending system-lvvar (253:1)
Found volume group system
Found volume group system
Suspending system-lvopt (253:2)
Found volume group system
Found volume group system
Suspending system-lvhome (253:3)
Found volume group system
Found volume group system
Suspending system-lvtmp (253:4)
Found volume group system
Creating system-pvmove0
Loading system-pvmove0 table
Resuming system-pvmove0 (253:5)
  system-pvmove0: device not found
Found volume group system
Loading system-pvmove0 table
Suppressed system-pvmove0 identical table reload.
Loading system-lvusr table
Resuming system-lvusr (253:0)
Found volume group system
Loading system-pvmove0 table
Suppressed system-pvmove0 identical table reload.
Loading system-lvvar table
Resuming system-lvvar (253:1)
Found volume group system
Loading system-pvmove0 table
Suppressed system-pvmove0 identical table reload.
Loading system-lvopt table
Resuming system-lvopt (253:2)
Found volume group system
Loading system-pvmove0 table
Suppressed system-pvmove0 identical table reload.
Loading system-lvhome table
Resuming system-lvhome (253:3)
Found volume group system
Loading system-pvmove0 table
Suppressed system-pvmove0 identical table reload.
Loading system-lvtmp table
Resuming system-lvtmp (253:4)
Checking progress every 15 seconds
  /dev/dasdd1: Moved: 5.5%
  /dev/dasdd1: Moved: 10.8%
  /dev/dasdd1: Moved: 16.1%
  /dev/dasdd1: Moved: 21.3%
  /dev/dasdd1: Moved: 26.5%
  /dev/dasdd1: Moved: 31.9%
  /dev/dasdd1: Moved: 32.8%
Updating volume group metadata
Creating volume group backup /etc/lvm/backup/system (seqno 9).
Found volume group system
Found volume group system
Suspending system-lvusr (253:0)
Suspending system-pvmove0 (253:5)
Found volume group system
Found volume group system
Suspending system-lvvar (253:1)
Found volume group system
Found volume group system
Suspending system-lvopt (253:2)
Found volume group system
Found volume group system
Suspending system-lvhome (253:3)
Found volume group system
Found volume group system
Suspending system-lvtmp (253:4)
Found volume group system
Found volume group system
Found volume group system
Loading system-pvmove0 table
Resuming system-pvmove0 (253:5)
(here the output stops)

Now the system freezes slowly: sshd still gives a login-prompt, but if
I enter the password no login is possible. Other open ssh sessions
freeze are working until I enter a command. After a few minutes sshd
doesn't answer any more. If I reboot the system it doesn't comes up,
because the pvmove is restartet during the boot and the system freezes
again.

I can rescue the system with a few steps: IPL into CMS, detach a LVM
managed minidisk, boot the guest which doesn't comes up (Give root
password to login), edit the fstab with ed and delete all mounts
with LVM volumes, attach the minidisk again and reboot. The system
comes up and pvmove starts to work 

Re: pvmove hangs

2009-02-09 Thread Dan Horák
Sebastian Korte píše v Po 09. 02. 2009 v 15:20 +0100:
 Hello list!
 
 I try to move all logical volumes which reside on a minidisk
 (/dev/dasdd1) on DASD MOD9 to a minidisk (/dev/dasde1) on DASD MOD54.
 / and /boot reside on separate minidisks, the rest is LVM managed.
 

IMHO it is not arch or distro specific. It looks exactly like as
https://bugzilla.redhat.com/show_bug.cgi?id=193330 and the LVM
developers are keen for somebody with a reproducer.

 I can reproduce the problem with a test system:
 zlinux:~ # df -h
 FilesystemSize  Used Avail Use% Mounted on
 /dev/dasdc1   879M  113M  767M  13% /
 udev  247M  108K  247M   1% /dev
 /dev/dasdb145M   13M   30M  31% /boot
 /dev/mapper/system-lvhome
   492M   33M  460M   7% /home
 /dev/mapper/system-lvopt
   656M   35M  622M   6% /opt
 /dev/mapper/system-lvtmp
   368M   33M  336M   9% /tmp
 /dev/mapper/system-lvusr
   1.2G  563M  606M  49% /usr
 /dev/mapper/system-lvvar
   876M   81M  796M  10% /var
 zlinux:~ # pvscan
   PV /dev/dasdd1   VG system   lvm2 [4.57 GB / 1.09 GB free]
   PV /dev/dasde1   VG system   lvm2 [4.80 GB / 4.80 GB free]
 
 zlinux:~ # pvmove -v /dev/dasdd1
 Wiping cache of LVM-capable devices
 Finding volume group system
 Archiving volume group system metadata (seqno 7).
 Creating logical volume pvmove0
 Executing: /sbin/modprobe dm-mirror
 Moving 292 extents of logical volume system/lvusr
 Moving 219 extents of logical volume system/lvvar
 Moving 164 extents of logical volume system/lvopt
 Moving 123 extents of logical volume system/lvhome
 Moving 92 extents of logical volume system/lvtmp
 Found volume group system
 Found volume group system
 Found volume group system
 Found volume group system
 Found volume group system
 Updating volume group metadata
 Creating volume group backup /etc/lvm/backup/system (seqno 8).
 Found volume group system
 Found volume group system
 Suspending system-lvusr (253:0)
 Found volume group system
 Found volume group system
 Suspending system-lvvar (253:1)
 Found volume group system
 Found volume group system
 Suspending system-lvopt (253:2)
 Found volume group system
 Found volume group system
 Suspending system-lvhome (253:3)
 Found volume group system
 Found volume group system
 Suspending system-lvtmp (253:4)
 Found volume group system
 Creating system-pvmove0
 Loading system-pvmove0 table
 Resuming system-pvmove0 (253:5)
   system-pvmove0: device not found
 Found volume group system
 Loading system-pvmove0 table
 Suppressed system-pvmove0 identical table reload.
 Loading system-lvusr table
 Resuming system-lvusr (253:0)
 Found volume group system
 Loading system-pvmove0 table
 Suppressed system-pvmove0 identical table reload.
 Loading system-lvvar table
 Resuming system-lvvar (253:1)
 Found volume group system
 Loading system-pvmove0 table
 Suppressed system-pvmove0 identical table reload.
 Loading system-lvopt table
 Resuming system-lvopt (253:2)
 Found volume group system
 Loading system-pvmove0 table
 Suppressed system-pvmove0 identical table reload.
 Loading system-lvhome table
 Resuming system-lvhome (253:3)
 Found volume group system
 Loading system-pvmove0 table
 Suppressed system-pvmove0 identical table reload.
 Loading system-lvtmp table
 Resuming system-lvtmp (253:4)
 Checking progress every 15 seconds
   /dev/dasdd1: Moved: 5.5%
   /dev/dasdd1: Moved: 10.8%
   /dev/dasdd1: Moved: 16.1%
   /dev/dasdd1: Moved: 21.3%
   /dev/dasdd1: Moved: 26.5%
   /dev/dasdd1: Moved: 31.9%
   /dev/dasdd1: Moved: 32.8%
 Updating volume group metadata
 Creating volume group backup /etc/lvm/backup/system (seqno 9).
 Found volume group system
 Found volume group system
 Suspending system-lvusr (253:0)
 Suspending system-pvmove0 (253:5)
 Found volume group system
 Found volume group system
 Suspending system-lvvar (253:1)
 Found volume group system
 Found volume group system
 Suspending system-lvopt (253:2)
 Found volume group system
 Found volume group system
 Suspending system-lvhome (253:3)
 Found volume group system
 Found volume group system
 Suspending system-lvtmp (253:4)
 Found volume group system
 Found volume group system
 Found volume group system
 Loading system-pvmove0 table
 Resuming system-pvmove0 (253:5)
 (here the output stops)
 
 Now the system freezes slowly: sshd still gives a login-prompt, but if
 I enter the password no login is possible. Other open ssh sessions
 freeze are working until I enter a command. After a few minutes sshd
 doesn't answer any more. If I reboot the system it doesn't comes up,
 because the pvmove 

Re: pvmove hangs

2009-02-09 Thread Mark Post
 On 2/9/2009 at  9:20 AM, Sebastian Korte zlinu...@gmail.com wrote: 
 Hello list!
 
 I try to move all logical volumes which reside on a minidisk
 (/dev/dasdd1) on DASD MOD9 to a minidisk (/dev/dasde1) on DASD MOD54.
 / and /boot reside on separate minidisks, the rest is LVM managed.
-snip-
 Now the system freezes slowly
-snip-
 The system
 comes up and pvmove starts to work again: (lvm2copyd)-a y After
 pvmove is ready I replace the fstab with its original and reboot.
 Voila, here we go again.

Try using pvmove --abort to kill the move in progress.

This looks very similar to https://bugzilla.novell.com/show_bug.cgi?id=473309 
pvmove freezes system that was just opened recently.  I would suggest opening 
up a service request with your service provider.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390