Public bug reported:
Multipath descripter is not removed in volume detachment
In the iSCSI multipath environment, after I detach a last volume from a
instance on the host. Althougth the devices under /dev/disk/by-path are
removed by the multipath descriptor is not removed. I am using VNX as
cinder backend.
========How I produced this issue=============
1. Before I do an detachment, there is only LUN 86 attached to the host.
stack@ubuntu-server7:/dev/disk/by-path$ ls
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part1
pci-0000:05:00.1-fc-0x500601610860080f-lun-0
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-86
pci-0000:04:00.0-scsi-0:1:1:0-part2
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part5
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10
pci-0000:04:00.0-scsi-0:1:0:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0
pci-0000:04:00.0-scsi-0:1:1:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10
stack@ubuntu-server7:/dev/disk/by-path$ ls -l
total 0
lrwxrwxrwx 1 root root 9 May 12 23:26
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-0 ->
../../sdh
lrwxrwxrwx 1 root root 9 May 12 23:26
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-86 ->
../../sdi
lrwxrwxrwx 1 root root 9 May 12 23:28
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0 ->
../../sdj
lrwxrwxrwx 1 root root 9 May 12 18:07 pci-0000:04:00.0-scsi-0:1:0:0 ->
../../sdb
lrwxrwxrwx 1 root root 9 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0 ->
../../sda
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part1 ->
../../sda1
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part2 ->
../../sda2
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part5 ->
../../sda5
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0 -> ../../sdc
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10 -> ../../sdd
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x500601610860080f-lun-0 -> ../../sde
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0 -> ../../sdf
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10 -> ../../sdg
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -l /dev/sdi
3600601601bd03200d8f0c1714bf9e411 dm-6 DGC,VRAID
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=-1 status=active
`- 12:0:0:86 sdi 8:128 active undef running
2. Trigger the detachment
stack@ubuntu-server7:/dev/disk/by-path$ nova volume-detach
818eb5fb-66fe-4179-9466-04ef1ec09a8d 837bd1e1-8d82-472e-bd42-f8edd4fbcc42
3. After the detachment, The devices under /dev/disk/by-path went away this is
as excpeted
stack@ubuntu-server7:/dev/disk/by-path$ ls
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part2
pci-0000:05:00.1-fc-0x500601610860080f-lun-0
pci-0000:04:00.0-scsi-0:1:0:0
pci-0000:04:00.0-scsi-0:1:1:0-part5
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0
pci-0000:04:00.0-scsi-0:1:1:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10
pci-0000:04:00.0-scsi-0:1:1:0-part1
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10
4. But the multipath descripter is not removed 3600601601bd03200d8f0c1714bf9e411
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -ll
3600601602ba034002c160db00960e411 dm-1 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| `- 9:0:2:10 sdg 8:96 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 9:0:0:10 sdd 8:48 active ready running
3600508e0000000001952e401650d0b0a dm-2 LSI,Logical Volume
size=929G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sda 8:0 active ready running
3600601601bd03200d8f0c1714bf9e411 dm-6 ,
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
`- #:#:#:# - #:# active faulty running
3600508e0000000009551404ce9efae0d dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:0:0 sdb 8:16 active ready running
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -l
3600601601bd03200d8f0c1714bf9e411
3600601601bd03200d8f0c1714bf9e411 dm-6 ,
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=-1 status=active
`- #:#:#:# - #:# active undef running
======Nova version ==========
stack@ubuntu-server7:/opt/stack/nova$ git log -1
commit 39bbc0d84624d95f13e3d911f38b7ae483c4505e
Author: OpenStack Proposal Bot <openstack-in...@lists.openstack.org>
Date: Sat May 2 06:19:07 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: Ifeccc8216e19a2234cd0be983f289fb7a2a0573f
========Problem ========================
Seems the multipath descripter will not be removed automatically after we
logout the iscsi targets. multipath -f <multipath id> may need to be invoked
explictly.
nova/nova/virt/libvirt/volume.py
in method _disconnect_volume_multipath_iscsi, when there is no other devices
attached or the iqn is not used by other devices, it just logout the targets
without delete the multipath descriptor
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1454621
Title:
Multipath device descripter is not deleted while the device pathes are
removed after detach last volume form VM on a host
Status in OpenStack Compute (Nova):
New
Bug description:
Multipath descripter is not removed in volume detachment
In the iSCSI multipath environment, after I detach a last volume from
a instance on the host. Althougth the devices under /dev/disk/by-path
are removed by the multipath descriptor is not removed. I am using VNX
as cinder backend.
========How I produced this issue=============
1. Before I do an detachment, there is only LUN 86 attached to the host.
stack@ubuntu-server7:/dev/disk/by-path$ ls
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part1
pci-0000:05:00.1-fc-0x500601610860080f-lun-0
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-86
pci-0000:04:00.0-scsi-0:1:1:0-part2
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part5
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10
pci-0000:04:00.0-scsi-0:1:0:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0
pci-0000:04:00.0-scsi-0:1:1:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10
stack@ubuntu-server7:/dev/disk/by-path$ ls -l
total 0
lrwxrwxrwx 1 root root 9 May 12 23:26
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-0 ->
../../sdh
lrwxrwxrwx 1 root root 9 May 12 23:26
ip-192.168.4.52:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.a6-lun-86 ->
../../sdi
lrwxrwxrwx 1 root root 9 May 12 23:28
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0 ->
../../sdj
lrwxrwxrwx 1 root root 9 May 12 18:07 pci-0000:04:00.0-scsi-0:1:0:0 ->
../../sdb
lrwxrwxrwx 1 root root 9 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0 ->
../../sda
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part1 ->
../../sda1
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part2 ->
../../sda2
lrwxrwxrwx 1 root root 10 May 12 18:07 pci-0000:04:00.0-scsi-0:1:1:0-part5 ->
../../sda5
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0 -> ../../sdc
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10 -> ../../sdd
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x500601610860080f-lun-0 -> ../../sde
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0 -> ../../sdf
lrwxrwxrwx 1 root root 9 May 12 18:07
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10 -> ../../sdg
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -l /dev/sdi
3600601601bd03200d8f0c1714bf9e411 dm-6 DGC,VRAID
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=-1 status=active
`- 12:0:0:86 sdi 8:128 active undef running
2. Trigger the detachment
stack@ubuntu-server7:/dev/disk/by-path$ nova volume-detach
818eb5fb-66fe-4179-9466-04ef1ec09a8d 837bd1e1-8d82-472e-bd42-f8edd4fbcc42
3. After the detachment, The devices under /dev/disk/by-path went away this
is as excpeted
stack@ubuntu-server7:/dev/disk/by-path$ ls
ip-192.168.4.53:3260-iscsi-iqn.1992-04.com.emc:cx.fnm00130200235.b6-lun-0
pci-0000:04:00.0-scsi-0:1:1:0-part2
pci-0000:05:00.1-fc-0x500601610860080f-lun-0
pci-0000:04:00.0-scsi-0:1:0:0
pci-0000:04:00.0-scsi-0:1:1:0-part5
pci-0000:05:00.1-fc-0x50060169086003ba-lun-0
pci-0000:04:00.0-scsi-0:1:1:0
pci-0000:05:00.1-fc-0x50060161086003ba-lun-0
pci-0000:05:00.1-fc-0x50060169086003ba-lun-10
pci-0000:04:00.0-scsi-0:1:1:0-part1
pci-0000:05:00.1-fc-0x50060161086003ba-lun-10
4. But the multipath descripter is not removed
3600601601bd03200d8f0c1714bf9e411
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -ll
3600601602ba034002c160db00960e411 dm-1 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| `- 9:0:2:10 sdg 8:96 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 9:0:0:10 sdd 8:48 active ready running
3600508e0000000001952e401650d0b0a dm-2 LSI,Logical Volume
size=929G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sda 8:0 active ready running
3600601601bd03200d8f0c1714bf9e411 dm-6 ,
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
`- #:#:#:# - #:# active faulty running
3600508e0000000009551404ce9efae0d dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:0:0 sdb 8:16 active ready running
stack@ubuntu-server7:/dev/disk/by-path$ sudo multipath -l
3600601601bd03200d8f0c1714bf9e411
3600601601bd03200d8f0c1714bf9e411 dm-6 ,
size=3.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=-1 status=active
`- #:#:#:# - #:# active undef running
======Nova version ==========
stack@ubuntu-server7:/opt/stack/nova$ git log -1
commit 39bbc0d84624d95f13e3d911f38b7ae483c4505e
Author: OpenStack Proposal Bot <openstack-in...@lists.openstack.org>
Date: Sat May 2 06:19:07 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: Ifeccc8216e19a2234cd0be983f289fb7a2a0573f
========Problem ========================
Seems the multipath descripter will not be removed automatically after we
logout the iscsi targets. multipath -f <multipath id> may need to be invoked
explictly.
nova/nova/virt/libvirt/volume.py
in method _disconnect_volume_multipath_iscsi, when there is no other devices
attached or the iqn is not used by other devices, it just logout the targets
without delete the multipath descriptor
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1454621/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
