Public bug reported: When Fibre Channel and multipath are used, the multipath device descriptor is not deleted after a volume is detached from a VM. This is always reproducible.
This was seen with git stable/kilo. Reproduce steps: 1. Check the multipath device on the system stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume size=222G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 6:1:1:0 sdb 8:16 active ready running 2. Attach a volume to a VM runing on the current system, and wait till the volume becomes to 'in-use' stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-attach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | fa7c94ae-e666-4923-be2b-8965036dce1a | | serverId | c6244601-73e0-4210-8654-356a4154883b | | volumeId | fa7c94ae-e666-4923-be2b-8965036dce1a | +----------+--------------------------------------+ stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | fa7c94ae-e666-4923-be2b-8965036dce1a | in-use | vol1 | 1 | None | false | c6244601-73e0-4210-8654-356a4154883b | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ 3. Check the multipath devices on the system. 3600601602ba03400efd6d0247c03e511 is the multipath device mapping to the attached volume stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511 dm-1 DGC,VRAID size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=130 status=active | `- 7:0:4:225 sdp 8:240 active ready running `-+- policy='round-robin 0' prio=10 status=enabled `- 7:0:2:225 sdo 8:224 active ready running 3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume size=222G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 6:1:1:0 sdb 8:16 active ready running 4. Detach the volume and wait till the volume becomes 'available' stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-detach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ | fa7c94ae-e666-4923-be2b-8965036dce1a | available | vol1 | 1 | None | false | | +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ 5. Check the multipath devices again. We can see the descriptor 3600601602ba03400efd6d0247c03e511 still exists stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511 3600601602ba03400efd6d0247c03e511 dm-1 , size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=-1 status=active | `- #:#:#:# - #:# active undef running `-+- policy='round-robin 0' prio=-1 status=enabled `- #:#:#:# - #:# active undef running ================ Code Version: stack@ubuntu-server12:/opt/stack/nova$ git status On branch stable/kilo Your branch is up-to-date with 'origin/stable/kilo'. nothing to commit, working directory clean stack@ubuntu-server12:/opt/stack/nova$ git log -1 commit 36fb00291d819b65f46d530eefbf07b883ca8a29 Merge: 0c60aca 8e9ecf7 Author: Jenkins <jenk...@review.openstack.org> Date: Fri May 15 16:16:00 2015 +0000 Merge "Libvirt: Use tpool to invoke guestfs api" into stable/kilo ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1459073 Title: Multipath device descriptor is not deleted after volume detached when FC and multipath are used Status in OpenStack Compute (Nova): New Bug description: When Fibre Channel and multipath are used, the multipath device descriptor is not deleted after a volume is detached from a VM. This is always reproducible. This was seen with git stable/kilo. Reproduce steps: 1. Check the multipath device on the system stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume size=222G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 6:1:1:0 sdb 8:16 active ready running 2. Attach a volume to a VM runing on the current system, and wait till the volume becomes to 'in-use' stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-attach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | fa7c94ae-e666-4923-be2b-8965036dce1a | | serverId | c6244601-73e0-4210-8654-356a4154883b | | volumeId | fa7c94ae-e666-4923-be2b-8965036dce1a | +----------+--------------------------------------+ stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | fa7c94ae-e666-4923-be2b-8965036dce1a | in-use | vol1 | 1 | None | false | c6244601-73e0-4210-8654-356a4154883b | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ 3. Check the multipath devices on the system. 3600601602ba03400efd6d0247c03e511 is the multipath device mapping to the attached volume stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511 dm-1 DGC,VRAID size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=130 status=active | `- 7:0:4:225 sdp 8:240 active ready running `-+- policy='round-robin 0' prio=10 status=enabled `- 7:0:2:225 sdo 8:224 active ready running 3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume size=222G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 6:1:1:0 sdb 8:16 active ready running 4. Detach the volume and wait till the volume becomes 'available' stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-detach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ | fa7c94ae-e666-4923-be2b-8965036dce1a | available | vol1 | 1 | None | false | | +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+ 5. Check the multipath devices again. We can see the descriptor 3600601602ba03400efd6d0247c03e511 still exists stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511 3600601602ba03400efd6d0247c03e511 dm-1 , size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=-1 status=active | `- #:#:#:# - #:# active undef running `-+- policy='round-robin 0' prio=-1 status=enabled `- #:#:#:# - #:# active undef running ================ Code Version: stack@ubuntu-server12:/opt/stack/nova$ git status On branch stable/kilo Your branch is up-to-date with 'origin/stable/kilo'. nothing to commit, working directory clean stack@ubuntu-server12:/opt/stack/nova$ git log -1 commit 36fb00291d819b65f46d530eefbf07b883ca8a29 Merge: 0c60aca 8e9ecf7 Author: Jenkins <jenk...@review.openstack.org> Date: Fri May 15 16:16:00 2015 +0000 Merge "Libvirt: Use tpool to invoke guestfs api" into stable/kilo To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1459073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp