Chandan Purushothama created CLOUDSTACK-1699: ------------------------------------------------
Summary: Volumes on Primary Storage of a VMWare Setup are not getting cleaned up after VM deletion&expunge Key: CLOUDSTACK-1699 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1699 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.2.0 Reporter: Chandan Purushothama Fix For: 4.2.0 ========== Observations: ========== **On the Database** mysql> select id,name,instance_name,state,removed from vm_instance where removed is not null; +----+------------+---------------+-----------+---------------------+ | id | name | instance_name | state | removed | +----+------------+---------------+-----------+---------------------+ | 3 | boron-VM-1 | i-3-3-VMSNAP | Expunging | 2013-03-14 21:35:09 | | 5 | boron-VM-2 | i-3-5-VMSNAP | Expunging | 2013-03-14 21:37:04 | | 6 | boron-VM-3 | i-3-6-VMSNAP | Expunging | 2013-03-14 02:19:43 | | 9 | boron-VM-6 | i-3-9-VMSNAP | Expunging | 2013-03-15 20:44:16 | +----+------------+---------------+-----------+---------------------+ 4 rows in set (0.00 sec) mysql> select id,name,state from volumes where instance_id in (3,5,6,9); +----+--------+----------+ | id | name | state | +----+--------+----------+ | 3 | ROOT-3 | Expunged | | 6 | ROOT-5 | Expunged | | 7 | ROOT-6 | Expunged | | 14 | ROOT-9 | Expunged | +----+--------+----------+ 4 rows in set (0.00 sec) mysql> select * from configuration where name like "storage.cleanup%"; +----------+----------+----------------+--------------------------+-------+------------------------------------------------------------------------------+ | category | instance | component | name | value | description | +----------+----------+----------------+--------------------------+-------+------------------------------------------------------------------------------+ | Advanced | DEFAULT | StorageManager | storage.cleanup.enabled | true | Enables/disables the storage cleanup thread. | | Advanced | DEFAULT | StorageManager | storage.cleanup.interval | 60 | The interval (in seconds) to wait before running the storage cleanup thread. | +----------+----------+----------------+--------------------------+-------+------------------------------------------------------------------------------+ 2 rows in set (0.00 sec) mysql> select * from configuration where name like "expunge%"; +----------+----------+---------------+------------------+-------+--------------------------------------------------------------------------------------------------------------------------------------------+ | category | instance | component | name | value | description | +----------+----------+---------------+------------------+-------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Advanced | DEFAULT | UserVmManager | expunge.delay | 60 | Determines how long (in seconds) to wait before actually expunging destroyed vm. The default value = the default value of expunge.interval | | Advanced | DEFAULT | UserVmManager | expunge.interval | 60 | The interval (in seconds) to wait before running the expunge thread. | | Advanced | DEFAULT | UserVmManager | expunge.workers | 1 | Number of workers performing expunge | +----------+----------+---------------+------------------+-------+--------------------------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec) **On the Primary Storage** [root@nfs2 primary]# ls 08c2826505b646b3bf3d3e03dfc2bce2-000001-delta.vmdk fba51dce35d24c19a224090f3cb597e8-000002-delta.vmdk ROOT-10-17-delta.vmdk ROOT-6-7-delta.vmdk 08c2826505b646b3bf3d3e03dfc2bce2-000001.vmdk fba51dce35d24c19a224090f3cb597e8-000002.vmdk ROOT-10-17.vmdk ROOT-7-11-000001-delta.vmdk 08c2826505b646b3bf3d3e03dfc2bce2-flat.vmdk fba51dce35d24c19a224090f3cb597e8-flat.vmdk ROOT-11-18-delta.vmdk ROOT-7-11-000001.vmdk 08c2826505b646b3bf3d3e03dfc2bce2.vmdk fba51dce35d24c19a224090f3cb597e8.vmdk ROOT-11-18.vmdk ROOT-7-11-delta.vmdk 3031fabd31b74d45843c9eccf6b61dac-flat.vmdk i-3-12-VMSNAP ROOT-1-1-delta.vmdk ROOT-7-11.vmdk 3031fabd31b74d45843c9eccf6b61dac.vmdk i-3-3-VMSNAP ROOT-1-1.vmdk ROOT-8-12-000001-delta.vmdk 721db558833d4bf59192af3c3d109cd7-flat.vmdk i-3-5-VMSNAP ROOT-12-19-delta.vmdk ROOT-8-12-000001.vmdk 721db558833d4bf59192af3c3d109cd7.vmdk i-3-6-VMSNAP ROOT-12-19.vmdk ROOT-8-12-delta.vmdk 8c26613e6e5f344b928e0530ec4817c3 i-3-7-VMSNAP ROOT-2-2-delta.vmdk ROOT-8-12.vmdk c7adbf5e2509484d8ac4959ae53e6146-000002-delta.vmdk i-3-8-VMSNAP ROOT-2-2.vmdk ROOT-9-14-000002-delta.vmdk c7adbf5e2509484d8ac4959ae53e6146-000002.vmdk i-3-9-VMSNAP ROOT-3-3-delta.vmdk ROOT-9-14-000002.vmdk c7adbf5e2509484d8ac4959ae53e6146-flat.vmdk i-6-10-VMSNAP ROOT-4-5-delta.vmdk s-1-VMSNAP c7adbf5e2509484d8ac4959ae53e6146.vmdk r-11-VMSNAP ROOT-4-5.vmdk v-2-VMSNAP cf04b6e2026f3369b893bdab955c3ff0 r-4-VMSNAP ROOT-5-6-delta.vmdk [root@nfs2 primary]# cd i-3-5-VMSNAP/ i-3-5-VMSNAP.nvram i-3-5-VMSNAP.vmsd i-3-5-VMSNAP.vmx i-3-5-VMSNAP.vmxf vmware-1.log vmware.log [root@nfs2 primary]# cd i-3-5-VMSNAP/ i-3-5-VMSNAP.nvram i-3-5-VMSNAP.vmsd i-3-5-VMSNAP.vmx i-3-5-VMSNAP.vmxf vmware-1.log vmware.log [root@nfs2 primary]# cd i-3-5-VMSNAP/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira