Thanks, it’s a good idea to back up those “removed” disks first before attempting DB surgery!
On 5/4/16, 9:57 PM, "ilya" <ilya.mailing.li...@gmail.com> wrote: >never mind - on the "removed" disks - it deletes well. > >On 5/4/16 9:55 PM, ilya wrote: >> I'm pretty certain cloudstack does not have purging on data disks as i >> had to write my own :) >> >> On 5/4/16 9:51 PM, Ahmad Emneina wrote: >>> I'm not sure if the expunge interval/delay plays a part... but you might >>> want to set: storage.cleanup.enabled to false. That might prevent your >>> disks from being purged. You might also look to export those volumes, or >>> copy them to a safe location, out of band. >>> >>> On Wed, May 4, 2016 at 8:49 PM, Yiping Zhang <yzh...@marketo.com> wrote: >>> >>>> Before I try the direct DB modifications, I would first: >>>> >>>> * shutdown the VM instances >>>> * stop cloudstack-management service >>>> * do a DB backup with mysqldump >>>> >>>> What I worry the most is that the volumes on new cluster’s primary storage >>>> device are marked as “removed”, so if I shutdown the instances, the >>>> cloudstack may kick off a storage cleanup job to remove them from new >>>> cluster’s primary storage before I can get the fixes in. >>>> >>>> Is there a way to temporarily disable storage cleanups ? >>>> >>>> Yiping >>>> >>>> >>>> >>>> >>>> On 5/4/16, 3:22 PM, "Yiping Zhang" <yzh...@marketo.com> wrote: >>>> >>>>> Hi, all: >>>>> >>>>> I am in a situation that I need some help: >>>>> >>>>> I did a live migration with storage migration required for a production >>>> VM instance from one cluster to another. The first migration attempt >>>> failed after some time, but the second attempt succeeded. During all this >>>> time the VM instance is accessible (and it is still up and running). >>>> However, when I use my api script to query volumes, it still reports that >>>> the volume is on the old cluster’s primary storage. If I shut down this >>>> VM, I am afraid that it won’t start again as it would try to use >>>> non-existing volumes. >>>>> >>>>> Checking database, sure enough, the DB still has old info about these >>>> volumes: >>>>> >>>>> >>>>> mysql> select id,name from storage_pool where id=1 or id=8; >>>>> >>>>> +----+------------------+ >>>>> >>>>> | id | name | >>>>> >>>>> +----+------------------+ >>>>> >>>>> | 1 | abprod-primary1 | >>>>> >>>>> | 8 | abprod-p1c2-pri1 | >>>>> >>>>> +----+------------------+ >>>>> >>>>> 2 rows in set (0.01 sec) >>>>> >>>>> >>>>> Here the old cluster’s primary storage has id=1, and the new cluster’s >>>> primary storage has id=8. >>>>> >>>>> >>>>> Here are the entries with wrong info in volumes table: >>>>> >>>>> >>>>> mysql> select id,name, uuid, path,pool_id, removed from volumes where >>>> name='ROOT-97' or name='DATA-97'; >>>>> >>>> >>>>> +-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+ >>>>> >>>>> | id | name | uuid | path >>>> | pool_id | removed | >>>>> >>>> >>>>> +-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+ >>>>> >>>>> | 124 | ROOT-97 | 224bf673-fda8-4ccc-9c30-fd1068aee005 | >>>> 5d1ab4ef-2629-4384-a56a-e2dc1055d032 | 1 | NULL | >>>>> >>>>> | 125 | DATA-97 | d385d635-9230-4130-8d1f-702dbcf0f22c | >>>> 6b75496d-5907-46c3-8836-5618f11dac8e | 1 | NULL | >>>>> >>>>> | 316 | ROOT-97 | 691b5c12-7ec4-408d-b66f-1ff041f149c1 | NULL >>>> | 8 | 2016-05-03 06:10:40 | >>>>> >>>>> | 317 | ROOT-97 | 8ba29fcf-a81a-4ca0-9540-0287230f10c7 | NULL >>>> | 8 | 2016-05-03 06:10:45 | >>>>> >>>> >>>>> +-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+ >>>>> >>>>> 4 rows in set (0.01 sec) >>>>> >>>>> On the xenserver of old cluster, the volumes do not exist: >>>>> >>>>> >>>>> [root@abmpc-hv01 ~]# xe vdi-list name-label='ROOT-97' >>>>> >>>>> [root@abmpc-hv01 ~]# xe vdi-list name-label='DATA-97' >>>>> >>>>> [root@abmpc-hv01 ~]# >>>>> >>>>> But the volumes are on the new cluster’s primary storage: >>>>> >>>>> >>>>> [root@abmpc-hv04 ~]# xe vdi-list name-label=ROOT-97 >>>>> >>>>> uuid ( RO) : a253b217-8cdc-4d4a-a111-e5b6ad48a1d5 >>>>> >>>>> name-label ( RW): ROOT-97 >>>>> >>>>> name-description ( RW): >>>>> >>>>> sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3 >>>>> >>>>> virtual-size ( RO): 34359738368 >>>>> >>>>> sharable ( RO): false >>>>> >>>>> read-only ( RO): true >>>>> >>>>> >>>>> uuid ( RO) : c46b7a61-9e82-4ea1-88ca-692cd4a9204b >>>>> >>>>> name-label ( RW): ROOT-97 >>>>> >>>>> name-description ( RW): >>>>> >>>>> sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3 >>>>> >>>>> virtual-size ( RO): 34359738368 >>>>> >>>>> sharable ( RO): false >>>>> >>>>> read-only ( RO): false >>>>> >>>>> >>>>> [root@abmpc-hv04 ~]# xe vdi-list name-label=DATA-97 >>>>> >>>>> uuid ( RO) : bc868e3d-b3c0-4c6a-a6fc-910bc4dd1722 >>>>> >>>>> name-label ( RW): DATA-97 >>>>> >>>>> name-description ( RW): >>>>> >>>>> sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3 >>>>> >>>>> virtual-size ( RO): 107374182400 >>>>> >>>>> sharable ( RO): false >>>>> >>>>> read-only ( RO): false >>>>> >>>>> >>>>> uuid ( RO) : a8c187cc-2ba0-4928-8acf-2afc012c036c >>>>> >>>>> name-label ( RW): DATA-97 >>>>> >>>>> name-description ( RW): >>>>> >>>>> sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3 >>>>> >>>>> virtual-size ( RO): 107374182400 >>>>> >>>>> sharable ( RO): false >>>>> >>>>> read-only ( RO): true >>>>> >>>>> >>>>> Following is how I plan to fix the corrupted DB entries. Note: using uuid >>>> of VDI volume with read/write access as the path values: >>>>> >>>>> >>>>> 1. for ROOT-97 volume: >>>>> >>>>> Update volumes set removed=NOW() where id=124; >>>>> Update volumes set removed=NULL where id=317; >>>>> Update volumes set path=c46b7a61-9e82-4ea1-88ca-692cd4a9204b where id=317; >>>>> >>>>> >>>>> 2) for DATA-97 volume: >>>>> >>>>> Update volumes set pool_id=8 where id=125; >>>>> >>>>> Update volumes set path=bc868e3d-b3c0-4c6a-a6fc-910bc4dd1722 where id=125; >>>>> >>>>> >>>>> Would this work? >>>>> >>>>> >>>>> Thanks for all the helps anyone can provide. I have a total of 4 VM >>>> instances with 8 volumes in this situation need to be fixed. >>>>> >>>>> >>>>> Yiping >>>> >>>
