The fact you didn't find all the references in database is OK - XenServer can "spawn" extra files by itself and rename new UUID-s to old ones in order to keep VHD UUIDs consistent. This is also described in an article I have sent you before.

I would only trust the output of "vhd-util". It seems XenServer can't coalesce 1a240d45[VHD] for some reason. Can it be hardware error? Have you tried to read content of this LV like dd if=/path/to/LV of=/dev/null ?

Second question - are you sure coalesce is done for correct UUIDs? You can check the chain by vhd-util manually. Like here: http://discussions.citrix.com/topic/236371-show-wich-vhd-files-are-linked-together



Vadim.


On 2015-12-10 13:40, France wrote:

Vladim, thank you again for follow up.

The references below, found or not found are to CS MySql database.
All the files are actually there, they are LV devices in CLVM, I just did not find any references in CS database for their UUIDs.

The corrupt one is:
/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba

I know that is corrupt, because there are lots of entries in SMlog for it like this: [8756] 2015-12-07 21:50:45.118758 ['/usr/sbin/vhd-util', 'check', '--debug', '-n', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba'] [8756] 2015-12-07 21:50:45.131975 FAILED: (rc 22) stdout: 'primary footer invalid: invalid cookie /dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba appears invalid; dumping metadata
VHD Footer Summary:
-------------------
Cookie : conectix
Features : (0x00000002) <RESV>
File format version : Major: 1, Minor: 0
Data offset : 512
Timestamp : Mon Dec 7 18:19:46 2015
Creator Application : 'tap'
Creator version : Major: 1, Minor: 3
Creator OS : Unknown!
Original disk size : 20480 MB (21474836480 Bytes)
Current disk size : 20480 MB (21474836480 Bytes)
Geometry : Cyl: 41610, Hds: 16, Sctrs: 63
: = 20479 MB (21474754560 Bytes)
Disk type : Differencing hard disk
Checksum : 0xffffef85|0xffffef85 (Good!)
UUID : 8f25299d-d7cc-44fe-a42b-95b4e9a31a47
Saved state : No
Hidden : 1

VHD Header Summary:
-------------------
Cookie : cxsparse
Data offset (unusd) : 18446744073709
Table offset : 1536
Header version : 0x00010000
Max BAT size : 1048576
Block size : 2097152 (2 MB)
Parent name : VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
Parent UUID : 8cece4c9-8c94-47b9-9f7a-cb6023da9b72
Parent timestamp : Mon Dec 7 18:19:46 2015
Checksum : 0xffffc6ed|0xffffc6ed (Good!)

VHD Parent Locators:
--------------------
locator: : 0
code : PLAT_CODE_MACX
data_space : 512
data_length : 110
data_offset : 4327424
decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

locator: : 1
code : PLAT_CODE_W2KU
data_space : 512
data_length : 206
data_offset : 4327936
decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

locator: : 2
code : PLAT_CODE_W2RU
data_space : 512
data_length : 206
data_offset : 4328448
decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

VHD Batmap Summary:
-------------------
Batmap offset : 4196352
Batmap size (secs) : 256
Batmap version : 0x00010002
Checksum : 0xffffff7f|0xffffff7f (Good!)

', stderr: ''
[8756] 2015-12-07 21:50:45.132293 ['/usr/sbin/vhd-util', 'query', '--debug', '-s', '-n', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba']
[8756] 2015-12-07 21:50:45.145222 SUCCESS
[8756] 2015-12-07 21:50:45.145502 lock: tried lock /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr, acquired: True (exists: True) [8756] 2015-12-07 21:50:45.145635 lock: released /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr [8756] 2015-12-07 21:50:45.145752 ['/usr/sbin/lvchange', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba', '-p', 'r']
..
..
..
<8756> 2015-12-07 21:50:45.229864 *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
<8756> 2015-12-07 21:50:45.229919 ***********************
<8756> 2015-12-07 21:50:45.229969 * E X C E P T I O N *
<8756> 2015-12-07 21:50:45.230017 ***********************
<8756> 2015-12-07 21:50:45.230077 coalesce: EXCEPTION util.SMException, VHD *1a240d45[VHD](20.000G//136.000M|n) corrupted <8756> 2015-12-07 21:50:45.230129 File "/opt/xensource/sm/cleanup.py", line 1397, in coalesce
self._coalesce(vdi)
File "/opt/xensource/sm/cleanup.py", line 1587, in _coalesce
vdi._doCoalesce()
File "/opt/xensource/sm/cleanup.py", line 1050, in _doCoalesce
self.parent.validate()
File "/opt/xensource/sm/cleanup.py", line 1043, in validate
VDI.validate(self, fast)
File "/opt/xensource/sm/cleanup.py", line 633, in validate
raise util.SMException("VHD %s corrupted" % self)

<8756> 2015-12-07 21:50:45.230181 *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
<8756> 2015-12-07 21:50:45.230234 Coalesce failed, skipping

Regards,
F.

On 10 Dec 2015, at 08:22, Vadim Kimlaychuk <va...@kickcloud.net> wrote:

Dear France,

It is more than 6 months I have used vhd-util, and xe to clean-up manually, but I remember that if you try to destroy VDI that has a child - you'll get an error. Since you have all the chain from the parent you should start from the last child and move to the top. You shouldn't be able to destroy VDI in the middle of the chain, since it will definitely corrupt data.

I didn't understand how exactly did you check existing VHD files? How do you know it is corrupted? Does vhd-util query shows that? What does it mean - "corrupted and not found". If it is not found, how do you know it is corrupted?

Try to work with vhd-util -- it is great. I think you can resolve all the issues with it.

Vadim.

On 2015-12-09 20:39, France wrote:

Thank you Vladim, for your response.
Ity is greatly appreciated.
I will surely read the document.
So If I understand correctly, I can try to remove it and if it would be bad for my setup, xe will not do it? :-)
I guess coalesce will not work, if one VHD in chain is corrupted.
I guess I will have to try and repair it first.
We will see. :-)
I vageuly remember that thin provisioning was not possible with CLVM over ISCSI at the time of cluster setup, so I guess we do not have it.
Regards,
F.
On 09 Dec 2015, at 18:18, Vadim Kimlaychuk <va...@kickcloud.net> wrote:
Dear France,
Hope this article helps you:http://support.citrix.com/filedownload/CTX122978/XenServer_Understanding_Snapshots.pdf [1] [1 [1]] Common practice is to use "vhd-util" to merge (coalesce) VHD files into usable image. Luckily xe tool is smart enough not to allow you destroy image that is a part of the chain. Of course it is more safe to copy all the files before destruction (in a case you have thin provisioning).
Vadim.
On 2015-12-09 18:27, France wrote:
Hi guys,
below is the chain cross referenced with CS MySql database.
Can I delete/destroy 1a240d45-ee0a-4c30-809b-3114dfaf85ba without bad consequences? Are next in chain dependant upon it?
*555b38bf[VHD](20.000G//2.695G|n) -> not foud
5afd5849[VHD](20.000G//8.000M|n) -> found in template_spool_ref. status ready, template ID 292
*02e8b56b[VHD](20.000G//12.000M|n) -> not found
*1a240d45[VHD](20.000G//136.000M|n) <- corrupted and not found
*c99e711b[VHD](20.000G//136.000M|n) -> not found
34a03c9a[VHD](20.000G//8.000M|a) -> found in shapshot_store_ref. state destroyed, volume id 661, from template id 292 5c12db0a[VHD](20.000G//8.000M|n) -> found in shapshot_store_ref. state destroyed -> probably not part of the above chain?
I would do xe vdi-destroy=1a240d45-ee0a-4c30-809b-3114dfaf85ba .
Alternatively I think about exporting template ID 292 in CS, deleting it and importing it back again. It should remove the whole chain, right? 'Cause noone is using it?
Regards,
F.

Links:
------
[1] http://support.citrix.com/filedownload/CTX122978/XenServer_Understanding_Snapshots.pdf [1]



Links:
------
[1] http://support.citrix.com/filedownload/CTX122978/XenServer_Understanding_Snapshots.pdf

Reply via email to