Hi This could be related to https://issues.apache.org/jira/browse/VCL-668
On Jan 24, 2013, at 15:41 , "Basilio, Norvin" <[email protected]> wrote: > Hello, > > I’ve recently upgraded the ESXi host from 4.1 to 5.1 and at first all seemed > as though it was working fine. Now I’ve found that I cannot capture or update > images. It seems to fail when copying the vmdk files to the Datastore > location. I’ve monitored the copy process and found that according the esxi > task’s bar the copy shows completed. When the code goes to copy from > Datastore to the Repository location the folder is no longer in the > Datastore. I’ve attached the log that is emailed when the error occurs. If > you have any pointers on where to look it would be appreciated. > > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 image creation failed > ------------------------------------------------------------------------ > time: 2013-01-24 08:45:04 > caller: image.pm:reservation_failed(387) ( 0) image.pm, reservation_failed > (line: 387) > (-1) image.pm, process (line: 167) > (-2) vcld, make_new_child (line: 571) > (-3) vcld, main (line: 350) > ------------------------------------------------------------------------ > management node: portman.vcls.priv.odu.edu reservation PID: 10161 parent vcld > PID: 3878 > > request ID: 50 > reservation ID: 50 > request state/laststate: image/image > request start time: 2013-01-24 08:38:50 > request end time: 2013-01-24 09:38:50 > for imaging: no > log ID: none > > computer: vcl480 > computer id: 2 > computer type: virtualmachine > computer eth0 MAC address: 00:50:56:00:03:be computer eth1 MAC address: > 00:50:56:00:03:bf computer private IP address: 192.168.131.240 computer > public IP address: 128.82.130.43 computer in block allocation: no > provisioning module: VCL::Module::Provisioning::VMware::VMware > > vm host: hepburn.vcls.priv.odu.edu > vm host ID: 1 > vm host computer ID: 1 > vm profile: ODU VNX NFS Storage > vm profile VM path: [VMPATH] > vm profile repository path: [REPOSITORY] vm profile datastore path: > [DATASTORE] vm profile disk type: shared > > image: vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 > image display name: Redhat Enterprise 5 Base (RHEL5x64) image ID: 10 image > revision ID: 10 image size: 1450 MB use Sysprep: yes root access: yes image > owner ID: 1 image owner affiliation: Local image revision date created: > 2013-01-24 08:39:00 image revision production: yes OS module: > VCL::Module::OS::Linux > > user: admin > user name: vcl admin > user ID: 1 > user affiliation: Local > ------------------------------------------------------------------------ > RECENT LOG ENTRIES FOR THIS PROCESS: > 2013-01-24 08:44:57|10161|50:50|image|VMware.pm:capture(770)|vmdk will be > copied directly from VM host hepburn to the image repository in the 2gbsparse > disk format > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2549)|searching for > matching file paths: base directory path: '[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0', search pattern: > 'vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2604)|retrieved info > for 1 matching files: > |10161|50:50|image| : { > |10161|50:50|image| : "[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk" > => bless( { > |10161|50:50|image| : "capacityKb" => 31457280, > |10161|50:50|image| : "controllerType" => "VirtualLsiLogicController", > |10161|50:50|image| : "diskExtents" => [ > |10161|50:50|image| : "[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0-flat.vmdk" > |10161|50:50|image| : ], > |10161|50:50|image| : "diskType" => "VirtualDiskFlatVer2BackingInfo", > |10161|50:50|image| : "fileSize" => "3161653248", > |10161|50:50|image| : "hardwareVersion" => 8, > |10161|50:50|image| : "modification" => "2013-01-24T13:44:56Z", > |10161|50:50|image| : "owner" => "root", > |10161|50:50|image| : "path" => > "vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk", > |10161|50:50|image| : "thin" => 1, > |10161|50:50|image| : "type" => "VmDiskFileInfo" > |10161|50:50|image| : }, 'VmDiskFileInfo' ) > |10161|50:50|image| : } > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:find_files(2149)|matching file > count: 2 > 2013-01-24 08:44:58|10161|50:50|image|vSphere_SDK.pm:file_exists(2017)|file > exists: [DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2549)|searching for > matching file paths: base directory path: '[REPOSITORY] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0', search pattern: > 'vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2560)|base directory > does not exist: '[REPOSITORY] vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0' > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:find_files(2149)|matching file > count: 0 > 2013-01-24 08:44:58|10161|50:50|image|vSphere_SDK.pm:file_exists(2021)|file > does not exist: [REPOSITORY] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2549)|searching for > matching file paths: base directory path: '[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0', search pattern: > 'vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > 2013-01-24 > 08:44:58|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2604)|retrieved info > for 1 matching files: > |10161|50:50|image| : { > |10161|50:50|image| : "[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk" > => bless( { > |10161|50:50|image| : "capacityKb" => 31457280, > |10161|50:50|image| : "controllerType" => "VirtualLsiLogicController", > |10161|50:50|image| : "diskExtents" => [ > |10161|50:50|image| : "[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0-flat.vmdk" > |10161|50:50|image| : ], > |10161|50:50|image| : "diskType" => "VirtualDiskFlatVer2BackingInfo", > |10161|50:50|image| : "fileSize" => "3161653248", > |10161|50:50|image| : "hardwareVersion" => 8, > |10161|50:50|image| : "modification" => "2013-01-24T13:44:56Z", > |10161|50:50|image| : "owner" => "root", > |10161|50:50|image| : "path" => > "vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk", > |10161|50:50|image| : "thin" => 1, > |10161|50:50|image| : "type" => "VmDiskFileInfo" > |10161|50:50|image| : }, 'VmDiskFileInfo' ) > |10161|50:50|image| : } > 2013-01-24 > 08:44:59|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2549)|searching for > matching file paths: base directory path: '[REPOSITORY]', search pattern: > 'vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0' > 2013-01-24 > 08:44:59|10161|50:50|image|vSphere_SDK.pm:_get_file_info(2604)|retrieved info > for 0 matching files: > |10161|50:50|image| : {} > 2013-01-24 > 08:44:59|10161|50:50|image|vSphere_SDK.pm:find_files(2149)|matching file > count: 0 > 2013-01-24 08:44:59|10161|50:50|image|vSphere_SDK.pm:file_exists(2021)|file > does not exist: [REPOSITORY] vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 > 2013-01-24 > 08:44:59|10161|50:50|image|vSphere_SDK.pm:create_directory(1575)|created > directory: '[REPOSITORY] vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0' > 2013-01-24 > 08:44:59|10161|50:50|image|vSphere_SDK.pm:copy_virtual_disk(684)|attempting > to copy virtual disk on VM host hepburn.vcls.priv.odu.edu: '[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > --> '[REPOSITORY] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > |10161|50:50|image| source adapter type: VirtualLsiLogicController > |10161|50:50|image| destination adapter type: ide 50:50|image| disk > |10161|type: VirtualDiskFlatVer2BackingInfo 50:50|image| source > |10161|capacity: 32,212,254,720 bytes - 31,457,280.0 KB - 30,720.0 MB - > |10161|30.00 GB 50:50|image| source space used: 3,161,653,248 bytes - > |10161|3,087,552.0 KB - 3,015.2 MB - 2.94 GB 50:50|image| ---- WARNING > |10161|---- 50:50|image| 2013-01-24 > 08:45:01|10161|50:50|image|vSphere_SDK.pm:copy_virtual_disk(731)|failed to > copy vmdk on VM host hepburn.vcls.priv.odu.edu using CopyVirtualDisk > function: '[DATASTORE] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > --> '[REPOSITORY] > vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > |10161|50:50|image| error: > |10161|50:50|image| SOAP Fault: > |10161|50:50|image| ----------- > |10161|50:50|image| Fault string: File [DATASTORE] > |10161|vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-Redhat > |10161|Enterprise5BaseRHEL5x6410-v0.vmdk was not found 50:50|image| > |10161|Fault detail: FileNotFound 50:50|image| ( 0) vSphere_SDK.pm, > |10161|copy_virtual_disk (line: 731) 50:50|image| (-1) VMware.pm, > |10161|copy_vmdk (line: 5813) 50:50|image| (-2) VMware.pm, capture > |10161|(line: 774) 50:50|image| (-3) image.pm, process (line: 162) > |10161|50:50|image| (-4) vcld, make_new_child (line: 571) 50:50|image| > |10161|(-5) vcld, main (line: 350) 50:50|image| ---- WARNING ---- > |10161|50:50|image| 2013-01-24 > |10161|08:45:01|10161|50:50|image|VMware.pm:copy_vmdk(5828)|failed to > |10161|copy vmdk using API's copy_virtual_disk subroutine 50:50|image| ( > |10161|0) VMware.pm, copy_vmdk (line: 5828) 50:50|image| (-1) VMware.pm, > |10161|capture (line: 774) 50:50|image| (-2) image.pm, process (line: > |10161|162) 50:50|image| (-3) vcld, make_new_child (line: 571) > |10161|50:50|image| (-4) vcld, main (line: 350) 50:50|image| ---- > |10161|WARNING ---- 50:50|image| 2013-01-24 > 08:45:01|10161|50:50|image|VMware.pm:capture(778)|failed to copy the vmdk > files to the repository mounted on the VM host after the VM was powered off: > '/vmfs/volumes/DATASTORE/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > --> > '/vmfs/volumes/REPOSITORY/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0.vmdk' > |10161|50:50|image| ( 0) VMware.pm, capture (line: 778) 50:50|image| > |10161|(-1) image.pm, process (line: 162) 50:50|image| (-2) vcld, > |10161|make_new_child (line: 571) 50:50|image| (-3) vcld, main (line: > |10161|350) > 2013-01-24 08:45:01|10161|50:50|image|VMware.pm:capture(870)|attempting to > delete directory where moved vmdk resided before reverting the name back to > the original: > /vmfs/volumes/DATASTORE/vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 > 2013-01-24 > 08:45:01|10161|50:50|image|vSphere_SDK.pm:delete_file(1625)|attempting to > delete file: [DATASTORE] vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 > 2013-01-24 > 08:45:01|10161|50:50|image|vSphere_SDK.pm:delete_file(1638)|deleted file: > [DATASTORE] vmwarelinux-RedhatEnterprise5BaseRHEL5x6410-v0 > 2013-01-24 08:45:01|10161|50:50|image|VMware.pm:capture(876)|attempting to > power the VM back on so that it can be captured again > 2013-01-24 08:45:04|10161|50:50|image|vSphere_SDK.pm:vm_power_on(378)|powered > on VM: [TESTVNX] rhel5/rhel5.vmx > |10161|50:50|image| ---- WARNING ---- > |10161|50:50|image| 2013-01-24 > |10161|08:45:04|10161|50:50|image|image.pm:process(166)|vmwarelinux-Redh > |10161|atEnterprise5BaseRHEL5x6410-v0 image failed to be captured by > |10161|provisioning module 50:50|image| ( 0) image.pm, process (line: > |10161|166) 50:50|image| (-1) vcld, make_new_child (line: 571) > |10161|50:50|image| (-2) vcld, main (line: 350) > 2013-01-24 > 08:45:04|10161|50:50|image|utils.pm:get_management_node_info(5489)|retrieving > current management node info for 'portman.vcls.priv.odu.edu' from database, > cached data is stale: 369 seconds old > 2013-01-24 > 08:45:04|10161|50:50|image|utils.pm:get_management_node_info(5636)|retrieved > management node info: 'portman.vcls.priv.odu.edu' (portman) > 2013-01-24 > 08:45:04|10161|50:50|image|DataStructure.pm:get_computer_private_ip_address(1606)|returning > private IP address previously retrieved: 192.168.131.240 > 2013-01-24 08:45:04|10161|50:50|image|utils.pm:is_inblockrequest(5793)|zero > rows were returned from database select > 2013-01-24 > 08:45:04|10161|50:50|image|DataStructure.pm:get_image_affiliation_name(2118)|image > owner id: 1 > 2013-01-24 > 08:45:04|10161|50:50|image|DataStructure.pm:retrieve_user_data(1401)|attempting > to retrieve and store data for user: user.id = '1' > 2013-01-24 > 08:45:04|10161|50:50|image|DataStructure.pm:retrieve_user_data(1464)|data has > been retrieved for user: admin (id: 1) > > > Norvin Basilio > Email: [email protected] > > -- Thank you, Dmitri Chebotarov VCL Sys Eng, Engineering & Architectural Support, TSD - Ent Servers & Messaging 223 Aquia Building, Ffx, MSN: 1B5 Phone: (703) 993-6175 | Fax: (703) 993-3404
