leduyquy opened a new issue, #11072: URL: https://github.com/apache/cloudstack/issues/11072
### problem In new version Cloudstack: 4.20.1.0, I see it support VMware 8U3 so I did upgrade vCenter to 8.0.3.00500, But affter upgrade vCenter i testing attach disk to VM running on it. everything is normal when i rollback vcenter to old version:  Log on management: 2025-06-23 09:50:37,646 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:167cef34) Add job-132862 into job monitoring 2025-06-23 09:50:37,649 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (qtp2038105753-51180:[ctx-356feba0, ctx-fb5487f9]) (logid:0d6e0b53) submit async job-132862, details: AsyncJob {"accountId":10,"cmd":"org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd","cmdInfo":"{\"virtualmachineid\":\"5007e13f-28e0-4f9e-b191-428c381c9288\",\"response\":\"json\",\"ctxUserId\":\"60\",\"sessionkey\":\"aZTCm6g7HBWrL9uH08ocEqYxnl8\",\"httpmethod\":\"GET\",\"ctxStartEventId\":\"1265802\",\"id\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"ctxDetails\":\"{\\\"interface com.cloud.storage.Volume\\\":\\\"d4a8082f-d4e4-415f-878e-667cba85d103\\\",\\\"interface com.cloud.vm.VirtualMachine\\\":\\\"5007e13f-28e0-4f9e-b191-428c381c9288\\\"}\",\"ctxAccountId\":\"10\",\"uuid\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"cmdEventType\":\"VOLUME.ATTACH\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":132862,"initMsid":345050445235,"instanceId":4274,"instanceType":"Volume","lastPolled":null,"lastUpdat ed":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":60,"uuid":"0654201d-f27e-4b7d-8d64-0a067bc74a80"} 2025-06-23 09:50:37,650 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Executing AsyncJob {"accountId":10,"cmd":"org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd","cmdInfo":"{\"virtualmachineid\":\"5007e13f-28e0-4f9e-b191-428c381c9288\",\"response\":\"json\",\"ctxUserId\":\"60\",\"sessionkey\":\"aZTCm6g7HBWrL9uH08ocEqYxnl8\",\"httpmethod\":\"GET\",\"ctxStartEventId\":\"1265802\",\"id\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"ctxDetails\":\"{\\\"interface com.cloud.storage.Volume\\\":\\\"d4a8082f-d4e4-415f-878e-667cba85d103\\\",\\\"interface com.cloud.vm.VirtualMachine\\\":\\\"5007e13f-28e0-4f9e-b191-428c381c9288\\\"}\",\"ctxAccountId\":\"10\",\"uuid\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"cmdEventType\":\"VOLUME.ATTACH\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":132862,"initMsid":345050445235,"instanceId":4274,"instanceType":"Volume","lastPolled":null,"lastUpdated":null,"processStatus" :0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":60,"uuid":"0654201d-f27e-4b7d-8d64-0a067bc74a80"} 2025-06-23 09:50:37,658 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Access to Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}] granted to Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}] by DomainChecker on behalf of user <<user-name>> 2025-06-23 09:50:37,662 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Access to VM {"id":1544,"instanceName":"i-10-1544-VM","type":"User","uuid":"5007e13f-28e0-4f9e-b191-428c381c9288"} granted to Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}] by DomainChecker on behalf of user <<user-name>> 2025-06-23 09:50:37,664 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Account [Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}]] has access to resource. 2025-06-23 09:50:37,690 DEBUG [c.c.h.d.HypervisorCapabilitiesDaoImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Hypervisor capabilities for hypervisor: VMware, version: 8.0.2.0 can not be found. Trying to find capabilities for the parent version: 8.0.2 2025-06-23 09:50:37,696 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Access to VolumeObject {"dataStore":"StoragePool {\"id\":2,\"name\":\"vsanDatastore\",\"poolType\":\"PreSetup\",\"uuid\":\"7baf972d-35f4-394c-b9c3-0c3e13b4144a\"}","volumeVO":"Volume {\"id\":4274,\"instanceId\":null,\"name\":\"data\",\"uuid\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"volumeType\":\"DATADISK\"}"} granted to Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}] by DomainChecker on behalf of user <<user-name>> 2025-06-23 09:50:37,697 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Access to VM instance {"id":1544,"instanceName":"i-10-1544-VM","state":"Running","type":"User","uuid":"5007e13f-28e0-4f9e-b191-428c381c9288"} granted to Account [{"accountName":"<<account-name>>","id":10,"uuid":"e653412e-93cf-472e-a96b-8a0efd8924be"}] by DomainChecker on behalf of user <<user-name>> 2025-06-23 09:50:37,703 INFO [c.c.s.VolumeApiServiceImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Trying to attach volume [VolumeObject {"dataStore":"StoragePool {\"id\":2,\"name\":\"vsanDatastore\",\"poolType\":\"PreSetup\",\"uuid\":\"7baf972d-35f4-394c-b9c3-0c3e13b4144a\"}","volumeVO":"Volume {\"id\":4274,\"instanceId\":null,\"name\":\"data\",\"uuid\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"volumeType\":\"DATADISK\"}"}] to VM instance [VM instance {"id":1544,"instanceName":"i-10-1544-VM","state":"Running","type":"User","uuid":"5007e13f-28e0-4f9e-b191-428c381c9288"}], update async job-132862 [AsyncJob {"accountId":10,"cmd":"org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd","cmdInfo":"{\"virtualmachineid\":\"5007e13f-28e0-4f9e-b191-428c381c9288\",\"response\":\"json\",\"ctxUserId\":\"60\",\"sessionkey\":\"aZTCm6g7HBWrL9uH08ocEqYxnl8\",\"httpmethod\":\"GET\",\"ctxStartEventId\":\"1265802\",\"id\":\"d4a8082f-d4e4-415f-878e-667 cba85d103\",\"ctxDetails\":\"{\\\"interface com.cloud.storage.Volume\\\":\\\"d4a8082f-d4e4-415f-878e-667cba85d103\\\",\\\"interface com.cloud.vm.VirtualMachine\\\":\\\"5007e13f-28e0-4f9e-b191-428c381c9288\\\"}\",\"ctxAccountId\":\"10\",\"uuid\":\"d4a8082f-d4e4-415f-878e-667cba85d103\",\"cmdEventType\":\"VOLUME.ATTACH\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":132862,"initMsid":345050445235,"instanceId":4274,"instanceType":"Volume","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":60,"uuid":"0654201d-f27e-4b7d-8d64-0a067bc74a80"}] progress status 2025-06-23 09:50:37,705 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Update async-job attachment, job-132862, instanceType: Volume, instanceId: 4274 2025-06-23 09:50:37,707 DEBUG [c.c.a.ApiServer] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Retrieved cmdEventType from job info: VOLUME.ATTACH 2025-06-23 09:50:37,733 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Sync job-132863 execution on object VmWorkJobQueue.1544 2025-06-23 09:50:37,762 DEBUG [c.c.s.VolumeApiServiceImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) New job VmWorkJobVO : {, step: Starting, vmType: Instance, vmInstanceId: 1544, secondaryObjectIdentifier: nullAsyncJob {"accountId":10,"cmd":"com.cloud.vm.VmWorkAttachVolume","cmdInfo":"rO0ABXNyAB9jb20uY2xvdWQudm0uVm1Xb3JrQXR0YWNoVm9sdW1lB62v-WGH4hwCAAJMAAhkZXZpY2VJZHQAEExqYXZhL2xhbmcvTG9uZztMAAh2b2x1bWVJZHEAfgABeHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAAoAAAAAAAAAPAAAAAAAAAYIdAAUVm9sdW1lQXBpU2VydmljZUltcGxwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAABCy","cmdVersion":0,"completeMsid":null,"created":null,"id":132863,"initMsid":345050445235,"instanceId":null,"instanceType":null,"lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROG RESS","userId":60,"uuid":"8d75d03d-44d6-463a-9150-94f7f6589203"}}, result field: null 2025-06-23 09:50:40,786 WARN [o.a.c.m.w.WebhookServiceImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Skipping delivering event Event {"description":"{\"details\":\"Volume Id: d4a8082f-d4e4-415f-878e-667cba85d103 VmId: 5007e13f-28e0-4f9e-b191-428c381c9288\",\"event\":\"VOLUME.ATTACH\",\"status\":\"Completed\"}","eventId":null,"eventType":"VOLUME.ATTACH","eventUuid":null,"resourceType":"Volume","resourceUUID":null} to any webhook as account ID is missing 2025-06-23 09:50:40,787 WARN [o.a.c.f.e.EventDistributorImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862, ctx-1557a9bc]) (logid:0654201d) Failed to publish event [category: ActionEvent, type: VOLUME.ATTACH] on bus webhookEventBus 2025-06-23 09:50:40,802 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Unexpected exception while executing org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd com.cloud.utils.exception.CloudRuntimeException: Failed to attach volume data to VM <<vm-namee>> 2025-06-23 09:50:40,803 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Complete async job-132862, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to attach volume data to VM <<vm-namee>>"} 2025-06-23 09:50:40,803 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Publish async job-132862 complete on message bus 2025-06-23 09:50:40,804 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Wake up jobs related to job-132862 2025-06-23 09:50:40,804 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Update db status for job-132862 2025-06-23 09:50:40,805 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Wake up jobs joined with job-132862 and disjoin all subjobs created from job- 132862 2025-06-23 09:50:40,813 DEBUG [c.c.a.ApiServer] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Retrieved cmdEventType from job info: VOLUME.ATTACH 2025-06-23 09:50:40,817 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Done executing org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd for job-132862 2025-06-23 09:50:40,817 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-51:[ctx-9935ac14, job-132862]) (logid:0654201d) Remove job-132862 from job monitoring Since i had to rollback to vcenter to secure workload i can only provide more information from old log ### versions Cloudstack: 4.20.1.0 vCenter: 8.0.3.00500, Build number: 24674346 VMware ESXi : 8.0.2, 22380479 ### The steps to reproduce the bug 1. Upgrade vCenter from 8.0.0 to 8.0.3 2. Attach addon disk VM 3. Error ... ### What to do about it? Attach addon disk working normal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
