This seems to be a bug. Please file a github issue. If the template is still used by active vms, you can find the template on primary storage.
-Wei On Tue, 20 Dec 2022 at 07:50, Granwille Strauss <granwi...@namhost.com.invalid> wrote: > Hi > > I can confirm it is indeed deleted. So in short, the error that said > something in the lines of You cannot delete template because there are > active VMs associated with it, means nothing. It still deleted it anyway. I > believe this needs to be looked at. > On 12/20/22 08:45, jordan j wrote: > > Here is the issue, ACS cannot find the template volume to copy - > d2c4feca-da5c-44b3-8ecc-40a01fd6c243 > > 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) > (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: > 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux > 8 cPanel > Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 > GB) > 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 > GB) > 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] > } > 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] > (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for > 141 seconds > 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] > (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for > 201 seconds > 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] > (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for > 261 seconds > 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) > (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: > 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { > CopyCmdAnswer } } > 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] > (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) > (logid:4c740aa8) Unable to create > Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't > find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243 > > > If other templates can be used from the same SR this means it is > accessible but the template file is not there anymore. > To look for it you can login on the secondary storage and look at a path > similar to this <mount dir>/../templates. > inside should be 1 folder for each template, it will include XML meta file > and volume file. The volume, if exists should be named by the ID > d2c4feca-da5c-44b3-8ecc-40a01fd6c243. > > if no such is present than indeed it is deleted. > > Regards, > Jordan > > On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <granwi...@namhost.com> > wrote: > >> Hi Jordan >> >> Please see log attached. Can you confirm what's the best way to confirm >> if a template exists on secondary storage? >> On 12/20/22 08:15, jordan j wrote: >> >> Hey Granwille Strauss, >> >> Can you upload a full log somewhere? >> This error usually means there is another underlying issue even though it >> says Network/interface scope. >> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid: >> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface >> com.cloud.dc.DataCenter; id=16 >> >> Can you also verify the template file is intact on the secondary server? >> >> Regards, >> Jordan >> >> On Tue, Dec 20, 2022 at 8:09 AM Granwille >> Strauss<granwi...@namhost.com.invalid> <granwi...@namhost.com.invalid> wrote: >> >> >> Hi Guys >> >> I accidentality "deleted" a wrong template. Fortunately, the delete was >> not successful and gave an error stating that it cannot be deleted because >> there are active VMs associated with it. However, since I did this, I can >> now no longer deploy new VMs using said template. Creating a VM now fails >> since I did this, see attached screenshot. >> >> And this is the management log entry: >> >> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy] >> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) >> (logid:4c740aa8) Rethrow exception >> com.cloud.exception.InsufficientServerCapacityException: Unable to create a >> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid: >> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface >> com.cloud.dc.DataCenter; id=16 >> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher] >> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done >> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin: >> 2783 >> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher] >> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) >> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2, >> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, >> cmdInfo: >> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, >> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated: >> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed: >> null}, job origin:2783 >> com.cloud.exception.InsufficientServerCapacityException: Unable to create >> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid: >> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface >> com.cloud.dc.DataCenter; id=16 >> >> This seems to ONLY happen with this one template that I deleted, I can >> deploy VMs successfully from other templates but this one that was >> "deleted", even though it was NOT deleted. So it seems the template broke >> somehow because now no volume is created for new VMs when using this >> template. I am using 4.17.1.0. Is this a known issue? >> -- >> Regards / Groete >> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // >> Senior Systems Admin >> >> *e:* granwi...@namhost.com >> *m:* +264 81 323 1260 <+264813231260> >> *w:* www.namhost.com >> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> >> <https://twitter.com/namhost> >> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> >> >> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> >> >> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> >> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> >> >> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> >> >> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> >> >> Namhost Internet Services (Pty) Ltd, >> >> 24 Black Eagle Rd, Hermanus, 7210, RSA >> >> >> >> The content of this message is confidential. If you have received it by >> mistake, please inform us by email reply and then delete the message. It is >> forbidden to copy, forward, or in any way reveal the contents of this >> message to anyone without our explicit consent. The integrity and security >> of this email cannot be guaranteed over the Internet. Therefore, the sender >> will not be held liable for any damage caused by the message. For our full >> privacy policy and disclaimers, please go >> tohttps://www.namhost.com/privacy-policy >> >> [image: Powered by >> AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> >> >> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> >> >> -- >> Regards / Groete >> >> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin >> >> *e:* granwi...@namhost.com >> *m:* +264 81 323 1260 <+264813231260> >> *w:* www.namhost.com >> >> <https://www.facebook.com/namhost> <https://twitter.com/namhost> >> <https://www.instagram.com/namhostinternetservices/> >> <https://www.linkedin.com/company/namhos> >> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> >> >> >> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> >> >> Namhost Internet Services (Pty) Ltd, >> >> 24 Black Eagle Rd, Hermanus, 7210, RSA >> >> >> >> The content of this message is confidential. If you have received it by >> mistake, please inform us by email reply and then delete the message. It is >> forbidden to copy, forward, or in any way reveal the contents of this >> message to anyone without our explicit consent. The integrity and security >> of this email cannot be guaranteed over the Internet. Therefore, the sender >> will not be held liable for any damage caused by the message. For our full >> privacy policy and disclaimers, please go to >> https://www.namhost.com/privacy-policy >> >> [image: Powered by AdSigner] >> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> >> > -- > Regards / Groete > > <https://www.namhost.com> Granwille Strauss // Senior Systems Admin > > *e:* granwi...@namhost.com > *m:* +264 81 323 1260 <+264813231260> > *w:* www.namhost.com > > <https://www.facebook.com/namhost> <https://twitter.com/namhost> > <https://www.instagram.com/namhostinternetservices/> > <https://www.linkedin.com/company/namhos> > <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > > > <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> > > Namhost Internet Services (Pty) Ltd, > > 24 Black Eagle Rd, Hermanus, 7210, RSA > > > > The content of this message is confidential. If you have received it by > mistake, please inform us by email reply and then delete the message. It is > forbidden to copy, forward, or in any way reveal the contents of this > message to anyone without our explicit consent. The integrity and security > of this email cannot be guaranteed over the Internet. Therefore, the sender > will not be held liable for any damage caused by the message. For our full > privacy policy and disclaimers, please go to > https://www.namhost.com/privacy-policy > > [image: Powered by AdSigner] > <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> >