Here is the issue, ACS cannot find the template volume to copy -
d2c4feca-da5c-44b3-8ecc-40a01fd6c243
2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId:
66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux
8 cPanel
Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00
GB)
64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00
GB)
64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}]
}
2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been
pending for 141 seconds
2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been
pending for 201 seconds
2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been
pending for 261 seconds
2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: ,
MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags:
110, { CopyCmdAnswer } }
2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Unable to create
Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
If other templates can be used from the same SR this means it is accessible
but the template file is not there anymore.
To look for it you can login on the secondary storage and look at a path
similar to this <mount dir>/../templates.
inside should be 1 folder for each template, it will include XML meta file
and volume file. The volume, if exists should be named by the ID
d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
if no such is present than indeed it is deleted.
Regards,
Jordan
On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <granwi...@namhost.com>
wrote:
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm if
> a template exists on secondary storage?
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille
> Strauss<granwi...@namhost.com.invalid> <granwi...@namhost.com.invalid> wrote:
>
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted because
> there are active VMs associated with it. However, since I did this, I can
> now no longer deploy new VMs using said template. Creating a VM now fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss //
> Senior Systems Admin
>
> *e:* granwi...@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost>
> <https://twitter.com/namhost>
> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/>
>
> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos>
>
> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go
> tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by
> AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwi...@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
