Hello everyone, I am having an issue with volume snapshots since I've upgraded to 4.15.0. None of the volumes are being snapshotted regardless if the snapshot is initiated manually or from the schedule. The strange thing is that if I manually take the snapshot, the GUI shows Success status, but the Storage>Snapshots show an Error status. Here is what I see in the management server logs:
2021-06-07 13:55:20,022 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) Done executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-86143 2021-06-07 13:55:20,024 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) Remove job-86143 from job monitoring 2021-06-07 13:55:20,094 DEBUG [o.a.c.s.s.SnapshotServiceImpl] (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Failed to copy snapshot com.cloud.utils.exception.CloudRuntimeException: can not find an image stores at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:271) at org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171) at com.cloud.storage.snapshot.SnapshotManagerImpl$BackupSnapshotTask.runInContext(SnapshotManagerImpl.java:1238) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2021-06-07 13:55:20,152 DEBUG [c.c.s.s.SnapshotManagerImpl] (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Backing up of snapshot failed, for snapshot with ID 53531, left with 2 more attempts I've checked and the Secondary storage is configured and visible in the GUI. I can also mount it manually from the management server and a couple of host servers that I've tested. In addition, I can successfully upload an ISO image and that registers just fine and I can create new VMs using the newly uploaded ISO image. I've had no such problems with 4.13.x ACS, so the issue seems to have been introduced after doing the upgrade to 4.15.0. Could you please let me know how do I fix the issue? Cheers andrei
