Oh, should have mentioned that in the original post. I am using KVM hypervisor with ceph/rbd as the primary storage and nfs as the secondary storage.
Andrei > From: "Andrija Panic" <andrija.pa...@gmail.com> > To: "users" <users@cloudstack.apache.org>, "andrei" <and...@arhont.com> > Sent: Tuesday, 8 June, 2021 21:38:28 > Subject: Re: Snapshots are not working after upgrading to 4.15.0 > [ mailto:and...@arhont.com | @Andrei Mikhailovsky ] can you advise which > hypervisor (and version) you are using, what primary storage - let's see if > the > same is true in 4.15.1 (voting happening right now - feel free to test and > vote > as well, please) > Best, > On Tue, 8 Jun 2021 at 14:25, Andrei Mikhailovsky <and...@arhont.com.invalid> > wrote: >> Thanks for the suggestions, Harikrishna. I will check it and revert back. >> Andrei >> ----- Original Message ----- >>> From: "Harikrishna Patnala" < [ mailto:harikrishna.patn...@shapeblue.com | >> > harikrishna.patn...@shapeblue.com ] > >>> To: "users" < [ mailto:users@cloudstack.apache.org | >>> users@cloudstack.apache.org >> > ] > >> > Sent: Tuesday, 8 June, 2021 03:33:57 >> > Subject: Re: Snapshots are not working after upgrading to 4.15.0 >> > Hi Andrei, >> > Can you check the following things and let us know? >> > 1. Can you try creating a new volume and then create snapshot of that, to >> > check >> > if this an issue with old entries >> > 2. For the snapshots which are failing can you check if you are seeing any >> > error messages like this "Can't find an image storage in zone with less >> > than". >> > This is to check if secondary storage free space check failed. >> > 3. For the snapshots which are failing and if it is delta snapshot can you >> > check if its parent's snapshot entry exists in "snapshot_store_ref" table >> > with >> > 'parent_snapshot_id' of the current snapshot with 'store_role' "Image". >> > This is >> > to find the secondary storage where the parent snapshot backup is located. >> > Regards, >> > Harikrishna >> > ________________________________ >> > From: Andrei Mikhailovsky <and...@arhont.com.INVALID> >> > Sent: Monday, June 7, 2021 7:00 PM >>> To: users < [ mailto:users@cloudstack.apache.org | >>> users@cloudstack.apache.org ] >> > > >> > Subject: Snapshots are not working after upgrading to 4.15.0 >> > Hello everyone, >> > I am having an issue with volume snapshots since I've upgraded to 4.15.0. >> > None >> > of the volumes are being snapshotted regardless if the snapshot is >> > initiated >> > manually or from the schedule. The strange thing is that if I manually >> > take the >> > snapshot, the GUI shows Success status, but the Storage>Snapshots show an >> > Error >> > status. Here is what I see in the management server logs: >> > 2021-06-07 13:55:20,022 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> > (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) >> > Done >> > executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-86143 >> > 2021-06-07 13:55:20,024 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> > (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) >> > Remove >> > job-86143 from job monitoring >> > 2021-06-07 13:55:20,094 DEBUG [o.a.c.s.s.SnapshotServiceImpl] >> > (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Failed to copy >> > snapshot >> > com.cloud.utils.exception.CloudRuntimeException: can not find an image >> > stores >> > at >> > org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:271) >> > at >> > org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171) >> > at >> > com.cloud.storage.snapshot.SnapshotManagerImpl$BackupSnapshotTask.runInContext(SnapshotManagerImpl.java:1238) >> > at >> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48) >> > at >> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) >> > at >> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) >> > at >> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) >> > at >> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45) >> > at >> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) >> > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >> > at >> > java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> > at java.base/java.lang.Thread.run(Thread.java:829) >> > 2021-06-07 13:55:20,152 DEBUG [c.c.s.s.SnapshotManagerImpl] >> > (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Backing up of snapshot >> > failed, for snapshot with ID 53531, left with 2 more attempts >> > I've checked and the Secondary storage is configured and visible in the >> > GUI. I >> > can also mount it manually from the management server and a couple of host >> > servers that I've tested. In addition, I can successfully upload an ISO >> > image >> > and that registers just fine and I can create new VMs using the newly >> > uploaded >> > ISO image. >> > I've had no such problems with 4.13.x ACS, so the issue seems to have been >> > introduced after doing the upgrade to 4.15.0. >> > Could you please let me know how do I fix the issue? >> > Cheers >> > andrei > -- > Andrija Panić
