Re: 4.11.0 - can't create guest vms with RBD storage!
Andrei, Nathan has pushed a PR to fix this. Please see: https://github.com/apache/cloudstack/pull/2623 He has done some basic testing on it, but your feedback I'm sure would be appreciated. - Si From: Simon Weller Sent: Wednesday, May 2, 2018 4:27 PM To: dev@cloudstack.apache.org Subject: Re: 4.11.0 - can't create guest vms with RBD storage! We've starting looking into this particular bug. We now have a 4.11 lab setup and can reproduce this. - Si From: Wei ZHOU Sent: Monday, April 30, 2018 1:25 PM To: dev@cloudstack.apache.org Subject: Re: 4.11.0 - can't create guest vms with RBD storage! Agreed. agent.log might be helpful for troubleshooting. it seems to be a bug within kvm plugin. -Wei 2018-04-30 15:36 GMT+02:00 Rafael Weingärtner : > We might need some extra log entries. Can you provide them? > > On Mon, Apr 30, 2018 at 10:14 AM, Andrei Mikhailovsky < > and...@arhont.com.invalid> wrote: > > > hello gents, > > > > I have just realised that after upgrading to 4.11.0 we are no longer able > > to create new VMs. This has just been noticed as we have previously used > > ready made templates, which work just fine. > > > > Setup: ACS 4.11.0 (upgraded from 4.9.3), KVM + CEPH, Ubuntu 16.04 on all > > servers > > > > When trying to create a new vm from an ISO image I get the following > > error: > > > > > > com.cloud.exception.StorageUnavailableException: Resource > [StoragePool:2] > > is unreachable: Unable to create Vol[3937|vm=2217|ROOT]:com. > > cloud.utils.exception.CloudRuntimeException: > > org.libvirt.LibvirtException: this function is not supported by the > > connection driver: only RAW volumes are supported by this storage pool > > > > at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator. > > recreateVolume(VolumeOrchestrator.java:1336) > > at org.apache.cloudstack.engine.orchestration. > VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413) > > > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:1110) > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:4927) > > at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( > > VmWorkJobHandlerProxy.java:107) > > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( > > VirtualMachineManagerImpl.java:5090) > > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5. > > runInContext(AsyncJobManagerImpl.java:581) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run( > > ManagedContextRunnable.java:49) > > at org.apache.cloudstack.managed.context.impl. > > DefaultManagedContext$1.call(DefaultManagedContext.java:56) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > callWithContext(DefaultManagedContext.java:103) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > runWithContext(DefaultManagedContext.java:53) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run( > > ManagedContextRunnable.java:46) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > AsyncJobManagerImpl.java:529) > > > > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > My guess is that ACS tried to create a QCOW2 image type whereas it should > > be RAW on ceph/rbd. > > > > I am really struggling to understand how this bug in a function of MAJOR > > importance could have been missed during the tests ran by developers and > > community before making a final realise. Anyways, I hope the fix will > make > > it to 4.11.1 release, otherwise it's really messed up! > > > > Cheers > > > > Andrei > > > > > > -- > Rafael Weingärtner >
Re: 4.11.0 - can't create guest vms with RBD storage!
We've starting looking into this particular bug. We now have a 4.11 lab setup and can reproduce this. - Si From: Wei ZHOU Sent: Monday, April 30, 2018 1:25 PM To: dev@cloudstack.apache.org Subject: Re: 4.11.0 - can't create guest vms with RBD storage! Agreed. agent.log might be helpful for troubleshooting. it seems to be a bug within kvm plugin. -Wei 2018-04-30 15:36 GMT+02:00 Rafael Weingärtner : > We might need some extra log entries. Can you provide them? > > On Mon, Apr 30, 2018 at 10:14 AM, Andrei Mikhailovsky < > and...@arhont.com.invalid> wrote: > > > hello gents, > > > > I have just realised that after upgrading to 4.11.0 we are no longer able > > to create new VMs. This has just been noticed as we have previously used > > ready made templates, which work just fine. > > > > Setup: ACS 4.11.0 (upgraded from 4.9.3), KVM + CEPH, Ubuntu 16.04 on all > > servers > > > > When trying to create a new vm from an ISO image I get the following > > error: > > > > > > com.cloud.exception.StorageUnavailableException: Resource > [StoragePool:2] > > is unreachable: Unable to create Vol[3937|vm=2217|ROOT]:com. > > cloud.utils.exception.CloudRuntimeException: > > org.libvirt.LibvirtException: this function is not supported by the > > connection driver: only RAW volumes are supported by this storage pool > > > > at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator. > > recreateVolume(VolumeOrchestrator.java:1336) > > at org.apache.cloudstack.engine.orchestration. > VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413) > > > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:1110) > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:4927) > > at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( > > VmWorkJobHandlerProxy.java:107) > > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( > > VirtualMachineManagerImpl.java:5090) > > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5. > > runInContext(AsyncJobManagerImpl.java:581) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run( > > ManagedContextRunnable.java:49) > > at org.apache.cloudstack.managed.context.impl. > > DefaultManagedContext$1.call(DefaultManagedContext.java:56) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > callWithContext(DefaultManagedContext.java:103) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > runWithContext(DefaultManagedContext.java:53) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run( > > ManagedContextRunnable.java:46) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > AsyncJobManagerImpl.java:529) > > > > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > My guess is that ACS tried to create a QCOW2 image type whereas it should > > be RAW on ceph/rbd. > > > > I am really struggling to understand how this bug in a function of MAJOR > > importance could have been missed during the tests ran by developers and > > community before making a final realise. Anyways, I hope the fix will > make > > it to 4.11.1 release, otherwise it's really messed up! > > > > Cheers > > > > Andrei > > > > > > -- > Rafael Weingärtner >
Re: 4.11.0 - can't create guest vms with RBD storage!
Agreed. agent.log might be helpful for troubleshooting. it seems to be a bug within kvm plugin. -Wei 2018-04-30 15:36 GMT+02:00 Rafael Weingärtner : > We might need some extra log entries. Can you provide them? > > On Mon, Apr 30, 2018 at 10:14 AM, Andrei Mikhailovsky < > and...@arhont.com.invalid> wrote: > > > hello gents, > > > > I have just realised that after upgrading to 4.11.0 we are no longer able > > to create new VMs. This has just been noticed as we have previously used > > ready made templates, which work just fine. > > > > Setup: ACS 4.11.0 (upgraded from 4.9.3), KVM + CEPH, Ubuntu 16.04 on all > > servers > > > > When trying to create a new vm from an ISO image I get the following > > error: > > > > > > com.cloud.exception.StorageUnavailableException: Resource > [StoragePool:2] > > is unreachable: Unable to create Vol[3937|vm=2217|ROOT]:com. > > cloud.utils.exception.CloudRuntimeException: > > org.libvirt.LibvirtException: this function is not supported by the > > connection driver: only RAW volumes are supported by this storage pool > > > > at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator. > > recreateVolume(VolumeOrchestrator.java:1336) > > at org.apache.cloudstack.engine.orchestration. > VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413) > > > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:1110) > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > > VirtualMachineManagerImpl.java:4927) > > at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( > > VmWorkJobHandlerProxy.java:107) > > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( > > VirtualMachineManagerImpl.java:5090) > > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5. > > runInContext(AsyncJobManagerImpl.java:581) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run( > > ManagedContextRunnable.java:49) > > at org.apache.cloudstack.managed.context.impl. > > DefaultManagedContext$1.call(DefaultManagedContext.java:56) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > callWithContext(DefaultManagedContext.java:103) > > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > > runWithContext(DefaultManagedContext.java:53) > > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run( > > ManagedContextRunnable.java:46) > > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > AsyncJobManagerImpl.java:529) > > > > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > My guess is that ACS tried to create a QCOW2 image type whereas it should > > be RAW on ceph/rbd. > > > > I am really struggling to understand how this bug in a function of MAJOR > > importance could have been missed during the tests ran by developers and > > community before making a final realise. Anyways, I hope the fix will > make > > it to 4.11.1 release, otherwise it's really messed up! > > > > Cheers > > > > Andrei > > > > > > -- > Rafael Weingärtner >
Re: 4.11.0 - can't create guest vms with RBD storage!
We might need some extra log entries. Can you provide them? On Mon, Apr 30, 2018 at 10:14 AM, Andrei Mikhailovsky < and...@arhont.com.invalid> wrote: > hello gents, > > I have just realised that after upgrading to 4.11.0 we are no longer able > to create new VMs. This has just been noticed as we have previously used > ready made templates, which work just fine. > > Setup: ACS 4.11.0 (upgraded from 4.9.3), KVM + CEPH, Ubuntu 16.04 on all > servers > > When trying to create a new vm from an ISO image I get the following > error: > > > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:2] > is unreachable: Unable to create Vol[3937|vm=2217|ROOT]:com. > cloud.utils.exception.CloudRuntimeException: > org.libvirt.LibvirtException: this function is not supported by the > connection driver: only RAW volumes are supported by this storage pool > > at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator. > recreateVolume(VolumeOrchestrator.java:1336) > at > org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413) > > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > VirtualMachineManagerImpl.java:1110) > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > VirtualMachineManagerImpl.java:4927) > at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( > VmWorkJobHandlerProxy.java:107) > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( > VirtualMachineManagerImpl.java:5090) > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5. > runInContext(AsyncJobManagerImpl.java:581) > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run( > ManagedContextRunnable.java:49) > at org.apache.cloudstack.managed.context.impl. > DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > callWithContext(DefaultManagedContext.java:103) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext. > runWithContext(DefaultManagedContext.java:53) > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run( > ManagedContextRunnable.java:46) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > > My guess is that ACS tried to create a QCOW2 image type whereas it should > be RAW on ceph/rbd. > > I am really struggling to understand how this bug in a function of MAJOR > importance could have been missed during the tests ran by developers and > community before making a final realise. Anyways, I hope the fix will make > it to 4.11.1 release, otherwise it's really messed up! > > Cheers > > Andrei > -- Rafael Weingärtner
4.11.0 - can't create guest vms with RBD storage!
hello gents, I have just realised that after upgrading to 4.11.0 we are no longer able to create new VMs. This has just been noticed as we have previously used ready made templates, which work just fine. Setup: ACS 4.11.0 (upgraded from 4.9.3), KVM + CEPH, Ubuntu 16.04 on all servers When trying to create a new vm from an ISO image I get the following error: com.cloud.exception.StorageUnavailableException: Resource [StoragePool:2] is unreachable: Unable to create Vol[3937|vm=2217|ROOT]:com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: this function is not supported by the connection driver: only RAW volumes are supported by this storage pool at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1336) at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1413) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1110) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4927) at sun.reflect.GeneratedMethodAccessor498.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5090) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) My guess is that ACS tried to create a QCOW2 image type whereas it should be RAW on ceph/rbd. I am really struggling to understand how this bug in a function of MAJOR importance could have been missed during the tests ran by developers and community before making a final realise. Anyways, I hope the fix will make it to 4.11.1 release, otherwise it's really messed up! Cheers Andrei
