Hi Si, Thanks so much for the follow-up!
> Could you tell us a bit more about your setup? I've got a little lab setup, with a dedicated router, switch, and several 1U's that I can test with. Will eventually set up a little micro-environment with VLAN's and such but for now just trying to get things working - which is turning out to be much more difficult than it sounds. > Hypervisor Type? KVM for now > Network design (basic or advanced)? Basic; using a single 192.168.253.0/24 range for the various networks similar to what's shown here: http://docs.cloudstack.apache.org/en/4.11.1.0/installguide/hypervisor/kvm.html#configuring-the-networking I've got the two bridges set up with the management IP assigned to cloudbr0 and no ip assigned to cloudbr1. > Primary storage type? I've tried NFS and local, though the result is the same regardless. > CloudStack version? 4.11 > OS being used? CentOS 7 More than just getting it working, I'd like to know where to find out what error is actually occurring here. I want to understand the details. I'm guessing the NPE is because the API was expecting to get something back from the storage server and it didn't? Is the command that was sent logged anywhere? What about the result? If the command that was sent to the storage server threw an error, is there a way that I can see what the error was specifically? Thanks so much for your time. I really appreciate it. -- Kind regards, Jordan Michaels Vivio Technologies ----- Original Message ----- From: "Simon Weller" <swel...@ena.com> To: "users" <users@cloudstack.apache.org>, "jordan" <jor...@viviotech.net> Sent: Thursday, 18 October, 2018 13:14:41 Subject: Re: Where to look when your SSVM creation NPE's? Jordan, Could you tell us a bit more about your setup? Hypervisor Type? Network design (basic or advanced)? Primary storage type? CloudStack version? OS being used? - Si ________________________________ From: Jordan Michaels <mic...@viviotech.net> Sent: Thursday, October 18, 2018 2:26 PM To: users Subject: Where to look when your SSVM creation NPE's? Hi Folks, I've added a host to the CloudStack Management server but now I'm getting NPE's when the Management server tries to create the System VM's: 2018-10-18 06:20:26,052 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:ctx-ac017e0e job-1248/job-1621) (logid:b6f3dd96) Done executing com.cloud.vm.VmWorkStart for job-1621 2018-10-18 06:20:26,054 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-40:ctx-ac017e0e job-1248/job-1621) (logid:b6f3dd96) Remove job-1621 from job monitoring 2018-10-18 06:20:26,073 WARN [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) Exception while trying to start secondary storage vm java.lang.NullPointerException at com.cloud.network.guru.DirectPodBasedNetworkGuru$2.doInTransactionWithoutResult(DirectPodBasedNetworkGuru.java:183) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:25) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:21) at com.cloud.utils.db.Transaction.execute(Transaction.java:40) at com.cloud.network.guru.DirectPodBasedNetworkGuru.getIp(DirectPodBasedNetworkGuru.java:173) at com.cloud.network.guru.DirectPodBasedNetworkGuru.reserve(DirectPodBasedNetworkGuru.java:161) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1594) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1565) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1111) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4930) at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5093) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-10-18 06:20:26,076 INFO [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) Unable to start secondary storage vm for standby capacity, vm id : 346, will recycle it and start a new one 2018-10-18 06:20:26,076 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) received secondary storage vm alert 2018-10-18 06:20:26,076 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) Secondary Storage Vm creation failure, zone: lab 2018-10-18 06:20:26,077 WARN [o.a.c.alerts] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) AlertType:: 19 | dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary Storage Vm creation failure. zone: lab, error details: null 2018-10-18 06:20:26,095 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-70aba7ab) (logid:f3c51c51) Primary secondary storage is not even started, wait until next turn Based on the error, it sounds like there's an issue with the SSVM being able to communicate with secondary storage, but I don't know where to look for additional details about what's going on, or how the SSVM's actually work. I need more information. I've been stumbling around in the dark and making sure I have secondary storage added without errors to the cloudstack management server, I've configured the SSVM template... not sure what else could be wrong or where else to look. Could someone point me in the right direction on where to look to get what's actually failing here? Thank you in advance! Kind regards, Jordan Michaels Vivio Technologies
