Sparc0 opened a new issue, #10517:
URL: https://github.com/apache/cloudstack/issues/10517
### problem
Just want to start saying that i and my company found Cloudstack to be a
very interesting project and we are looking at it as a replacement for our
oVirt env.
We are in the middle of a POC so this is our first cluster at the moment.
I am using S3 as secondary storage with a NFS staging connected to it.
My consoleproxy VM starts without issues but my SSVM refuse to starts and i
see following error in my logs
```
2025-03-06 13:19:14,173 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-135:[ctx-9dbacb5f, job-1299/job-3426]) (logid:5e35be66)
Unable to complete AsyncJobVO: {id:3426, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2
Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAES
gAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAABinQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 95037156360330, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Thu Mar 06 13:19:09 CET 2025, re
moved: null}, job origin: 1299 java.lang.NullPointerException: Cannot invoke
"String.split(String)" because "url" is null
at
org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.addSecondaryStorageServerAddressToBuffer(SecondaryStorageManagerImpl.java:1228)
at
org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.finalizeVirtualMachineProfile(SecondaryStorageManagerImpl.java:1216)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1272)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5467)
at jdk.internal.reflect.GeneratedMethodAccessor193.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
```
My guess it has to do with that the URL under Infrastructure -> Secondary
storage is null? (See picture)
Even using CMK the url field is null or not even exist in the response.
```
(localcloud) 🐱 > list imagestores id=a8bbf162-cc05-4cac-92d2-f18d43627cdd
filter=name,url
{
"count": 1,
"imagestore": [
{
"name": "S3"
}
]
}
```
Also if i try to enter the secondary storage and browse it i see following
error log that might be connected to this.
```
2025-03-06 13:56:18,487 ERROR [c.c.a.ApiServer]
(qtp652176954-348:[ctx-c959a22d, ctx-21cc0661]) (logid:6136e9bd) unhandled
exception executing api command: [Ljava.lang.String;@2f1ab954
com.cloud.utils.exception.CloudRuntimeException: No remote endpoint to send
command
at
org.apache.cloudstack.storage.browser.StorageBrowserImpl.listObjectsInStore(StorageBrowserImpl.java:183)
at
org.apache.cloudstack.storage.browser.StorageBrowserImpl.listImageStoreObjects(StorageBrowserImpl.java:128)
at
org.apache.cloudstack.api.command.admin.storage.ListImageStoreObjectsCmd.execute(ListImageStoreObjectsCmd.java:72)
```
### versions
ACS: 4.20.0
Hypervisor: KVM with Rocky 9.5
Network: OpenVswitch 3.3
Primary Storage: Ceph RBD
Secondary Storage: NetApp S3 + Ceph NFS Staging
### The steps to reproduce the bug
1. Add a S3 Secondary storage either using WebUI or CMK/API
2. Wait for templates to download and SystemVMs to start
3. ConsoleProxy starts while SSVM will not start.
...
### What to do about it?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
