Hi CloudStack Community, 

I am facing an issue with Secondary Storage VM ( s-*-VM ) deployment in Apache 
CloudStack 4. 21. 0 on Rocky Linux 9 KVM hypervisors when integrating with Ceph 
Ingress NFSv4 proxy. Environment Details: 


    * 

CloudStack Version: 4.21.0.0 
    * 

Management Server: Rocky Linux 9 
    * 

Hypervisors: KVM on Rocky Linux 9 (modular libvirt / monolithic libvirtd with 
TCP 16509 ) 
    * 

Primary Storage: Ceph RBD ( titan-vm pool, TitanFlash ) 
    * 

Secondary Storage: Ceph NFS Ganesha exported via Ceph Ingress VIP ( 
10.0.25.9:/secondary ) 

Observed Behavior: 


    1. 

When using Ceph Ingress VIP ( 10.0.25.9:/secondary ): 
        * 

The Secondary Storage VM ( s-2-VM / s-4-VM ) fails during orchestration and 
goes into Stopped state. 
        * 

On the KVM agent side, the VM creation hangs or triggers a NullPointerException 
/ interface plug failure: 
Caught: java.lang. NullPointerException: Cannot invoke 
"java.net.URI.getScheme()" because "uri" is null
    at com.cloud.network.Networks$ BroadcastDomainType. 
getSchemeValue(Networks.java: 180)
    at com.cloud.hypervisor.kvm. resource.BridgeVifDriver.plug( 
BridgeVifDriver.java:258)
    at com.cloud.hypervisor.kvm. resource. LibvirtComputingResource. createVif( 
LibvirtComputingResource.java: 3816) 

        * 

Management server reports: 
ERROR [c.c.v. ClusteredVirtualMachineManager Impl] Unable to orchestrate start 
VM instance due to [Unable to get answer that is of class com.cloud.agent.api. 
StartAnswer]. 

        * 

On Ceph RBD, leftover images and exclusive locks are retained, causing 
subsequent attempts to fail opening the disk device. 
    2. 

When using Direct Node IP ( 10.0.25.20:/secondary ): 
        * 

Mounts without issues, system VMs start up cleanly, and StartAnswer: true is 
returned. 

Questions: 


    1. 

Is Ceph Ingress VIP (Keepalived + HAProxy on port 2049) supported natively for 
Secondary Storage NFS mounts, or does the lack of RPC portmapper / HAProxy TCP 
session reset break the KVM agent's NFSv4 state handling? 
    2. 

Are there recommended NFS mount options ( secstorage.nfs.version , proto=tcp , 
sync , etc.) or specific HAProxy timeout settings when using an Ingress VIP for 
Secondary Storage in CloudStack 4.21? 
    3. 

What is the recommended production architecture for providing high-availability 
to Ceph NFS Secondary Storage without falling into session/lock drops on 
failover? 



Any insights, recommendations, or architecture best practices from the 
community would be greatly appreciated. 




Best regards, 

Md Titu Hossain 

Cloud & Systems Infrastructure Architect 

Reply via email to