Hi, I had a couple of other questions if you or someone else doesn't mind.
This is basically the network configuration of the POC host and the system VMs. 192.168.1.112/29 is vlan 143 (labeled as management in cloudstack) 10.1.65.0/24 is vlan 100 (labeled as public in cloudstack) Hypervisor: Eno8303 192.168.1.114/29 Eno8403 bridge cloudbr1 vlan 100 and untagged/native (vlan 143) Console SVM: 192.168.1.115/10.1.65.2 Storage SVM 192.168.1.116/10.1.65.3 With firewalld enabled: The HV IP address 192.168.1.114 replies with "filtered" any time any traffic hits either of the System VMs on the 10.1.65.0/24 network (which are the public NICs on the VMs) But it permits traffic to go through to 192.168.1.115 and 192.168.1.116 without any issue (which are the private NICs on the VMs) With firewalld disabled nothing is filtered. I don't really understand why firewalld is making a distinction between whether it will filter or reply to traffic from those two subnets unless it's allowing the "management traffic to the system VMs" to pass simply because eno8303 has an IP in the same subnet as the bridge. But that seems somewhat arbitrary. Is it possible to use firewalld to protect the Hypervisor's self IP while not interfering with bridged traffic while not totally breaking security groups? If not I can deploy this POC behind an actual firewall I'm just trying to figure out what is expected. Assuming this switches from being a POC to an actual use case our workload VMs will only have a single IP address attached to the "public" network. How do people in this scenario usually handle the systemVMs networking? I would honestly prefer that they not exist on the "public" network at all but it doesn't seem like that is really a supported use case. Thanks, -Drew -----Original Message----- From: Wei ZHOU <[email protected]> Sent: Thursday, September 10, 2026 11:35 AM To: [email protected] Subject: Re: An issue adding a host to a cluster ALMA10 Although the error logs mention the certificate, as far as I know, these messages can be misleading, and the root cause may be elsewhere. On the host, there is a log file at /var/log/cloudstack/agent/setup.log that you may want to check. You can also search for cloudstack-setup-agent in management-server.log to find some additional information. -Wei On Thu, Sep 10, 2026 at 5:24 PM Drew Weaver <[email protected]> wrote: > Hello, > > Yes, I did pre-create the specified bridge on the server. > > Because of the insistence that Java17 was the only version that would > work in the documentation and Google, etc I installed JRE17 and tried > that. It didn't immediately make much of a difference. > > I was able to get the first hypervisor to provision but sadly I'm not > really sure what actually caused it to finally function properly as I > feel like I've deleted the > https://urldefense.proofpoint.com/v2/url?u=http-3A__agent.properties&d=DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=rc8c4bEXFUCeAKWv8TiZisIRUloKKkXrBcSIAtQLu54oVgm-0sF-TTGHnsDvhpL7&s=J8twFVZnybEdUjej6heoPRlRcw2OMMYsvq_9kgm8_3g&e= > file 100 times just trying different things along the way. > > I did this: > > rm -rf > https://urldefense.proofpoint.com/v2/url?u=http-3A__agent.properties&d > =DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpz > foijO_w76wskMALE1o4LtA3tMGmuw&m=rc8c4bEXFUCeAKWv8TiZisIRUloKKkXrBcSIAt > QLu54oVgm-0sF-TTGHnsDvhpL7&s=J8twFVZnybEdUjej6heoPRlRcw2OMMYsvq_9kgm8_ > 3g&e= touch /etc/cloudstack/agent/agent.properties > chmod 644 /etc/cloudstack/agent/agent.properties > > and then re-ran the add host via SSH process and it "got further along > the trail" but then the agent couldn't connect to the mgmt. server > because the passphrase in the > https://urldefense.proofpoint.com/v2/url?u=http-3A__agent.properties&d > =DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpz > foijO_w76wskMALE1o4LtA3tMGmuw&m=rc8c4bEXFUCeAKWv8TiZisIRUloKKkXrBcSIAt > QLu54oVgm-0sF-TTGHnsDvhpL7&s=J8twFVZnybEdUjej6heoPRlRcw2OMMYsvq_9kgm8_ > 3g&e= file no longer matched what was needed by > /etc/cloudstack/agent/cloud.jks > > So then I deleted this file: sudo rm -f > /etc/cloudstack/agent/cloud.jks and re-ran the add host process again > and verified that the passphrase listed in > https://urldefense.proofpoint.com/v2/url?u=http-3A__agent.properties&d=DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=rc8c4bEXFUCeAKWv8TiZisIRUloKKkXrBcSIAtQLu54oVgm-0sF-TTGHnsDvhpL7&s=J8twFVZnybEdUjej6heoPRlRcw2OMMYsvq_9kgm8_3g&e= > could unlock the new jks file. > > Then I noticed that the connection between libvirtd and the mgmt. > server was "insecure" in the UI and that the keys were symlinked to > this file (which still didn't exist) /etc/cloudstack/agent/cloud.key > > So I ran > sudo keytool -exportcert -keystore /etc/cloudstack/agent/cloud.jks > -alias cloud -storepass passphrase -rfc -file > /etc/cloudstack/agent/cloud.crt sudo keytool -importkeystore > -srckeystore /etc/cloudstack/agent/cloud.jks -destkeystore > /tmp/cloud.p12 -deststoretype PKCS12 -srcstorepass passphrase > -deststorepass passphrase sudo openssl pkcs12 -in /tmp/cloud.p12 > -nodes -nocerts -out /etc/cloudstack/agent/cloud.key -passin > pass:passphrase sudo rm -f /tmp/cloud.p12 sudo chmod 0600 > /etc/cloudstack/agent/cloud.key > > To create that file then restarted libvirtd and the agent and that > seems to have fixed the certificate issue. > > This was my initial process to setup this hypervisor: > > copy the system SSH key to /root/.ssh/authorized_keys disable selinux > created the bridge cloudbr1 ->eno8403 added the NFS share for > secondary storage created this /etc/yum.repos.d/cloudstack.repo > configured a hostname made sure chrony was running installed Java21, > cloudstack-agent and libvirtd, etc (java21 was replaced with Java17 > later) added these rules in firewall-cmd: > rule family="ipv4" source address="192.168.7.2" port port="1798" > protocol="tcp" accept > rule family="ipv4" source address="192.168.7.2" port port="5900-6100" > protocol="tcp" accept > rule family="ipv4" source address="192.168.7.2" port port="16514" > protocol="tcp" accept > rule family="ipv4" source address="192.168.7.2" service name="ssh" > accept rule family="ipv4" source address="192.168.7.2" port port="49152-49216" > protocol="tcp" accept > > and then tried to add the host via SSH (using the system key) and it > pretty much immediately had issues with what it was doing regarding > the provisioning of certificates, etc. > > I would like to make a process that works "on the first try" as far as > adding hosts. Does anyone see any particular madness in my process above? > > I'm going to reformat this host and see if I can get it to provision > the hypervisor cleanly the next time but if anyone has any tips > especially if they are specific to EL/AlmaLinux 10 I would really appreciate > it. > > Thanks, > -Drew > >
