Team, Additionally I forgot to mention, v-154-VM <public-IP-3> <Private-IP> <control-ip-169.x.x.x> [State-Running] [Agent State-Gray] s-285-VM <public-IP-2> <Private-IP> <control-ip-169.x.x.y> [State-Running] [Agent State-Gray]
is [Agent State-Gray] status OK, or the normal Status, being Gray dot color leads to thing is not normal. What should I check, I reread again the instructions saw videos https://www.youtube.com/watch?v=9gXEmWbgX2o <<Actually this video shown both VMs are down' https://www.youtube.com/watch?v=DlJg3LYvIIs << this shows both on green. https://www.youtube.com/watch?v=vZfHfoYEMdY << this one shows the [Agent State] green while the Vms are in starting mode with a bue dot. And the Documentation does not state what the normal state should be If both [State] and [Agent State] need to be green, What other debug can I enable, as at this point the logs: At the Agent side only WARNINGS I am not using uefi. 2025-04-03 00:01:44,744 ERROR [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) uefi properties file not found due to: Unable to find file uefi.properties. 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1:[]) (logid:) Process [15539] for command [/bin/bash -c virt-v2v --version ] encountered the error: [127]. 2025-04-03 00:01:47,921 ERROR [cloud.agent.Agent] (agentRequest-Handler-5:[]) (logid:09929bd6) Unexpected arch null, expected x86_64 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1:[]) (logid:) Execution of process [15539] for command [/bin/bash -c virt-v2v --version ] failed. 2025-04-03 00:01:46,815 WARN [utils.script.Script] (Agent-Handler-1:[]) (logid:) Process [15539] for command [/bin/bash -c virt-v2v --version ] encountered the error: [127]. AT the management the rror is: 2025-04-03 00:00:28,115 DEBUG [c.c.c.ClusterServiceServletImpl] (Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Executing ClusterServicePdu with service URL: https://10.1.1.1:9090/clusterservice 2025-04-03 00:21:28,122 ERROR [c.c.c.ClusterServiceServletImpl] (Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Exception from : https://10.1.1.1:9090/clusterservice, method : null, exception : javax.net.ssl.SSLPeerUnverifiedException: Certificate for <10.1.1.1> doesn't match any of the subject alternative names: [fde0:f:2897:1:1:0:0:1, 172.1.1.1, fe80:0:0:0:e643:4bff:fe81:9660, cs1.myinternaldomain.int, cloudstack.internal] Thank you Tata Y. > On Apr 2, 2025, at 3:00 PM, Chi vediamo <tatay...@gmail.com> wrote: > > If I use a single Management server it works: > > > Question with the IP addresses assigned: > > the initial VMs are running: > v-154-VM <public-IP-3> <Private-IP> <control-ip-169.x.x.x> > s-285-VM <public-IP-2> <Private-IP> <control-ip-169.x.x.y> > > I do not see the Public IP anywhere, not even advertised on vni1000 or any > other, the public VNI is there was created by the system, but nothing is > there - No IP addresses advertised > > I used VLAN/VNI vxlan://1000 > > vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 master brvx-1000 state > forwarding priority > There are 2 VNETS assigned > > virsh domiflist s-285-VM > Interface Type Source Model MAC > -------------------------------------------------------------- > vnet3 bridge cloud0 virtio 0e:00:a9:fe:a1:14 > vnet4 bridge cloudbr1 virtio 1e:00:3f:00:03:33 > vnet5 bridge brvx-1000 virtio 1e:00:b8:00:0c:fe > > virsh domifaddr s-285-VM > Name MAC address Protocol Address > ------------------------------------------------------------------------------- > ----- Nothing here ----- > > > the command arp -n, does not showPublic IPaddresses, just the private IP > addresses and the <control-ip-169.x.x.x> > > sh bgp evpn route vni 1000 > BGP table version is 1, local router ID is 10.1.1.14 > Status codes: s suppressed, d damped, h history, * valid, > best, i - internal > Origin codes: i - IGP, e - EGP, ? - incomplete > EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id] > EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP] > EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP] > EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP] > EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP] > > Network Next Hop Metric LocPrf Weight Path > *> [3]:[0]:[32]:[10.1.1.14] > 10.1.1.14(hv1) > 32768 i > ET:8 RT:5641:1000 > > Displayed 1 prefixes (1 paths) > > > > Please Let me know if I should see the Public IP addresses advertised for > this VMs or any IP. > > > Tata Y. > > > > >> On Apr 1, 2025, at 9:35 AM, Wido den Hollander <w...@widodh.nl> wrote: >> >> >> >> Op 01-04-2025 om 13:16 schreef Chi vediamo: >>> Hello Wido, >>> Sorry I meant I only have cloudbr1, there is no cloudbr0 on the hypervisor >>> On the management server on the UI interface >>> when the VNI/VXLAN for Public is requested then I should enter vxlan://777 >>> in your case ? >> >> Yes, for the first network you create for example. >> >>> And for the VNI/VXLAN Range for guest leave it blank ? or ALso use a full >>> range including VXLAN200 in your case vxlan://1 through vxlan://1000 ? >> >> I don't think we ever used a VNI range in our setups. We just manually add >> the VNI for every network we create. >> >> Wido >> >>> Tata Y. >>>> On Apr 1, 2025, at 7:06 AM, Wido den Hollander <w...@widodh.nl> wrote: >>>> >>>> >>>> >>>> Op 01-04-2025 om 12:16 schreef Chi vediamo: >>>>> Thank you Wido for jumping in, as always >>>>> So far I only Have cloudbr1 as several VXLAN e-mail trails you indicated. >>>>> Then I need to create the cloudbr0 with another VXLAN in addition to >>>>> cloudbr1 ? cloudbr1 should have the management per the video you posted >>>>> right? >>>> >>>> In our case we use a static VNI underneath cloudbr1 where it communicates >>>> with the other hosts, the mgmt server and the secondary storage (NFS). >>>> This is a /22 IPv4 we have allocated for that VNI. >>>> >>>> vxlan200 device which we created using networkd and is a slave of cloudbr1 >>>> >>>> cloudbr0 is never used as the VXLAN script creates brvx-XXX devices on the >>>> fly. >>>> >>>>> Hope you recall, should I include in the VNI range the initial VNI/VXLANs >>>>> i am using for cloudbr1 and cloudbr0 >>>> >>>> Not that I'm aware of. We simply specify the broadcast domain when >>>> creating a new network: vxlan://777 for example to use VNI 777 for that >>>> network. >>>> >>>> This will result in brvx-777 being created on a host when needed there and >>>> the VM being attached to that bridge. >>>> >>>> Wido >>>> >>>>> Thank you so much. >>>>> Tata Y. >>>>>> On Apr 1, 2025, at 6:00 AM, Wido den Hollander <w...@widodh.nl> wrote: >>>>>> >>>>>> >>>>>> >>>>>> Op 29-03-2025 om 15:42 schreef Chi vediamo: >>>>>>> Hello Wei, >>>>>>> The VXLAN with EPVN is poorly documented, then I need all tthe guidance >>>>>>> as possible from the community. >>>>>> >>>>>> There is no silver bullet here as each EVPN+VXLAN setup will differ from >>>>>> the other :-) >>>>>> >>>>>>> Wido, please kindly allow me to use some of your multiple answers to >>>>>>> several members of the community as reference. As there is really not >>>>>>> much info. >>>>>>> * Wei what you suggesting I did try that, already, I did try several >>>>>>> options so far. The cloudstack-Agent keeps restarting itself, I copy >>>>>>> the logs in the e-mail. >>>>>>> * >>>>>>> * Saw countless videos, and read the manual, the Manual only states in >>>>>>> summary "VXLAN with EVPN is complex", there is nothing else. >>>>>>> * >>>>>>> * Manual states use and modify "modifyvxlan-evpn.sh" which does not >>>>>>> exists. Using the one form github. modifyvxlan.sh >>>>>>> * >>>>>> >>>>>> Yes, the Github one is good. It will be in packages from 4.21 >>>>>> >>>>>>> * The manual indicates: showing a picture of Management,public and >>>>>>> storage on vlans. and indicates - The only thing you have to do is >>>>>>> configure the physical Network to use VXLAN as the isolation method >>>>>>> for Guest Network - We do not have VLANS at all. >>>>>>> * >>>>>> >>>>>> Don't worry about different networks, VXLAN creates it's own bridges on >>>>>> the fly. >>>>>> >>>>>>> * Per one of the e-mails iIread long time ago from Wido to another >>>>>>> member of the community: we do left the UI settings by default. That >>>>>>> didn't work. And we start experimenting, and none seems to be >>>>>>> working. >>>>>> >>>>>> It has been a long time since I've set it up. But you should just have >>>>>> cloudbr0 and cloudbr1, nothing else. >>>>>> >>>>>>> Per Wido, we only need cloudbr1, with no VLANs. >>>>>>> I only need to know on the cludstack UI, What do we point to the >>>>>>> Cloudbr1 My understanding is This is going to be used for Migrations ad >>>>>>> that. >>>>>>> 4.20.0 does not allow me to add Management alone, nor the Guest Alone, >>>>>>> when creating the first zone. >>>>>>> I need Public and Management at least. Right? Or please indicate which >>>>>>> ones. I try to add only the Guest one, it also fails >>>>>>> Additionally, seems like the Cloudstack is not smart enough to see what >>>>>>> VNI VXLANs are already configured, then I have to provide a vni range >>>>>>> for the Guest. >>>>>>> Although the modifyvxlan.sh script should take care of it - that is >>>>>>> based on reading the script code. >>>>>>> So far we do have VXLAN-EVPN reachability and working. >>>>>>> I think we are almost there: >>>>>>> * I try adding Guest Only during the zone creation, we do have >>>>>>> management, public and storage on VXLANS, all are reacheable by >>>>>>> routing, and the cloudstack-agent server reach them and have >>>>>>> cloudbr1 created. >>>>>>> o Should I have only Managment and Guest on the cloudbr1? >>>>>>> * Need help Guidance with The cloudstack-agent keeps Bouncing comes up >>>>>>> and immediately down, and even reinstalled removing all files. Still >>>>>>> the same. >>>>>>> * Any example of the initial agent.properties: Or do not touch the >>>>>>> agent.properties >>>>>> >>>>>> Only set the network devices as you've done below. The guid should match >>>>>> what the hypervisor thinks the UUID of this Agent is. Again, long time >>>>>> ago since I've set this up for the first time. >>>>>> >>>>>> Wido >>>>>> >>>>>>> My agent.properties I have modified specifically 3 parameters: >>>>>>> host=<MAMGT-IP1>,<MGMT-IP2> >>>>>>> guest.cpu.mode=host-model >>>>>>> private.network.device=cloudbr1 >>>>>>> Thank you for your guidance. >>>>>>> Thank you >>>>>>> Tata Y. >>>>>>>> On Mar 28, 2025, at 1:43 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote: >>>>>>>> >>>>>>>> I think you do not need to change agent.properties >>>>>>>> Instead, you need to set the network traffic label of each traffic >>>>>>>> type, for example cloudbr1. >>>>>>>> >>>>>>>> By default, management traffic uses cloudbr0, public and guest traffic >>>>>>>> use cloudbr1, which might not be what you use. >>>>>>>> >>>>>>>> >>>>>>>> -Wei >>>>>>>> >>>>>>>> On Friday, March 28, 2025, Chi vediamo <tatay...@gmail.com >>>>>>>> <mailto:tatay...@gmail.com>> wrote: >>>>>>>> >>>>>>>> Hello Apache Cloudstack users, >>>>>>>> >>>>>>>> >>>>>>>> After attempted some options, I am recurring to what should work. >>>>>>>> As there is no info in the documentation about this deployment. >>>>>>>> >>>>>>>> I did replaced the agent server with private.network.device=cloudbr1 >>>>>>>> >>>>>>>> I have my VXLAN 1 on Cloudbr1 that reaches management and SS, We >>>>>>>> only have cloudbr1 created, No other bridge, and everything is BGP. >>>>>>>> >>>>>>>> >>>>>>>> Then at the UI creating the ZONE I got errors: >>>>>>>> >>>>>>>> We have had tryed the following Options: >>>>>>>> >>>>>>>> *Option 1: *No tags >>>>>>>> I put only [Physical Network 1] [VXLAN] Management, throws a popup >>>>>>>> error " All required traffic types should be added and with >>>>>>>> multiple physical networks each traffic type should have a label" >>>>>>>> >>>>>>>> *Option 2:* No Tags, Throws and Error " All required traffic types >>>>>>>> should be added and with multiple physical networks each traffic >>>>>>>> type should have a label" >>>>>>>> I put : >>>>>>>> [Physical Network 1] [VXLAN] Management >>>>>>>> [Physical Network 2] [VXLAN] Public >>>>>>>> [Physical Network 3] [VXLAN] Guest >>>>>>>> [Physical Network 4] [VXLAN] Storage >>>>>>>> >>>>>>>> >>>>>>>> *Option 3: * >>>>>>>> I put : >>>>>>>> [Physical Network 1] [VXLAN] Management, Public, Guest, it goes to >>>>>>>> the next window - which is Great >>>>>>>> The the system throws exceptions like: >>>>>>>> *Something went wrong; please correct the following:* >>>>>>>> *Unable to add the host: Cannot find the server resources at >>>>>>>> http://<KVM-IP>* >>>>>>>> >>>>>>>> sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml >>>>>>>> * >>>>>>>> * >>>>>>>> *Management logs shows:* >>>>>>>> Succeeded to import certificate in the keystore for agent on the >>>>>>>> KVM host: <ip>. Agent secured and trusted >>>>>>>> >>>>>>>> * >>>>>>>> * >>>>>>>> *The Agent Logs:* >>>>>>>> ** >>>>>>>> 2025-03-28 16:48:42,550 DEBUG [commons.beanutils.ConvertUtils] >>>>>>>> (main:[]) (logid:) Convert string '<IP-MGMT1>,<IP-MGMT2>' to class >>>>>>>> 'java.lang.String' >>>>>>>> 2025-03-28 16:48:42,550 DEBUG >>>>>>>> [beanutils.converters.StringConverter] (main:[]) (logid:) >>>>>>>> Converting 'String' value '<IP-MGMT1>,<IP-MGMT2>' to type 'String' >>>>>>>> 2025-03-28 16:48:42,551 DEBUG >>>>>>>> [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:) >>>>>>>> Property [guid] has empty or null value. Using default value [null]. >>>>>>>> 2025-03-28 16:48:42,552 DEBUG >>>>>>>> [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:) >>>>>>>> Property [developer] has empty or null value. Using default value >>>>>>>> [false]. >>>>>>>> 2025-03-28 16:48:53,897 INFO [cloud.agent.AgentShell] (main:[]) >>>>>>>> (logid:) Agent started >>>>>>>> 2025-03-28 16:48:53,902 INFO [cloud.agent.AgentShell] (main:[]) >>>>>>>> (logid:) Implementation Version is 4.20.0.0 >>>>>>>> 2025-03-28 16:48:53,904 INFO [cloud.agent.AgentShell] (main:[]) >>>>>>>> (logid:) agent.properties found at /etc/cloudstack/agent/ >>>>>>>> agent.properties >>>>>>>> 2025-03-28 16:48:53,913 DEBUG >>>>>>>> [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:) >>>>>>>> Property [host] was altered. Now using the value >>>>>>>> [10.23.123.61,10.23.123.62]. >>>>>>>> 2025-03-28 16:48:53,923 DEBUG >>>>>>>> [beanutils.converters.BooleanConverter] (main:[]) (logid:) Setting >>>>>>>> default value: false >>>>>>>> 2025-03-28 16:48:53,923 DEBUG >>>>>>>> [beanutils.converters.BooleanConverter] (main:[]) (logid:) >>>>>>>> Converting 'Boolean' value 'false' to type 'Boolean' >>>>>>>> 2025-03-28 16:48:53,923 DEBUG >>>>>>>> [beanutils.converters.BooleanConverter] (main:[]) (logid:) No >>>>>>>> conversion required, value is already a Boolean >>>>>>>> 2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter] >>>>>>>> (main:[]) (logid:) Setting default value: 0 >>>>>>>> 2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter] >>>>>>>> (main:[]) (logid:) Converting 'Integer' value '0' to type 'Byte' >>>>>>>> 2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter] >>>>>>>> (main:[]) (logid:) Converted to Byte value '0' >>>>>>>> 2025-03-28 16:48:53,926 DEBUG >>>>>>>> [beanutils.converters.CharacterConverter] (main:[]) (logid:) >>>>>>>> Setting default value: >>>>>>>> 2025-03-28 16:48:53,926 DEBUG >>>>>>>> [beanutils.converters.CharacterConverter] (main:[]) (logid:) >>>>>>>> Converting 'Character' value ' ' to type 'Character' >>>>>>>> 2025-03-28 16:48:53,926 DEBUG >>>>>>>> [beanutils.converters.CharacterConverter] (main:[]) (logid:) >>>>>>>> No conversion required, value is already a Character >>>>>>>> 2025-03-28 16:48:53,926 DEBUG >>>>>>>> [beanutils.converters.DoubleConverter] (main:[]) (logid:) Setting >>>>>>>> default value: 0 >>>>>>>> 2025-03-28 16:48:53,926 DEBUG >>>>>>>> [beanutils.converters.DoubleConverter] (main:[]) (logid:) >>>>>>>> Converting 'Integer' value '0' to type 'Double' >>>>>>>> 2025-03-28 16:48:53,927 DEBUG >>>>>>>> [beanutils.converters.DoubleConverter] (main:[]) (logid:) >>>>>>>> Converted to Double value '0.0' >>>>>>>> 2025-03-28 16:48:53,927 DEBUG >>>>>>>> [beanutils.converters.FloatConverter] (main:[]) (logid:) Setting >>>>>>>> default value: 0 >>>>>>>> 2025-03-28 16:48:53,927 DEBUG >>>>>>>> [beanutils.converters.FloatConverter] (main:[]) (logid:) >>>>>>>> Converting 'Integer' value '0' to type 'Float' >>>>>>>> 2025-03-28 16:48:53,927 DEBUG >>>>>>>> [beanutils.converters.FloatConverter] (main:[]) (logid:) >>>>>>>> Converted to Float value '0.0' >>>>>>>> 2025-03-28 16:48:53,928 DEBUG >>>>>>>> [beanutils.converters.IntegerConverter] (main:[]) (logid:) Setting >>>>>>>> default value: 0 >>>>>>>> * >>>>>>>> * >>>>>>>> AGENT out : >>>>>>>> Unable to find the guid >>>>>>>> Unable to find the guid >>>>>>>> Unable to find the guid >>>>>>>> Unable to find the guid >>>>>>>> Unable to find the guid >>>>>>>> >>>>>>>> There is nothing there >>>>>>>> >>>>>>>> And the agent come up, and immediately goes down!! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Tata Y. >>>>>>>> >>>>>> >>>> >> >
