Team,
Additionally I forgot to mention,
v-154-VM   <public-IP-3>   <Private-IP>  <control-ip-169.x.x.x>  
[State-Running] [Agent State-Gray]
s-285-VM  <public-IP-2>   <Private-IP>  <control-ip-169.x.x.y>   
[State-Running] [Agent State-Gray]

is [Agent State-Gray] status OK, or the normal Status, being Gray dot color 
leads to thing is not normal.

What should I check, I reread again the instructions saw videos 
https://www.youtube.com/watch?v=9gXEmWbgX2o  <<Actually this video shown both 
VMs are down'
https://www.youtube.com/watch?v=DlJg3LYvIIs  << this shows both on green.
https://www.youtube.com/watch?v=vZfHfoYEMdY << this one shows the [Agent State] 
green while the Vms are in starting mode with a bue dot.

And the Documentation does not state what the normal state should be


If both [State] and [Agent State] need to be green, What other debug can I 
enable, as at this point the logs:


At the Agent side only WARNINGS  I am not using uefi.

2025-04-03 00:01:44,744 ERROR [kvm.resource.LibvirtComputingResource] (main:[]) 
(logid:) uefi properties file not found due to: Unable to find file 
uefi.properties.
2025-04-03 00:01:46,815 WARN  [utils.script.Script] (Agent-Handler-1:[]) 
(logid:) Process [15539] for command [/bin/bash -c virt-v2v --version ] 
encountered the error: [127].
2025-04-03 00:01:47,921 ERROR [cloud.agent.Agent] (agentRequest-Handler-5:[]) 
(logid:09929bd6) Unexpected arch null, expected x86_64
2025-04-03 00:01:46,815 WARN  [utils.script.Script] (Agent-Handler-1:[]) 
(logid:) Execution of process [15539] for command [/bin/bash -c virt-v2v 
--version ] failed.
2025-04-03 00:01:46,815 WARN  [utils.script.Script] (Agent-Handler-1:[]) 
(logid:) Process [15539] for command [/bin/bash -c virt-v2v --version ] 
encountered the error: [127].


AT the management  the rror is:
2025-04-03 00:00:28,115 DEBUG [c.c.c.ClusterServiceServletImpl] 
(Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Executing ClusterServicePdu 
with service URL: https://10.1.1.1:9090/clusterservice
2025-04-03 00:21:28,122 ERROR [c.c.c.ClusterServiceServletImpl] 
(Cluster-Worker-5:[ctx-c96bca9e]) (logid:3fbff637) Exception from : 
https://10.1.1.1:9090/clusterservice, method : null, exception : 
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <10.1.1.1> doesn't 
match any of the subject alternative names: [fde0:f:2897:1:1:0:0:1, 172.1.1.1, 
fe80:0:0:0:e643:4bff:fe81:9660, cs1.myinternaldomain.int, cloudstack.internal]


Thank you


Tata Y.


> On Apr 2, 2025, at 3:00 PM, Chi vediamo <tatay...@gmail.com> wrote:
> 
> If I use a single Management server it works:
> 
> 
> Question with the IP addresses assigned:
> 
> the initial VMs are running:
> v-154-VM   <public-IP-3>   <Private-IP>  <control-ip-169.x.x.x>
> s-285-VM  <public-IP-2>   <Private-IP>  <control-ip-169.x.x.y>
> 
> I do not see the Public IP anywhere, not even advertised on vni1000 or any 
> other, the public VNI is there was created by the system, but nothing is 
> there - No IP addresses advertised
> 
> I used VLAN/VNI  vxlan://1000 
> 
> vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 master brvx-1000 state 
> forwarding priority
> There are 2 VNETS assigned
> 
> virsh domiflist s-285-VM
>  Interface   Type     Source      Model    MAC
> --------------------------------------------------------------
>  vnet3       bridge   cloud0      virtio   0e:00:a9:fe:a1:14
>  vnet4       bridge   cloudbr1    virtio   1e:00:3f:00:03:33
>  vnet5       bridge   brvx-1000   virtio   1e:00:b8:00:0c:fe
> 
>  virsh domifaddr s-285-VM
>  Name       MAC address          Protocol     Address
> -------------------------------------------------------------------------------
> ----- Nothing here -----
> 
> 
> the command arp -n, does not showPublic IPaddresses, just the private IP 
> addresses and the <control-ip-169.x.x.x>
> 
> sh bgp evpn route vni 1000
> BGP table version is 1, local router ID is 10.1.1.14
> Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
> Origin codes: i - IGP, e - EGP, ? - incomplete
> EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
> EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
> EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
> EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
> EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
> 
>    Network          Next Hop            Metric LocPrf Weight Path
>  *>  [3]:[0]:[32]:[10.1.1.14]
>                     10.1.1.14(hv1)
>                                                        32768 i
>                     ET:8 RT:5641:1000
> 
> Displayed 1 prefixes (1 paths)
> 
> 
> 
> Please Let me know if I should see the Public IP addresses advertised for 
> this VMs or any IP.
> 
> 
> Tata Y.
> 
> 
> 
> 
>> On Apr 1, 2025, at 9:35 AM, Wido den Hollander <w...@widodh.nl> wrote:
>> 
>> 
>> 
>> Op 01-04-2025 om 13:16 schreef Chi vediamo:
>>> Hello Wido,
>>> Sorry I meant I only have cloudbr1, there is no cloudbr0  on the hypervisor
>>> On the management server on the UI interface
>>> when the VNI/VXLAN for Public is requested then I should enter vxlan://777 
>>> in your case ?
>> 
>> Yes, for the first network you create for example.
>> 
>>> And for the VNI/VXLAN Range for guest leave it blank ? or ALso use a full 
>>> range including VXLAN200 in your case vxlan://1  through vxlan://1000 ?
>> 
>> I don't think we ever used a VNI range in our setups. We just manually add 
>> the VNI for every network we create.
>> 
>> Wido
>> 
>>> Tata Y.
>>>> On Apr 1, 2025, at 7:06 AM, Wido den Hollander <w...@widodh.nl> wrote:
>>>> 
>>>> 
>>>> 
>>>> Op 01-04-2025 om 12:16 schreef Chi vediamo:
>>>>> Thank you Wido for jumping in, as always
>>>>> So far I only Have cloudbr1 as several VXLAN e-mail trails you indicated.
>>>>> Then I need to create the cloudbr0 with another VXLAN in addition to 
>>>>> cloudbr1 ? cloudbr1 should have the management per the video you posted 
>>>>> right?
>>>> 
>>>> In our case we use a static VNI underneath cloudbr1 where it communicates 
>>>> with the other hosts, the mgmt server and the secondary storage (NFS). 
>>>> This is a /22 IPv4 we have allocated for that VNI.
>>>> 
>>>> vxlan200 device which we created using networkd and is a slave of cloudbr1
>>>> 
>>>> cloudbr0 is never used as the VXLAN script creates brvx-XXX devices on the 
>>>> fly.
>>>> 
>>>>> Hope you recall, should I include in the VNI range the initial VNI/VXLANs 
>>>>> i am using for cloudbr1 and cloudbr0
>>>> 
>>>> Not that I'm aware of. We simply specify the broadcast domain when 
>>>> creating a new network: vxlan://777 for example to use VNI 777 for that 
>>>> network.
>>>> 
>>>> This will result in brvx-777 being created on a host when needed there and 
>>>> the VM being attached to that bridge.
>>>> 
>>>> Wido
>>>> 
>>>>> Thank you so much.
>>>>> Tata Y.
>>>>>> On Apr 1, 2025, at 6:00 AM, Wido den Hollander <w...@widodh.nl> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Op 29-03-2025 om 15:42 schreef Chi vediamo:
>>>>>>> Hello Wei,
>>>>>>> The VXLAN with EPVN is poorly documented, then I need all tthe guidance 
>>>>>>> as possible from the community.
>>>>>> 
>>>>>> There is no silver bullet here as each EVPN+VXLAN setup will differ from 
>>>>>> the other :-)
>>>>>> 
>>>>>>> Wido, please kindly allow me to use some of your multiple answers to 
>>>>>>> several members of the community as reference. As there is really not 
>>>>>>> much info.
>>>>>>>  * Wei what you suggesting I did try that, already, I did try several
>>>>>>>    options so far. The cloudstack-Agent keeps restarting itself, I copy
>>>>>>>    the logs in the e-mail.
>>>>>>>  *
>>>>>>>  * Saw countless videos, and read the manual, the Manual only states in
>>>>>>>    summary "VXLAN with EVPN is complex", there is nothing else.
>>>>>>>  *
>>>>>>>  * Manual states use and modify "modifyvxlan-evpn.sh" which does not
>>>>>>>    exists. Using the one form github. modifyvxlan.sh
>>>>>>>  *
>>>>>> 
>>>>>> Yes, the Github one is good. It will be in packages from 4.21
>>>>>> 
>>>>>>>   *  The manual indicates: showing a picture of Management,public and
>>>>>>>    storage on vlans. and indicates - The only thing you have to do is
>>>>>>>    configure the physical Network to use VXLAN as the isolation method
>>>>>>>    for Guest Network - We do not have VLANS at all.
>>>>>>>  *
>>>>>> 
>>>>>> Don't worry about different networks, VXLAN creates it's own bridges on 
>>>>>> the fly.
>>>>>> 
>>>>>>>  * Per one of the e-mails iIread long time ago from Wido to another
>>>>>>>    member of the community: we do left the UI settings by default. That
>>>>>>>    didn't work. And we start experimenting, and none seems to be 
>>>>>>> working.
>>>>>> 
>>>>>> It has been a long time since I've set it up. But you should just have 
>>>>>> cloudbr0 and cloudbr1, nothing else.
>>>>>> 
>>>>>>> Per Wido, we only need cloudbr1, with no VLANs.
>>>>>>> I only need to know on the cludstack UI, What do we point to the 
>>>>>>> Cloudbr1 My understanding is This is going to be used for Migrations ad 
>>>>>>> that.
>>>>>>> 4.20.0 does not allow me to add Management alone, nor the Guest Alone, 
>>>>>>> when creating the first zone.
>>>>>>> I need Public and Management at least. Right? Or please indicate which 
>>>>>>> ones. I try to add only the Guest one, it also fails
>>>>>>> Additionally, seems like the Cloudstack is not smart enough to see what 
>>>>>>> VNI VXLANs are already configured, then I have to provide a vni range 
>>>>>>> for the Guest.
>>>>>>> Although the modifyvxlan.sh script should take care of it - that is 
>>>>>>> based on reading the script code.
>>>>>>> So far we do have VXLAN-EVPN reachability and working.
>>>>>>> I think we are almost there:
>>>>>>>  * I try adding Guest Only during the zone creation, we do have
>>>>>>>    management, public and storage on VXLANS, all are reacheable by
>>>>>>>    routing, and the cloudstack-agent server reach them and have
>>>>>>>    cloudbr1 created.
>>>>>>>      o Should I have only Managment and Guest on the cloudbr1?
>>>>>>>  * Need help Guidance with The cloudstack-agent keeps Bouncing comes up
>>>>>>>    and immediately down, and even reinstalled removing all files. Still
>>>>>>>    the same.
>>>>>>>  * Any example of the initial agent.properties: Or do not touch the
>>>>>>>    agent.properties
>>>>>> 
>>>>>> Only set the network devices as you've done below. The guid should match 
>>>>>> what the hypervisor thinks the UUID of this Agent is. Again, long time 
>>>>>> ago since I've set this up for the first time.
>>>>>> 
>>>>>> Wido
>>>>>> 
>>>>>>> My agent.properties I have modified specifically 3 parameters:
>>>>>>> host=<MAMGT-IP1>,<MGMT-IP2>
>>>>>>> guest.cpu.mode=host-model
>>>>>>> private.network.device=cloudbr1
>>>>>>> Thank you for your guidance.
>>>>>>> Thank you
>>>>>>> Tata Y.
>>>>>>>> On Mar 28, 2025, at 1:43 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I think you do not need to change agent.properties
>>>>>>>> Instead, you need to set the network traffic label of each traffic 
>>>>>>>> type, for example cloudbr1.
>>>>>>>> 
>>>>>>>> By default, management traffic uses cloudbr0, public and guest traffic 
>>>>>>>> use cloudbr1, which might not be what you use.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -Wei
>>>>>>>> 
>>>>>>>> On Friday, March 28, 2025, Chi vediamo <tatay...@gmail.com 
>>>>>>>> <mailto:tatay...@gmail.com>> wrote:
>>>>>>>> 
>>>>>>>>    Hello Apache Cloudstack users,
>>>>>>>> 
>>>>>>>> 
>>>>>>>>    After attempted some options, I am recurring to what should work.
>>>>>>>>    As there is no info in the documentation about this deployment.
>>>>>>>> 
>>>>>>>>    I did replaced the agent server with private.network.device=cloudbr1
>>>>>>>> 
>>>>>>>>    I have my VXLAN 1 on Cloudbr1 that reaches management and SS, We
>>>>>>>>    only have cloudbr1 created, No other bridge, and everything is BGP.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>    Then at the UI creating the ZONE I got errors:
>>>>>>>> 
>>>>>>>>    We have had tryed the following Options:
>>>>>>>> 
>>>>>>>>    *Option 1: *No tags
>>>>>>>>    I put only [Physical Network 1] [VXLAN] Management, throws a popup
>>>>>>>>    error " All required traffic types should be added and with
>>>>>>>>    multiple physical networks each traffic type should have a label"
>>>>>>>> 
>>>>>>>>    *Option 2:* No Tags, Throws and Error " All required traffic types
>>>>>>>>    should be added and with multiple physical networks each traffic
>>>>>>>>    type should have a label"
>>>>>>>>    I put :
>>>>>>>>    [Physical Network 1] [VXLAN] Management
>>>>>>>>    [Physical Network 2] [VXLAN] Public
>>>>>>>>    [Physical Network 3] [VXLAN] Guest
>>>>>>>>    [Physical Network 4] [VXLAN] Storage
>>>>>>>> 
>>>>>>>> 
>>>>>>>>    *Option 3: *
>>>>>>>>    I put :
>>>>>>>>    [Physical Network 1] [VXLAN] Management, Public, Guest, it goes to
>>>>>>>>    the next window - which is Great
>>>>>>>>    The the system throws exceptions like:
>>>>>>>>    *Something went wrong; please correct the following:*
>>>>>>>>    *Unable to add the host: Cannot find the server resources at
>>>>>>>>    http://<KVM-IP>*
>>>>>>>> 
>>>>>>>>    sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml
>>>>>>>>    *
>>>>>>>>    *
>>>>>>>>    *Management logs shows:*
>>>>>>>>    Succeeded to import certificate in the keystore for agent on the
>>>>>>>>    KVM host: <ip>. Agent secured and trusted
>>>>>>>> 
>>>>>>>>    *
>>>>>>>>    *
>>>>>>>>    *The Agent Logs:*
>>>>>>>>    **
>>>>>>>>    2025-03-28 16:48:42,550 DEBUG [commons.beanutils.ConvertUtils]
>>>>>>>>    (main:[]) (logid:) Convert string '<IP-MGMT1>,<IP-MGMT2>' to class
>>>>>>>>    'java.lang.String'
>>>>>>>>    2025-03-28 16:48:42,550 DEBUG
>>>>>>>>    [beanutils.converters.StringConverter] (main:[]) (logid:)
>>>>>>>>    Converting 'String' value '<IP-MGMT1>,<IP-MGMT2>' to type 'String'
>>>>>>>>    2025-03-28 16:48:42,551 DEBUG
>>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>>    Property [guid] has empty or null value. Using default value [null].
>>>>>>>>    2025-03-28 16:48:42,552 DEBUG
>>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>>    Property [developer] has empty or null value. Using default value
>>>>>>>>    [false].
>>>>>>>>    2025-03-28 16:48:53,897 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>>    (logid:) Agent started
>>>>>>>>    2025-03-28 16:48:53,902 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>>    (logid:) Implementation Version is 4.20.0.0
>>>>>>>>    2025-03-28 16:48:53,904 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>>    (logid:) agent.properties found at /etc/cloudstack/agent/
>>>>>>>>    agent.properties
>>>>>>>>    2025-03-28 16:48:53,913 DEBUG
>>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>>    Property [host] was altered. Now using the value
>>>>>>>>    [10.23.123.61,10.23.123.62].
>>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:) Setting
>>>>>>>>    default value: false
>>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:)
>>>>>>>>    Converting 'Boolean' value 'false' to type 'Boolean'
>>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:)     No
>>>>>>>>    conversion required, value is already a Boolean
>>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>>    (main:[]) (logid:) Setting default value: 0
>>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>>    (main:[]) (logid:) Converting 'Integer' value '0' to type 'Byte'
>>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>>    (main:[]) (logid:)     Converted to Byte value '0'
>>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)
>>>>>>>>    Setting default value:
>>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)
>>>>>>>>    Converting 'Character' value ' ' to type 'Character'
>>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)        
>>>>>>>> No conversion required, value is already a Character
>>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:) Setting
>>>>>>>>    default value: 0
>>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:)
>>>>>>>>    Converting 'Integer' value '0' to type 'Double'
>>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:)        
>>>>>>>> Converted to Double value '0.0'
>>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:) Setting
>>>>>>>>    default value: 0
>>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:)
>>>>>>>>    Converting 'Integer' value '0' to type 'Float'
>>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:)        
>>>>>>>> Converted to Float value '0.0'
>>>>>>>>    2025-03-28 16:48:53,928 DEBUG
>>>>>>>>    [beanutils.converters.IntegerConverter] (main:[]) (logid:) Setting
>>>>>>>>    default value: 0
>>>>>>>>    *
>>>>>>>>    *
>>>>>>>>    AGENT out :
>>>>>>>>    Unable to find the guid
>>>>>>>>    Unable to find the guid
>>>>>>>>    Unable to find the guid
>>>>>>>>    Unable to find the guid
>>>>>>>>    Unable to find the guid
>>>>>>>> 
>>>>>>>>    There is nothing there
>>>>>>>> 
>>>>>>>>    And the agent come up, and immediately goes down!!
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>    Tata Y.
>>>>>>>> 
>>>>>> 
>>>> 
>> 
> 

Reply via email to