If I use a single Management server it works:

Question with the IP addresses assigned:

the initial VMs are running:
v-154-VM   <public-IP-3>   <Private-IP>  <control-ip-169.x.x.x>
s-285-VM  <public-IP-2>   <Private-IP>  <control-ip-169.x.x.y>

I do not see the Public IP anywhere, not even advertised on vni1000 or any 
other, the public VNI is there was created by the system, but nothing is there 
- No IP addresses advertised

I used VLAN/VNI  vxlan://1000 

vxlan1000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9050 master brvx-1000 state 
forwarding priority
There are 2 VNETS assigned

virsh domiflist s-285-VM
 Interface   Type     Source      Model    MAC
--------------------------------------------------------------
 vnet3       bridge   cloud0      virtio   0e:00:a9:fe:a1:14
 vnet4       bridge   cloudbr1    virtio   1e:00:3f:00:03:33
 vnet5       bridge   brvx-1000   virtio   1e:00:b8:00:0c:fe

 virsh domifaddr s-285-VM
 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
----- Nothing here -----


the command arp -n, does not showPublic IPaddresses, just the private IP 
addresses and the <control-ip-169.x.x.x>

sh bgp evpn route vni 1000
BGP table version is 1, local router ID is 10.1.1.14
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
 *>  [3]:[0]:[32]:[10.1.1.14]
                    10.1.1.14(hv1)
                                                       32768 i
                    ET:8 RT:5641:1000

Displayed 1 prefixes (1 paths)



Please Let me know if I should see the Public IP addresses advertised for this 
VMs or any IP.


Tata Y.




> On Apr 1, 2025, at 9:35 AM, Wido den Hollander <w...@widodh.nl> wrote:
> 
> 
> 
> Op 01-04-2025 om 13:16 schreef Chi vediamo:
>> Hello Wido,
>> Sorry I meant I only have cloudbr1, there is no cloudbr0  on the hypervisor
>> On the management server on the UI interface
>> when the VNI/VXLAN for Public is requested then I should enter vxlan://777 
>> in your case ?
> 
> Yes, for the first network you create for example.
> 
>> And for the VNI/VXLAN Range for guest leave it blank ? or ALso use a full 
>> range including VXLAN200 in your case vxlan://1  through vxlan://1000 ?
> 
> I don't think we ever used a VNI range in our setups. We just manually add 
> the VNI for every network we create.
> 
> Wido
> 
>> Tata Y.
>>> On Apr 1, 2025, at 7:06 AM, Wido den Hollander <w...@widodh.nl> wrote:
>>> 
>>> 
>>> 
>>> Op 01-04-2025 om 12:16 schreef Chi vediamo:
>>>> Thank you Wido for jumping in, as always
>>>> So far I only Have cloudbr1 as several VXLAN e-mail trails you indicated.
>>>> Then I need to create the cloudbr0 with another VXLAN in addition to 
>>>> cloudbr1 ? cloudbr1 should have the management per the video you posted 
>>>> right?
>>> 
>>> In our case we use a static VNI underneath cloudbr1 where it communicates 
>>> with the other hosts, the mgmt server and the secondary storage (NFS). This 
>>> is a /22 IPv4 we have allocated for that VNI.
>>> 
>>> vxlan200 device which we created using networkd and is a slave of cloudbr1
>>> 
>>> cloudbr0 is never used as the VXLAN script creates brvx-XXX devices on the 
>>> fly.
>>> 
>>>> Hope you recall, should I include in the VNI range the initial VNI/VXLANs 
>>>> i am using for cloudbr1 and cloudbr0
>>> 
>>> Not that I'm aware of. We simply specify the broadcast domain when creating 
>>> a new network: vxlan://777 for example to use VNI 777 for that network.
>>> 
>>> This will result in brvx-777 being created on a host when needed there and 
>>> the VM being attached to that bridge.
>>> 
>>> Wido
>>> 
>>>> Thank you so much.
>>>> Tata Y.
>>>>> On Apr 1, 2025, at 6:00 AM, Wido den Hollander <w...@widodh.nl> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> Op 29-03-2025 om 15:42 schreef Chi vediamo:
>>>>>> Hello Wei,
>>>>>> The VXLAN with EPVN is poorly documented, then I need all tthe guidance 
>>>>>> as possible from the community.
>>>>> 
>>>>> There is no silver bullet here as each EVPN+VXLAN setup will differ from 
>>>>> the other :-)
>>>>> 
>>>>>> Wido, please kindly allow me to use some of your multiple answers to 
>>>>>> several members of the community as reference. As there is really not 
>>>>>> much info.
>>>>>>  * Wei what you suggesting I did try that, already, I did try several
>>>>>>    options so far. The cloudstack-Agent keeps restarting itself, I copy
>>>>>>    the logs in the e-mail.
>>>>>>  *
>>>>>>  * Saw countless videos, and read the manual, the Manual only states in
>>>>>>    summary "VXLAN with EVPN is complex", there is nothing else.
>>>>>>  *
>>>>>>  * Manual states use and modify "modifyvxlan-evpn.sh" which does not
>>>>>>    exists. Using the one form github. modifyvxlan.sh
>>>>>>  *
>>>>> 
>>>>> Yes, the Github one is good. It will be in packages from 4.21
>>>>> 
>>>>>>   *  The manual indicates: showing a picture of Management,public and
>>>>>>    storage on vlans. and indicates - The only thing you have to do is
>>>>>>    configure the physical Network to use VXLAN as the isolation method
>>>>>>    for Guest Network - We do not have VLANS at all.
>>>>>>  *
>>>>> 
>>>>> Don't worry about different networks, VXLAN creates it's own bridges on 
>>>>> the fly.
>>>>> 
>>>>>>  * Per one of the e-mails iIread long time ago from Wido to another
>>>>>>    member of the community: we do left the UI settings by default. That
>>>>>>    didn't work. And we start experimenting, and none seems to be working.
>>>>> 
>>>>> It has been a long time since I've set it up. But you should just have 
>>>>> cloudbr0 and cloudbr1, nothing else.
>>>>> 
>>>>>> Per Wido, we only need cloudbr1, with no VLANs.
>>>>>> I only need to know on the cludstack UI, What do we point to the 
>>>>>> Cloudbr1 My understanding is This is going to be used for Migrations ad 
>>>>>> that.
>>>>>> 4.20.0 does not allow me to add Management alone, nor the Guest Alone, 
>>>>>> when creating the first zone.
>>>>>> I need Public and Management at least. Right? Or please indicate which 
>>>>>> ones. I try to add only the Guest one, it also fails
>>>>>> Additionally, seems like the Cloudstack is not smart enough to see what 
>>>>>> VNI VXLANs are already configured, then I have to provide a vni range 
>>>>>> for the Guest.
>>>>>> Although the modifyvxlan.sh script should take care of it - that is 
>>>>>> based on reading the script code.
>>>>>> So far we do have VXLAN-EVPN reachability and working.
>>>>>> I think we are almost there:
>>>>>>  * I try adding Guest Only during the zone creation, we do have
>>>>>>    management, public and storage on VXLANS, all are reacheable by
>>>>>>    routing, and the cloudstack-agent server reach them and have
>>>>>>    cloudbr1 created.
>>>>>>      o Should I have only Managment and Guest on the cloudbr1?
>>>>>>  * Need help Guidance with The cloudstack-agent keeps Bouncing comes up
>>>>>>    and immediately down, and even reinstalled removing all files. Still
>>>>>>    the same.
>>>>>>  * Any example of the initial agent.properties: Or do not touch the
>>>>>>    agent.properties
>>>>> 
>>>>> Only set the network devices as you've done below. The guid should match 
>>>>> what the hypervisor thinks the UUID of this Agent is. Again, long time 
>>>>> ago since I've set this up for the first time.
>>>>> 
>>>>> Wido
>>>>> 
>>>>>> My agent.properties I have modified specifically 3 parameters:
>>>>>> host=<MAMGT-IP1>,<MGMT-IP2>
>>>>>> guest.cpu.mode=host-model
>>>>>> private.network.device=cloudbr1
>>>>>> Thank you for your guidance.
>>>>>> Thank you
>>>>>> Tata Y.
>>>>>>> On Mar 28, 2025, at 1:43 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
>>>>>>> 
>>>>>>> I think you do not need to change agent.properties
>>>>>>> Instead, you need to set the network traffic label of each traffic 
>>>>>>> type, for example cloudbr1.
>>>>>>> 
>>>>>>> By default, management traffic uses cloudbr0, public and guest traffic 
>>>>>>> use cloudbr1, which might not be what you use.
>>>>>>> 
>>>>>>> 
>>>>>>> -Wei
>>>>>>> 
>>>>>>> On Friday, March 28, 2025, Chi vediamo <tatay...@gmail.com 
>>>>>>> <mailto:tatay...@gmail.com>> wrote:
>>>>>>> 
>>>>>>>    Hello Apache Cloudstack users,
>>>>>>> 
>>>>>>> 
>>>>>>>    After attempted some options, I am recurring to what should work.
>>>>>>>    As there is no info in the documentation about this deployment.
>>>>>>> 
>>>>>>>    I did replaced the agent server with private.network.device=cloudbr1
>>>>>>> 
>>>>>>>    I have my VXLAN 1 on Cloudbr1 that reaches management and SS, We
>>>>>>>    only have cloudbr1 created, No other bridge, and everything is BGP.
>>>>>>> 
>>>>>>> 
>>>>>>>    Then at the UI creating the ZONE I got errors:
>>>>>>> 
>>>>>>>    We have had tryed the following Options:
>>>>>>> 
>>>>>>>    *Option 1: *No tags
>>>>>>>    I put only [Physical Network 1] [VXLAN] Management, throws a popup
>>>>>>>    error " All required traffic types should be added and with
>>>>>>>    multiple physical networks each traffic type should have a label"
>>>>>>> 
>>>>>>>    *Option 2:* No Tags, Throws and Error " All required traffic types
>>>>>>>    should be added and with multiple physical networks each traffic
>>>>>>>    type should have a label"
>>>>>>>    I put :
>>>>>>>    [Physical Network 1] [VXLAN] Management
>>>>>>>    [Physical Network 2] [VXLAN] Public
>>>>>>>    [Physical Network 3] [VXLAN] Guest
>>>>>>>    [Physical Network 4] [VXLAN] Storage
>>>>>>> 
>>>>>>> 
>>>>>>>    *Option 3: *
>>>>>>>    I put :
>>>>>>>    [Physical Network 1] [VXLAN] Management, Public, Guest, it goes to
>>>>>>>    the next window - which is Great
>>>>>>>    The the system throws exceptions like:
>>>>>>>    *Something went wrong; please correct the following:*
>>>>>>>    *Unable to add the host: Cannot find the server resources at
>>>>>>>    http://<KVM-IP>*
>>>>>>> 
>>>>>>>    sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml
>>>>>>>    *
>>>>>>>    *
>>>>>>>    *Management logs shows:*
>>>>>>>    Succeeded to import certificate in the keystore for agent on the
>>>>>>>    KVM host: <ip>. Agent secured and trusted
>>>>>>> 
>>>>>>>    *
>>>>>>>    *
>>>>>>>    *The Agent Logs:*
>>>>>>>    **
>>>>>>>    2025-03-28 16:48:42,550 DEBUG [commons.beanutils.ConvertUtils]
>>>>>>>    (main:[]) (logid:) Convert string '<IP-MGMT1>,<IP-MGMT2>' to class
>>>>>>>    'java.lang.String'
>>>>>>>    2025-03-28 16:48:42,550 DEBUG
>>>>>>>    [beanutils.converters.StringConverter] (main:[]) (logid:)
>>>>>>>    Converting 'String' value '<IP-MGMT1>,<IP-MGMT2>' to type 'String'
>>>>>>>    2025-03-28 16:48:42,551 DEBUG
>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>    Property [guid] has empty or null value. Using default value [null].
>>>>>>>    2025-03-28 16:48:42,552 DEBUG
>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>    Property [developer] has empty or null value. Using default value
>>>>>>>    [false].
>>>>>>>    2025-03-28 16:48:53,897 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>    (logid:) Agent started
>>>>>>>    2025-03-28 16:48:53,902 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>    (logid:) Implementation Version is 4.20.0.0
>>>>>>>    2025-03-28 16:48:53,904 INFO  [cloud.agent.AgentShell] (main:[])
>>>>>>>    (logid:) agent.properties found at /etc/cloudstack/agent/
>>>>>>>    agent.properties
>>>>>>>    2025-03-28 16:48:53,913 DEBUG
>>>>>>>    [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:)
>>>>>>>    Property [host] was altered. Now using the value
>>>>>>>    [10.23.123.61,10.23.123.62].
>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:) Setting
>>>>>>>    default value: false
>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:)
>>>>>>>    Converting 'Boolean' value 'false' to type 'Boolean'
>>>>>>>    2025-03-28 16:48:53,923 DEBUG
>>>>>>>    [beanutils.converters.BooleanConverter] (main:[]) (logid:)     No
>>>>>>>    conversion required, value is already a Boolean
>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>    (main:[]) (logid:) Setting default value: 0
>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>    (main:[]) (logid:) Converting 'Integer' value '0' to type 'Byte'
>>>>>>>    2025-03-28 16:48:53,925 DEBUG [beanutils.converters.ByteConverter]
>>>>>>>    (main:[]) (logid:)     Converted to Byte value '0'
>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)
>>>>>>>    Setting default value:
>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)
>>>>>>>    Converting 'Character' value ' ' to type 'Character'
>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>    [beanutils.converters.CharacterConverter] (main:[]) (logid:)        
>>>>>>> No conversion required, value is already a Character
>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:) Setting
>>>>>>>    default value: 0
>>>>>>>    2025-03-28 16:48:53,926 DEBUG
>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:)
>>>>>>>    Converting 'Integer' value '0' to type 'Double'
>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>    [beanutils.converters.DoubleConverter] (main:[]) (logid:)        
>>>>>>> Converted to Double value '0.0'
>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:) Setting
>>>>>>>    default value: 0
>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:)
>>>>>>>    Converting 'Integer' value '0' to type 'Float'
>>>>>>>    2025-03-28 16:48:53,927 DEBUG
>>>>>>>    [beanutils.converters.FloatConverter] (main:[]) (logid:)        
>>>>>>> Converted to Float value '0.0'
>>>>>>>    2025-03-28 16:48:53,928 DEBUG
>>>>>>>    [beanutils.converters.IntegerConverter] (main:[]) (logid:) Setting
>>>>>>>    default value: 0
>>>>>>>    *
>>>>>>>    *
>>>>>>>    AGENT out :
>>>>>>>    Unable to find the guid
>>>>>>>    Unable to find the guid
>>>>>>>    Unable to find the guid
>>>>>>>    Unable to find the guid
>>>>>>>    Unable to find the guid
>>>>>>> 
>>>>>>>    There is nothing there
>>>>>>> 
>>>>>>>    And the agent come up, and immediately goes down!!
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>    Tata Y.
>>>>>>> 
>>>>> 
>>> 
> 

Reply via email to